In-sensor multilevel image adjustment for high-clarity contour extraction using adjustable synaptic phototransistors
Data files
Apr 15, 2025 version files 40.52 KB
Abstract
Robotic vision has traditionally relied on high-performance yet resource-intensive computing solutions, which necessitate high-throughput data transmission from vision sensors to remote computing servers, sacrificing energy-efficiency and processing speed. A promising solution is data compaction through contour extraction, visualizing only the outlines of objects while eliminating superfluous backgrounds. Here, we introduce an in-sensor multi-level image adjustment method using adjustable synaptic phototransistors, enabling the capture of well-defined images with optimal brightness and contrast suitable for achieving high-clarity contour extraction. This is enabled by emulating dopamine-mediated neuronal excitability regulation mechanisms. Electrostatic gating effect either facilitates or inhibits time-dependent photocurrent accumulation, adjusting photo-responses to varying lighting conditions. Through excitatory and inhibitory modes, the adjustable synaptic phototransistor enhances visibility of dim and bright regions, respectively, facilitating distinct contour extraction and high-accuracy semantic segmentation. Evaluations using road images demonstrate improvement of both object detection accuracy and Intersection over Union, and significant compression of data volume.
https://doi.org/10.5061/dryad.bcc2fqzpt
Description:
This dataset supports the study titled “In-sensor multilevel image adjustment for high-clarity contour extraction using adjustable synaptic phototransistors.” It includes Python scripts and associated data for preprocessing and analyzing image data using the parameter of synaptic phototransistor and in-sensor multilevel image adjustment method.
Directory and File Structure
- Adaptive_thresholding.py
- Merge_using_logic_operation.py
- Nonlinear_optoelectronic_conversion.py
- Sobel_edge_detection.py
- synaptic_phototransistor_parameter_for_nonlinear_optoelectronic_conversion.xlsx
- [raw_csv_data_folder] Folder with raw CSV image data (not provided here)
- [output_folders] Output folders for processed images (not provided here)
Processing Pipeline and Scripts
0. Synaptic_phototransistor_parameterNonlinear Optoelectronic Conversion
-
File: synaptic_phototransistor_parameter_for_nonlinear_optoelectronic_conversion.xlsx
- Function: This file provides the experimentally measured nonlinear optoelectronic response characteristics of the adjustable synaptic phototransistor under a 250 ms optical input condition. It is essential for device-inspired nonlinear preprocessing of raw image data.
- Input:
- Intensity Level (0-255). The raw grayscale pixel intensity value before conversion. This corresponds to the input light intensity to the phototransistor.
- Output:
- Response Value. The corresponding output electrical response of the synaptic phototransistor to each input intensity level and Vel. This value was experimentally obtained and represents the nonlinear behavior of the device.
These output response values are directly utilized to construct phototransistor_parameter_250ms.csv used in Nonlinear_optoelectronic_conversion.py. This enables mapping raw pixel intensities to the actual electrical characteristics of the phototransistor, reflecting its nonlinear optoelectronic properties.
1. Nonlinear Optoelectronic Conversion
- File: Nonlinear_optoelectronic_conversion.py
- Function: Converts raw pixel data from CSV files using intensity-dependent synaptic phototransistor parameters. Adjusts data for nonlinear optoelectronic conversion using intensity-dependent parameters and normalizes the output for further processing.
- Input:
- CSV files containing raw grayscale image data.
- Parameter file: synaptic_phototransistor_parameter_for_nonlinear_optoelectronic_conversion.xlsx
- Output:
- Processed CSV files saved in the same input folder.
2. Contour Extraction with Sobel Edge Detection
- File: Sobel_edge_detection.py
- Function: Extracts contours from processed CSV data through gradient computation using the Sobel operator, facilitating the evaluation of in-sensor multilevel image adjustment performance.
- Input: Processed CSV files.
- Output: PNG images of detected contours.
3. Contour Extraction with Adaptive Thresholding
- File: Adaptive_thresholding.py
- Function: Extracts contours from processed CSV data by applying adaptive thresholding for binary conversion, enabling the evaluation of in-sensor multilevel image adjustment performance.
- Use Case: Effective for images with uneven lighting.
- Input: Processed CSV files.
- Output: PNG images of extracted contours.
4. Merging Contours Using Logical Operations
- File: Merge_using_logic_operation.py
- Function: Merges contour images (e.g., from excitatory/inhibitory modes) using logical AND/OR operations.
- Input:
- Two sets of processed contour data (from separate modes).
- Output: Merged CSV files with refined contour information.
Example Usage Workflow
1. Preprocess:
Run Nonlinear_optoelectronic_conversion.py with input CSV folder
2. Contour extraction (choose one method):
- Run Sobel_edge_detection.py
OR
- Run Adaptive_thresholding.py
3. Merge results:
Run Merge_using_logic_operation.py to merge two contour images respectively obtained from the excitatory and inhibitory modes of the multilevel image adjustment process, using logical operations (AND/OR), to generate a refined contour image.
Variables Description
1. Nonlinear_optoelectronic_conversion.py
| Variable | Description |
|---|---|
| csv_folder_path | Directory path where raw CSV image files are stored. Each file contains grayscale pixel intensity values (0-255). |
| conversion_path | File path of synaptic phototransistor parameter CSV file. |
| conversion_data | Data loaded from conversion_path containing phototransistor response parameters. |
| conversion_max | Maximum output value from conversion_data used for normalization. |
| post_optical_data | Converted and normalized 2D pixel data array (values 0-255). |
2. Sobel_edge_detection.py
| Variable | Description |
|---|---|
| csv_folder_path | Directory path of processed CSV image files (output of nonlinear conversion). |
| output_folder_path | Directory path where edge-detected PNG images are saved. |
| image | 2D numpy array loaded from CSV representing grayscale image. |
| sobelX, sobelY | Gradient values in x and y directions calculated by Sobel operator. |
| sobel | Gradient magnitude calculated from sobelX and sobelY, representing edge intensity. |
3. Adaptive_thresholding.py
| Variable | Description |
|---|---|
| csv_folder_path | Directory path of processed CSV image files (output of nonlinear conversion). |
| output_folder_path | Directory path where binary contour PNG images are saved. |
| image | 2D numpy array loaded from CSV representing grayscale image. |
| adaptive_threshold | Binary image obtained using adaptive thresholding (pixel values: 0 or 255). |
4. Merge_using_logic_operation.py
| Variable | Description |
|---|---|
| folder1_path | Directory path of contour images from excitatory mode (processed CSV). |
| folder2_path | Directory path of contour images from inhibitory mode (processed CSV). |
| output_folder_path | Directory path where merged CSV files are saved. |
| data1, data2 | 2D arrays loaded from folder1_path and folder2_path for merging. |
| result | Merged 2D array obtained using logical AND (or OR) operation between data1 and data2. |
Citation is encouraged when using this dataset. Please refer to the provided DOI.
Notes
- Input CSV files must be formatted with grayscale pixel intensity values (0–255).
- The directories [raw_csv_data_folder] and [output_folders] are example placeholders. Users should create these folders as needed and place their own CSV-formatted grayscale image data in the input folder to run the scripts.
- File paths must be set manually within each script.
- Logical operation in the merge script (AND, OR) can be configured based on analysis needs.