Data from: Novel color via stimulation of individual photoreceptors at population scale
Abstract
We introduce a new principle, Oz, for displaying color imagery: directly controlling the human eye’s photoreceptor activity via cell-by-cell light delivery. Theoretically, novel colors are possible through bypassing the constraints set by the cone spectral sensitivities and activating M cone cells exclusively. In practice, we confirm a partial expansion of colorspace towards that theoretical ideal. Attempting to activate M cones exclusively is shown to elicit a novel color beyond the natural human gamut, formally measured with color matching by human subjects. They describe the color as blue-green of unprecedented saturation. Further experiments show that subjects perceive Oz colors in image and video form. The prototype targets laser microdoses to thousands of spectrally classified cones under fixational eye motion. These results are proof-of-principle for programmable control over individual photoreceptors at population scale.
https://doi.org/10.5061/dryad.pc866t206
This dataset contains:
- LMS values for each color match, and for averages of color matches.
- Individual responses for each alternative forced-choice (AFC) trial, and aggregate accuracies.
- Code for reproducing the data plots presented in the manuscript.
Also included are the data needed to compute the LMS values from the subject’s raw responses during color matching trials.
- Raw 0–255 RGB bitlevels chosen by subjects.
- Raw intensities and center wavelengths chosen by subjects in matches which use the tunable wavelength source.
- Spectral power distributions for the RGB projector primaries.
- Look-up table used to convert 0–255 RGB bitlevels to linear intensities for each projector primary.
- Spectral power distributions for the subjects’ chosen tunable-wavelength source settings.
Other supporting data and code provided here:
- Cone classification maps for each subject.
- The target stimuli used for the AFC experiments.
- Microdose intensities for each cone class used during color-matching experiments.
- Spectral power distributions for a sampling of tunable-wavelength source settings used to find the edge of that source’s gamut. These measurements have an unknown scale relative to the other SPD’s.
- The 543nm and 488nm visible-wavelength stimulating beams. These measurements have an unknown scale relative to the other SPD’s.
- Code for estimating the prototype Oz display’s fractional leak given the color match data.
- Code for estimating theoretical fractional leaks under an ideal Oz display.
Data
afc/afc_aggregate.xlsx
Spreadsheet containing the total number of correct/incorrect responses for each subject under each stimulus condition during the AFC experiments. This is processed from afc/afc_raw.xlsx.
Each row is a single subject’s performance under either the line or disk condition.
Columns:
- subject_id: Subject ID.
- num_test_correct: Total number of correct responses under the test condition (no jitter).
- total_test_trials: Total number of trials (total correct + total incorrect) under the test condition.
- test_accuracy_percent: Test accuracy as a percent.
- num_control_correct / total_control_trials / control_accuracy_percent: Analogously, the performance under the control condition (with jitter).
- media_type: “disk” if the rotating disk was used (2-AFC), or “line” if the stationary line was used (4-AFC).
afc/afc_raw.xlsx
Spreadsheet containing each individual response for each trial in the AFC experiments.
Each row is a single trial, i.e. one subject response.
Columns:
- afc_id: ID number of this response.
- subject_id: Subject ID.
- media_filename: Which target stimulus was used. See
afc/media. - response: Subject response, one of “u”, “d”, “l”, “r”.
- uses_jitter: Whether jitter was used, either TRUE or FALSE.
- correct: Whether the response was correct, either TRUE or FALSE.
- media_type: “disk” if the rotating disk was used (2-AFC), or “line” if the stationary line was used (4-AFC).
- microdose_foreground_L/M/S: The intensity of microdoses sent to each of the three LMS cone types, for the foreground (the dot or the line).
- microdose_background_L/M/S: Analogously, but for the background.
afc/media
Folder containing false-color visualizations of the target stimuli used during the AFC experiments.
color_matching/color_matches_LMS.xlsx
Spreadsheet containing the LMS values submitted by subjects during the color matching experiments. This is processed from color_matching/color_matches_raw.xlsx.
Each row is a single color match under the given conditions.
Columns:
- color_match_id: ID number of this response.
- subject_id: Subject ID.
- stim_wavelength: Wavelength (nm) for the visible-wavelength stimulating laser. Either 543 or 488.
- microdose_pattern_type: Indicates the relative amount of light sent to the three cone classes. One of “L-only”, “M-only”, “S-only”, “M-dominant”, or “L-dominant”. These correspond to the 5 conditions given (in the same order as listed here) in Figure 3D in the manuscript.
- matching_field_source: Light source used by subjects for the matching field. Either “projector” or “tunable”.
- uses_jitter: Whether jitter was used, either TRUE or FALSE.
- match_L/M/S: The LMS value of the subject’s match.
color_matching/color_matches_averages_LMS.xlsx
Spreadsheet containing the LMS values for each subject’s average match under each stimulus condition. This is processed from color_matching/color_matches_raw.xlsx.
Each row provides a single subject’s average match under the given stimulation wavelength, microdose pattern type, matching field source, and whether jitter was used.
Columns:
- subject_id: Subject ID.
- stim_wavelength: Wavelength (nm) for the visible-wavelength stimulating laser. Either 543 or 488.
- microdose_pattern_type: Indicates the relative amount of light sent to the three cone classes. One of “L-only”, “M-only”, “S-only”, “M-dominant”, or “L-dominant”. These correspond to the 5 conditions given (in the same order as listed here) in Figure 3D in the manuscript.
- matching_field_source: Light source used by subjects for the matching field. Either “projector” or “tunable”.
- uses_jitter: Whether jitter was used, either TRUE or FALSE.
- average_match_L/M/S: The LMS value of the subject’s average match.
color_matching/color_matches_raw.xlsx
Spreadsheet containing the raw responses of each subject for each match.
Each row is a single color match under the given conditions.
Columns:
- color_match_id: ID number of this response.
- subject_id: Subject ID.
- stim_wavelength: Wavelength (nm) for the visible-wavelength stimulating laser. Either 543 or 488.
- microdose_pattern_type: Indicates the relative amount of light sent to the three cone classes. One of “L-only”, “M-only”, “S-only”, “M-dominant”, or “L-dominant”. These correspond to the 5 conditions given (in the same order as listed here) in Figure 3D in the manuscript.
- matching_field_source: Light source used by subjects for the matching field. Either “projector” or “tunable”.
- uses_jitter: Whether jitter was used, either TRUE or FALSE.
- projector_positive_R/G/B: The subject’s 0–255 RGB bitlevels for the light in the “positive” matching field.
- projector_negative_R/G/B: The subject’s 0–255 RGB bitlevels for the light in the “negative” desaturating white.
- projector_background_R/G/B: The 0–255 RGB bitlevels for the “gray background” described in the supplementary.
- tunable_wavelength: The subject’s chosen center wavelength (nm) for the tunable wavelength source, if applicable.
- tunable_intensity: The subject’s chosen relative intensity for the tunable wavelength source, if applicable.
color_matching/microdose_intensities.xlsx
Spreadsheet containing the 0–1 intensities of the microdoses sent to each cone class.
Each row corresponds to one of the 5 intensity settings used for color matching.
Columns:
- microdose_pattern_type: One of “L-only”, “M-only”, “S-only”, “M-dominant”, or “L-dominant”. These correspond to the 5 conditions given (in the same order as listed here) in Figure 3D in the manuscript.
- microdose_intensity_L/M/S: The 0–1 intensity of the microdose sent to the given L/M/S class.
color_matching/projector_bitlevel_lut.xlsx
Spreadsheet containing the linearizing lookup-table mapping from 0–255 bitlevels for each RGB primary into a 0–1 scalar linear intensity.
Columns:
- bitlevel: The 8-bit 0–255 bitlevel used for a given R/G/B primary.
- linear_R/G/B: The 0–1 scalar intensity of the given R/G/B primary at that bitlevel.
color_matching/projector_primaries_spectra.xlsx
Spreadsheet containing spectral power distributions for each of the RGB primaries.
Columns:
- primary: The given primary, one of “R”, “G”, “B”.
- L/M/S: The Euclidean LMS coordinate for this SPD.
- intensity_[lambda]: Intensity at wavelength lambda (nm).
color_matching/subject_chosen_tunable_spectra.xlsx
Spreadsheet containing spectral power distributions for each tunable wavelength setting chosen by subjects for color matching.
Columns:
- color_match_id: ID number of the color match this selection was used for.
- tunable_wavelength: The subject’s chosen center wavelength (nm) for the tunable wavelength source.
- tunable_intensity: The subject’s chosen relative intensity for the tunable wavelength source.
- L/M/S: The Euclidean LMS coordinate for this SPD.
- intensity_[lambda]: Intensity at wavelength lambda (nm).
other/retina_maps
Cone classification map for each subject. Organized by subject ID, as pairs of png and csv files. The png file is a composite image of the retina in IR, in fundus view. The csv file gives the cone classifications. Each row is x, y, type. Location is in pixels, with (0,0) being the top-left corner of the image, +x pointing right, and +y pointing down. Resolution is 569 pixels per degree.
other/sampled_tunable_spectra.xlsx
Spreadsheet containing additional spectral power distributions for a sampling of center wavelengths for the tunable wavelength source. This is used to find the edge of the tunable wavelength source’s gamut.
Columns:
- tunable_wavelength: A sampled center wavelength (nm) for the tunable wavelength source.
- l/m/s: The lms chromaticity for this SPD.
- intensity_[lambda]: Intensity at wavelength lambda (nm).
other/stimulation_beams_spectra.xlsx
Spreadsheet containing the spectral power distributions for the stimulating laser sources.
Columns:
- stim_wavelength: Wavelength (nm) for the visible-wavelength stimulating laser. Either 543 or 488.
- l/m/s: The lms chromaticity for this SPD.
- intensity_[lambda]: Intensity at wavelength lambda (nm).
Code
Running the code requires the cone fundamentals to be downloaded from CVRL at: http://www.cvrl.org/cones.htm (Units: Energy (linear), Stepsize: 0.1 nm, Format: csv). This should be added as data/linss2_10e_fine.csv.
The files starting with fig_ generate plots used in the corresponding figure.
fig_3_color_match_results.py additionally generates the processed data spreadsheets color_matches_LMS.xlsx and color_matches_averages_LMS.xlsx.
estimate_prototype_fractional_leak.py estimates the prototype Oz display’s fractional leak given the color match data. This leak is highlighted in Figure 2.
estimate_theoretical_ideal_fractional_leaks.py estimates how much fractional leak would occur under ideal conditions in the foveola and at 4 degrees eccentricity. These leaks are highlighted in Figure 2.
human_gamut_chromaticity_visualization.py generates the colorful visualization of the human gamut shown in the background of the lms triangle.