https://doi.org/10.5061/dryad.sxksn038s

This dataset contains the raw data for the Berkeley Single Cell Computational Microscopy Dataset. The data is compressed and chunked to facilitate downloading. The easiest way to download and use it is through the bsccm python package. The code for this package can be found at https://github.com/Waller-Lab/BSCCM/blob/main/Getting_started.ipynb and is archived with a DOI at https://zenodo.org/doi/10.5281/zenodo.10392182

Loading data&

The Getting Started jupyter notebook shows the full documentation for how to use this dataset, including installation, downloading, image/metadata querying, and more. Here we reproduce the first few steps of the notebook:

• First, install the bsccm python package with pip install bsccm

• Then download the data:

from bsccm import download_dataset
dataset_path = download_dataset('/path/to/download', tiny=True)
print('Downloaded dataset to ' + dataset_path)

• Then open an image

dataset = BSCCM
(dataset_path)
valid_indices = dataset.get_indices()
an_index = valid_indices[0]
image = dataset.read_image(an_index, channel='DPC_Left')

Data organization

After downloading and decompressing the data as described in the Getting_Started notebook, the dataset will be organized as follows. This section describes how the data is organized: which files contain which data, what metadata is available, etc. We note that a full understanding of these details is not necessary for using the dataset, as the Python package we provide abstracts away many of these implementation details.

File structure and organization

All image data are stored in Zarr datasets using Blosc/zstd compression. Tabular metadata (i.e., per-cell metadata) are stored in .csv files. Global metadata, which contains information that is not specific to individual cells, but rather pertains to the whole dataset is stored in text files in Javascript Object Notation (JSON) format.

Each top-level BSCCM (regular, coherent, tiny, or coherent-tiny) contains (up to) 5 items:

• BSCCM_images.zarr: A Zarr dataset containing all the images of cells
• BSCCM_backgrounds.zarr: A Zarr dataset containing the background intensity over the full field of view, for each LED array illumination pattern.
• BSCCM_global_metadata.json: A text file containing metadata about the full dataset (pixel size, wavelength, channel names, etc.) in JSON format
• BSCCM_index.csv: A comma-separated value (CSV) file containing metadata specific to each cell in the dataset
• BSCCM_surface_markers.csv: A comma-separated value (CSV) file containing information about the surface protein marker levels of each cell, along with many measurements derived from the fluorescence images and intermediate values used in computing these levels

BSCCM_images.zarr

Zarr datasets contain a hierarchy of directories. For the BSCCM_images.zarr file, this has the following structure:

+-- antibodies_CD16
|    +-- batch_0
|    |    +-- slide_replicate_0
|    |        +-- dpc
|    |            +-- cell_0
|    |            +-- cell_1
|    |            ...
|    |        +-- fluor
|    |            ...
|    |        +-- led_array
|    |            ...
|    |        +-- histology
|    ...
+-- antibodies_CD45

The outermost directory contains "antibodies_" followed by the name of the antibody used to stain the cells, or "unstained"/"all" for the no-antibody and all-antibody conditions, respectively.

BSCCM_backgrounds.zarr

This file contains the background images for each channel across the full field of view (2056x2056 pixels). The top-level directory contains the channel name. The structure is as shown below.

+-- Brightfield
|   +-- 5\_percentile
|   +-- 10\_percentile
|   +-- 20\_percentile
|   +-- 40\_percentile
|   +-- 50\_percentile
+-- DF\_50

BSCCM_global_metadata.json

This file contains metadata specific to the full dataset, including names of channels, collection settings like exposure, and useful information for calibration like wavelength, objective NA, etc. It is a text file with JSON structure.

{
"led_array": 
    {
    "image_shape": [128, 128],
    "channel_names": ["Brightfield", ... "LED119"], 
    "channel_indices": {"Brightfield": 0, ... "LED119": 200}, 
    "exposure_ms": {"Brightfield": 8, ... "LED119": 200},
    "camera": {
        "offset": 30,
        "gain_db": 4,
        "quantum_efficiency": 0.68
        },
    "wavelength_nm": 515, 
    "pixel_size_um": 0.166, 
    "objective": {"NA": 0.5, "magnification": 20}}, 
"fluorescence": 
    {
    "image_shape": ...
    ...

BSCCM_index.csv

This contains per-cell metadata in a single, large CSV file with one row per each cell. It has the following columns:

"global_index": An integer uniquely identifying the cell
"position_in_fov_y_pix"/"position_in_fov_x_pix":Location of the cell center within the image field of view
"detection_radius": The radius reported by the blob finding algorithm that initially located the cell, which gives a rough estimate of its size
"has_matched_histology_cell": Whether or not the cell has a matching cell in histology contrast
"fov_center_x"/"fov_center_y"/"fov_center_z": the microscope stage coordinates of the field of view from which the cell was drawn
"batch": the index of the cell isolation experiment the cells were drawn from (either 0 or 1)
"antibodies": the name or the single antibody used to stain the cells, or 'all' or 'unstained' if every antibody or no antibodies were used
"imaging_date": the date the slide of cells was imaged
"data_path": the path to the image data with the BSCCM\_images.zarr file
"slide_replicate": the index of the slide replicate within the same antibody/batch conditions (either 0 or 1)

BSCCM_surface_markers.csv

This contains per-cell calculations about fluorescence surface marker levels. It is entirely derived from the fluorescence imaging data. It contains metadata in a single, large CSV file with one row per each cell.

"global_index": An integer uniquely identifying the cell
// raw measurements derived from fluorescence images
"Fluor_426-446_total_raw" //Raw foreground fluorescence in 426-446nm channel
"Fluor_500-550_total_raw" //Raw background fluorescence in 500-550nm channel
...
"Fluor_426-446_background" //Raw background fluorescence in 426-446nm channel
...
(Many more intermediate calculations)
...
// protein levels estimates from unmixing procedure
"CD45_single_antibody_model_unmixed" // CD45 protein levels using 2 spectrum unmixing model
"CD123_single_antibody_model_unmixed"
...
"CD123/HLA-DR/CD14_full_model_unmixed" // Combined CD123/HLA-DR/CD14 protein levels using 4 spectrum unmixng model
...