Berkeley Single-Cell Computational Microscopy (BSCCM) dataset

Pinkard, Henry1 ; Waller, Laura1

Published Feb 08, 2024 on Dryad. https://doi.org/10.5061/dryad.sxksn038s

Abstract

Computational microscopy, in which hardware and algorithms of an imaging system are jointly designed, shows promise for making imaging systems that cost less, perform more robustly, and collect new types of information. Often, the performance of computational imaging systems, especially those that incorporate machine learning, is sample-dependent. Thus, standardized datasets are an essential tool for comparing the performance of different approaches. Here, we introduce the Berkeley Single Cell Computational Microscopy (BSCCM) dataset, which contains over 400,000 images of individual white blood cells. The dataset contains images captured with multiple illumination patterns on an LED array microscope and fluorescent measurements of the abundance of surface proteins that mark different cell types. We hope this dataset will provide a valuable resource for the development and testing of new algorithms in computational microscopy and computer vision with practical biomedical applications.

README: Berkeley Single-Cell Computational Microscopy (BSCCM) Dataset

https://doi.org/10.5061/dryad.sxksn038s

This dataset contains the raw data for the Berkeley Single Cell Computational Microscopy Dataset. The data is compressed and chunked to facilitate downloading. The easiest way to download and use it is through the bsccm python package. The code for this package can be found at https://github.com/Waller-Lab/BSCCM/blob/main/Getting_started.ipynb and is archived with a DOI at https://zenodo.org/doi/10.5281/zenodo.10392182

Loading data&

The Getting Started jupyter notebook shows the full documentation for how to use this dataset, including installation, downloading, image/metadata querying, and more. Here we reproduce the first few steps of the notebook:

First, install the bsccm python package with pip install bsccm
Then download the data:

from bsccm import download_dataset
dataset_path = download_dataset('/path/to/download', tiny=True)
print('Downloaded dataset to ' + dataset_path)

Then open an image

dataset = BSCCM
(dataset_path)
valid_indices = dataset.get_indices()
an_index = valid_indices[0]
image = dataset.read_image(an_index, channel='DPC_Left')

Data organization

After downloading and decompressing the data as described in the Getting_Started notebook, the dataset will be organized as follows. This section describes how the data is organized: which files contain which data, what metadata is available, etc. We note that a full understanding of these details is not necessary for using the dataset, as the Python package we provide abstracts away many of these implementation details.

File structure and organization

All image data are stored in Zarr datasets using Blosc/zstd compression. Tabular metadata (i.e., per-cell metadata) are stored in .csv files. Global metadata, which contains information that is not specific to individual cells, but rather pertains to the whole dataset is stored in text files in Javascript Object Notation (JSON) format.

Each top-level BSCCM (regular, coherent, tiny, or coherent-tiny) contains (up to) 5 items:

BSCCM_images.zarr: A Zarr dataset containing all the images of cells
BSCCM_backgrounds.zarr: A Zarr dataset containing the background intensity over the full field of view, for each LED array illumination pattern.
BSCCM_global_metadata.json: A text file containing metadata about the full dataset (pixel size, wavelength, channel names, etc.) in JSON format
BSCCM_index.csv: A comma-separated value (CSV) file containing metadata specific to each cell in the dataset
BSCCM_surface_markers.csv: A comma-separated value (CSV) file containing information about the surface protein marker levels of each cell, along with many measurements derived from the fluorescence images and intermediate values used in computing these levels

BSCCM_images.zarr

Zarr datasets contain a hierarchy of directories. For the BSCCM_images.zarr file, this has the following structure:

+-- antibodies_CD16
|    +-- batch_0
|    |    +-- slide_replicate_0
|    |        +-- dpc
|    |            +-- cell_0
|    |            +-- cell_1
|    |            ...
|    |        +-- fluor
|    |            ...
|    |        +-- led_array
|    |            ...
|    |        +-- histology
|    ...
+-- antibodies_CD45

The outermost directory contains "antibodies_" followed by the name of the antibody used to stain the cells, or "unstained"/"all" for the no-antibody and all-antibody conditions, respectively.

BSCCM_backgrounds.zarr

This file contains the background images for each channel across the full field of view (2056x2056 pixels). The top-level directory contains the channel name. The structure is as shown below.

+-- Brightfield
|   +-- 5\_percentile
|   +-- 10\_percentile
|   +-- 20\_percentile
|   +-- 40\_percentile
|   +-- 50\_percentile
+-- DF\_50

BSCCM_global_metadata.json

This file contains metadata specific to the full dataset, including names of channels, collection settings like exposure, and useful information for calibration like wavelength, objective NA, etc. It is a text file with JSON structure.

{
"led_array": 
    {
    "image_shape": [128, 128],
    "channel_names": ["Brightfield", ... "LED119"], 
    "channel_indices": {"Brightfield": 0, ... "LED119": 200}, 
    "exposure_ms": {"Brightfield": 8, ... "LED119": 200},
    "camera": {
        "offset": 30,
        "gain_db": 4,
        "quantum_efficiency": 0.68
        },
    "wavelength_nm": 515, 
    "pixel_size_um": 0.166, 
    "objective": {"NA": 0.5, "magnification": 20}}, 
"fluorescence": 
    {
    "image_shape": ...
    ...

BSCCM_index.csv

This contains per-cell metadata in a single, large CSV file with one row per each cell. It has the following columns:

"global_index": An integer uniquely identifying the cell
"position_in_fov_y_pix"/"position_in_fov_x_pix":Location of the cell center within the image field of view
"detection_radius": The radius reported by the blob finding algorithm that initially located the cell, which gives a rough estimate of its size
"has_matched_histology_cell": Whether or not the cell has a matching cell in histology contrast
"fov_center_x"/"fov_center_y"/"fov_center_z": the microscope stage coordinates of the field of view from which the cell was drawn
"batch": the index of the cell isolation experiment the cells were drawn from (either 0 or 1)
"antibodies": the name or the single antibody used to stain the cells, or 'all' or 'unstained' if every antibody or no antibodies were used
"imaging_date": the date the slide of cells was imaged
"data_path": the path to the image data with the BSCCM\_images.zarr file
"slide_replicate": the index of the slide replicate within the same antibody/batch conditions (either 0 or 1)

BSCCM_surface_markers.csv

This contains per-cell calculations about fluorescence surface marker levels. It is entirely derived from the fluorescence imaging data. It contains metadata in a single, large CSV file with one row per each cell.

"global_index": An integer uniquely identifying the cell
// raw measurements derived from fluorescence images
"Fluor_426-446_total_raw" //Raw foreground fluorescence in 426-446nm channel
"Fluor_500-550_total_raw" //Raw background fluorescence in 500-550nm channel
...
"Fluor_426-446_background" //Raw background fluorescence in 426-446nm channel
...
(Many more intermediate calculations)
...
// protein levels estimates from unmixing procedure
"CD45_single_antibody_model_unmixed" // CD45 protein levels using 2 spectrum unmixing model
"CD123_single_antibody_model_unmixed"
...
"CD123/HLA-DR/CD14_full_model_unmixed" // Combined CD123/HLA-DR/CD14 protein levels using 4 spectrum unmixng model
...