Dataset DOI: 10.5061/dryad.w9ghx3g4v

Description of the data and file structure

Overview

This dataset contains whole-brain neural activity recordings and behavioral measurements from individual Caenorhabditis elegans (C. elegans) worms freely moving in a circular arena surrounded by agar containing copper. Copper is aversive to C. elegans, and animals avoid the boundary when they encounter it. We imaged transgenic animals expressing pan-neuronal NLS-GCaMP7f and the NeuroPAL transgene, which provides fluorescent barcoding for neuron identification.

This dataset contains data from 27 animals. Each animal is stored as one .h5 file. Neuron identity assignments across all animals are stored in a single .jld2 file. The .h5 files contain various measurements and analyses related to neural and behavioral data, including velocity and neural traces. The .jld2 file contains the neuronal identity of the recorded neurons.

File List and Naming Conventions

Each .h5 file is an HDF5-format binary file containing all neural traces and behavioral measurements for one animal. HDF5 files can be read using standard libraries available in Julia (HDF5.jl).

Each .h5 file is named using the convention:

YYYY-MM-DD-NN-cudata.h5

where YYYY-MM-DD is the recording date and NN is a zero-padded index identifying the individual animal recorded on that date (e.g., 01, 02). For example, 2023-03-30-01-cudata.h5 refers to the first animal recorded on March 30, 2023.

NeuroPAL_labels.jld2 file contains the NeuroPAL-based neuron identity assignments for all recorded animals. This file is in JLD2 format, a Julia-native format based on HDF5. It can be read using the standard JLD2.jl package in Julia. This file stores, for each animal and each identified neuron class, the row index into the neural trace arrays in the corresponding .h5 file, along with associated metadata. 

load_data_example.jl is a Julia script (v1.9.2) demonstrating how to load a single .h5 file and the NeuroPAL_labels.jld2 file.

Data Structure For .h5 Files

Each .h5 file is organized into the following HDF5 groups and datasets. All datasets are arrays indexed along the time dimension unless otherwise noted. There are 1600 frames over 16 minute long recordings.

Behavioral Measurements

• behavior/velocity: Axial (forward/backward) velocity measurements in mm/s.
• behavior/reversal_vec: Binary vector indicating whether the worm is reversing (backward locomotion) at each timepoint. 1 = reversing, 0 = not reversing.
• behavior/reversal_events: Matrix of detected reversal (backward locomotion) events. Each row is one reversal; column 1 is the start frame and column 2 is the end frame. Values are integer indices into the 1,600-timepoint recording.
• behavior/dist_from_cuCenter: Continuous distance of the worm's head from the arena center over 1,600 frames. The copper boundary is located at approximately 40,000, corresponding to a physical distance of 4 mm from the center.
• behavior/pumping: Pharyngeal pumping rate measurements in Hz.
• behavior/head_angle: Head angle measurements in radian. Sign convention may differ between animals depending on imaging orientation.
• behavior/angular_velocity: Angular velocity measurements in rad/s.

Neuronal Data

• gcamp/trace_array_original: Raw neural activity traces. Each row is a neuron, and each column is a confocal imaging time point.
• gcamp/trace_array: z-scored neural activity traces.
• gcamp/traces_array_F_F20: Neural activity traces normalized by the 20th percentile of raw neural activity of the same neuron.
• gcamp/traces_array_F_Fmean: Neural activity traces normalized by mean neural activity of the same neuron.

Timing Information

• timing/timestamp_confocal: Timestamp of each sampled time point in neural traces (sampling frequency ~1.67 Hz).
• timing/timestamp_nir: Timestamps for each frame of the near-infrared behavioral camera, recorded at 20 Hz. The full 20 Hz timeseries is stored as-is; alignment to the confocal timepoints (1,600 frames) is performed during analysis and is not reflected in this variable.

Additional variables are present in the behavior/ (e.g., body angles, worm curvature, stage coordinates), gcamp/ and neuropal_registration/ groups but are not needed for analysis.

Example Dataset

["behavior", "gcamp", "neuropal_registration", "timing"]
📁 behavior/
📄 angular_velocity — size: (1600,) type: Float64
📄 body_angle — size: (30, 1600) type: Float64
📄 body_angle_absolute — size: (62, 1600) type: Float64
📄 body_angle_all — size: (62, 1600) type: Float64
📄 circle_center — size: (2,) type: Float64
📄 circle_radius — size: () type: Float64
📄 dist_from_cuCenter — size: (1600,) type: Float64
📄 head_angle — size: (1600,) type: Float64
📄 pumping — size: (1600,) type: Float64
📄 reversal_events — size: (42, 2) type: Int64
📄 reversal_vec — size: (1600,) type: Int64
📄 stage_x — size: (1600,) type: Float64
📄 stage_xStart_nir — size: () type: Float64
📄 stage_x_nir — size: (19696,) type: Float64
📄 stage_y — size: (1600,) type: Float64
📄 stage_yStart_nir — size: () type: Float64
📄 stage_y_nir — size: (19696,) type: Float64
📄 velocity — size: (1600,) type: Float64
📄 worm_curvature — size: (1600,) type: Float64
📁 gcamp/
📄 idx_splits — size: (1, 2) type: Int64
📄 match_org_to_skip — size: (139, 2) type: Int64
📄 match_skip_to_org — size: (139, 2) type: Int64
📄 trace_array — size: (139, 1600) type: Float64
📄 trace_array_original — size: (139, 1600) type: Float64
📄 traces_array_F_F20 — size: (139, 1600) type: Float64
📄 traces_array_F_Fmean — size: (139, 1600) type: Float64
📁 neuropal_registration/
📄 roi_match — size: (195,) type: Int64
📄 roi_match_confidence — size: (195,) type: Float64
📁 timing/
📄 timestamp_confocal — size: (1600,) type: Float64
📄 timestamp_nir — size: (19696,) type: Float64

Data Structure for .jld2 file

The file NeuroPAL_labels.jld2 contains a nested dictionary (dict_match_dict) storing neuron identity assignments for each recording, identified using NeuroPAL. Each recording key (e.g. "2023-03-30-01") maps to a 2-element Tuple:

Both elements contain the same underlying neuron metadata; they simply provide two different ways to access the same data.

Example Dataset

To look up the identity of a neuron at a known row index (e.g., row 123 in dataset "2023-03-30-01"), use element [1]:

NeuroPAL_labels["2023-03-30-01"][1][123]

This returns the metadata dictionary directly for that neuron.

• label: Individual neuron name (e.g."AVAL").
• neuron_class: Neuron class name (e.g. "AVA").
• LR: "L" or "R" designation, or "undefined" if the neuron doesn't have a pair.
• DV: Dorsal ("D")/ ventral("V") designation, or "undefined" if not applicable.
• region: Anatomical region within the nervous system; empty string if not annotated.
• roi_id: Region-of-interest identifier used during image segmentation. Note: this is not the trace row index and is not relevant for this analysis.
• confidence: Annotator confidence in the neuron identity assignment (scale of 1–5, where 5 = highest confidence); only neurons with confidence 2 or higher were included in the analysis.

To retrieve all neurons assigned to a given neuron class (e.g., "AVA") for a specific recording (e.g., "2023-03-30-01"), use element [2]:

NeuroPAL_labels["2023-03-30-01"][2]["AVA"]

This returns a Vector with one entry per labeled neuron of that class. Each entry is a 3-element tuple:

Running NeuroPAL_labels["2023-03-30-01"][2]["AVA"] returns:

2-element Vector{Any}:

(Dict{String, Any}("label" => "AVAL", "neuron_class" => "AVA", "LR" => "L", "region" => "", "roi_id" => [1], "confidence" => 5, "DV" => "undefined"), 123, 68.35...)

(Dict{String, Any}("label" => "AVAR", "neuron_class" => "AVA", "LR" => "R", "region" => "", "roi_id" => [104], "confidence" => 5, "DV" => "undefined"), 110, 21.98...)

In this example, the trace row index for AVAL is 123 and for AVAR is 110 (second element of each tuple).

Code

The Julia loading code is provided as load_data_example.jl. Only two standard Julia packages are required: HDF5.jl for reading .h5 files and JLD2.jl for reading the .jld2 file, both compatible with Julia 1.9.2. Install them by running Pkg.add("HDF5") and Pkg.add("JLD2") in the Julia REPL.

The script demonstrates two independent tasks. The first block loads a single .h5 file into a flat dictionary keyed by HDF5 path (e.g., "gcamp/trace_array", "behavior/velocity"). Before running, set folder_path to the directory containing the .h5 files and file_name to the name of the file you want to load. The second block loads NeuroPAL_labels.jld2; set dict_path to the full path of that file before running.

To run the script, open a terminal, navigate to the folder containing load_data_example.jl using cd path/to/folder, and execute julia load_data_example.jl. This requires Julia 1.9.2, available at https://julialang.org/downloads/.