Pipeline for data extraction, motion correction, denoising, segmentation, event detection, and visualization of voltage imaging data collected on the Femtonics AOD scope. Denoising data requires a CUDA compatible GPU.

Requirements

I) Install Miniconda: https://docs.anaconda.com/free/miniconda/index.html

II) Create environment:

conda install -n base -c conda-forge mamba          # install mamba in base environment
mamba create -n voltage                             # create environment
mamba env update -n voltage --file environment.yml  # install caiman and SUPPORT dependencies
conda activate voltage

III) This pipeline works best when you take all your mesc & tdml files and place them in a new empty directory, then use that directory path for all subsequent steps. Any pixel timestamp files should also be in this directory and follow the naming convention <mouse_id>expt<expt_num><scan_num>_OffsetGlobalMs.txt.

Workflow

1) Activate environment

conda activate voltage

2) Extract imaging data from mesc files (supported scan modes: multiROILineScan, multiROIMultiLine, eMultiROITransverseRibbonScan, eMultiROILongitudinalRibbonScan)

The output of this step will be a folder for each scan (MUnit) containing a separate HDF5 (.h5) file for each ROI. The raw data for each ROI will be in the "/raw" group of the corresponding HDF5 file.

python mesc2h5.py <path_to_mesc>

Example:

python mesc2h5.py ./mesc_files

3) Convert behavior tdml files to pickle files.

python tdml_pickler.py <path_to_tdmls>

4) Pair imaging and behavior data

This will add the relevant behavior data and metadata to the "/behavior" group of the HDF5 file. No new files will be created.

python pair_imaging_behavior.py <path_to_h5s> <path_to_pickles>

Note: The behavior and imaging PCs may not be synchronized to the same timezone. In which case, the timezone_offset argument can be used to allow the H5s and behavior pickle files to be paired. For example, if the behavior PC is 1 hour ahead of the imaging PC, the correct command would look like:

python pair_imaging_behavior.py --timezone_offset 1 <path_to_h5s> <path_to_pickles>

5) Package imaging and behavior data into a pickle file.

This will iterate through each HDF5 in the directory provided and package data from the "/raw", "/mc", "/denoised", and "behavior" groups into a single pickle. Maybe-do: provide an option for splitting into separate pickles based on mouse_id?

python package_data.py <path_to_h5s> <pkl_output_path>

6) Event detection notebook

This notebook, will read the packaged pickle file, group ROIs in commented multiROILineScan scans, perform simple filtering/smoothing, compute PSTHs, and run simple PCA-based clustering to determine which detected events are likely to be biological signals.

Comment naming convention for multiROILineScan:

    Example 1:
        soma=[0,1]; spine1=[2,3]; spine2=[4,5]; spine3=[6,7]; spine4=[8,9]; shaft=[10,11]

    Example 2:
        soma=[0]; spine1=[1]; spine2=[2]; spine3=[3]; spine4=[4]; shaft=[5]