This repository provides the data and code necessary to reproduce the manuscript “Peering into the world of wild passerines with 3D-SOCS: synchronized video capture for posture estimation” by Michael Chimento, Alex Hoi Hang Chan, Lucy M. Aplin & Fumihiro Kano. Bold denotes co-first authorship. Note: This is separate from the code necessary to run 3D-SOCS yourself, which can be found at this github repository.

3D tracking pipeline and system accuracy validation were performed in Python, and any questions related to these should be directed to Alex Chan (hoi-hang.chan at uni-konstanz.de). Bayesian statistical analysis, figures and tables were all performed in R, and any questions related to these (or the Python scripts that control the data collection system) should be directed to Michael Chimento (mchimento at ab.mpg.de). We provide required packages, directory contents and column descriptions for all analyses below.

Contents of reproducibility_3D_SOCS.zip

R Requirements

To reproduce R code, the following packages are required/tested

tidyverse==2.0.0
ggpubr==0.6.0
ggExtra==0.10.1
MASS_7.3==60.0.1
rethinking==2.40
kableExtra==1.4.0
HDInterval==0.2.4

Python Requirements

To reproduce python code, the following packages are required/tested

python==3.8
opencv-python==4.9.0.80
numpy==1.24.4
pandas==1.3.5
matploblib==3.7.5
natsort==8.4.0
scipy==1.10.1
tqdm==4.66.4

Analysis

Contents of python code in analysis_R

Contents of python code in analysis_R/stan_models

Contents of python code in analysis_python

Data

Contents in data/cleaned_rda

Column names/descriptions for df_look3s.rda.

Column names/descriptions for VolumeComparison.rda.

Contents in data/fits

This folder contains Bayesian model fits from STAN.

Contents in data/3DTracking

3D Tracking dataset contains all 3D coordinates used in the 3D-SOCS paper, organized into sub-trials.
Dataset is structured as below:

Data
└── 3DTracking
    └── {Date}
        └── {trials}
            ├── metadata.csv
            └── data
                └── Out3DDict.csv
                └── Out3DDict.p
                └── NumCam2_Out3DDict.csv
                └── NumCam2_Out3DDict.p
                ...
                └── NumCam6_Out3DDict.csv
                └── NumCam6_Out3DDict.p

Each trial folder follows the naming convention “bird_PIT-ID_trial_Trial number_YYYY-MM-DD HH_MM_SS.FFFFFF”. PIT-ID is the uniquely identifiable alphanumeric ID from passive integrated transponders. Trial number is a running tally of the unique visits to 3D-SOCS each day.

Column names/descriptions for metadata.csv. Contains frame aquisition metadata for each sub-trial

Column names/descriptions for CSV files (NumCam[2-6]_Out3DDict.csv and Out3DDict.csv).

Contains 3D tracking results for each sub-trial. Out3DDict is identical to NumCam6_Out3DDict and was the data used in the final analysis of gaze presented in the manuscript. Other NumCams were for accuracy validation with an artificially reduced number of cameras.

Names/descriptions for pickle files (NumCam[2-6]_Out3DDict.p and Out3DDict.p). Contains 3D tracking results for each sub-trial.

• Same data as CSV files above but in pickle format, can be loaded as a python dictionary: {frame:{id:{keypoint:[x,y,z]}}}.

Contents in data/SystemAccuracy

All data required to reproduce system accuracy test comparing 3D-SOCS and SMART-BARN
Dataset is structured as below:

data
└── System Accuracy
    └── Raspi
        └── 2024_02_13
            └── data
            └── data_stereo
        └── 2024_02_13_wide
            └── data
            └── data_stereo
    └── Vicon
    └── Wild
    └── MetaData.csv

Column names/descriptions for MetaData.csv. Contains metadata for each trial collected in system accuracy test

Contents for Raspi/{date}/data/. Contains calibration and 3D tracking results from 3D-SOCS for each trial

• All data files used to reproduce system accuracy test
• 3D tracking trials are all stored in pickles,can be loaded as a python dictionary: {frame:{id:{keypoint:[x,y,z]}}}
• Intrinsic and extrinsic parameters for each camera are also stored as pickles.

Contents for Vicon. All data and measurements from SMART-BARN in system accuracy test

• All data files used to reproduce system accuracy test
• We refer to SMART-BARN publication and Vicon documentation for detailed description of file types (https://help.vicon.com/space/Nexus216/11611993/PDF+downloads+for+Vicon+Nexus)

Contents for Wild. All data for in multi-view annotations in the wild

• JSON of 3D and 2D estimates
• Original images and the full dataset can be found in the sample dataset zip.

Contents of supplementary_videos.zip

• Supplementary Video S1
  Qualitative results for multi-individual 3D tracking in 3D-SOCS. Left hand panels show detected 2D keypoints over six synchronized cameras. Detected 3D points were reprojected back to 2D and shown in the right panel, with line representing foveas, 60 degrees left and right from the bill tip projection. Video was slowed down to 0.5x for visualization.
• Supplementary Video S2
  Example of great tit looking at screen mealworm stimulus. Detected 3D points were reprojected back to 2D, with lines representing fovea projections, 60 degrees left and right from bill tip
• Supplementary Video S3
  Example sequence in system accuracy test with taxidermy great tit. A taxidermy great tit was systematically rotated in space, as part of the system accuracy test. The detected 3D keypoint estimate from 3D-SOCS is reprojected to 2D. The video was shown in 4x speed.

Contents of SampleDataset.zip

Data set used as exemplar from github repository. Please see the repository contained in this archive.