https://doi.org/10.5061/dryad.r7sqv9sn2

Description of the data and file structure

Raw data (available here: https://doi.org/10.5061/dryad.zcrjdfngp) was collected from a Pupil Labs Core eye tracker and a Shadow motion capture system. POV video was run through Meshroom to produce a 3D reconstruction of the environment and localize motion capture data in the same reference frame.

This dataset has processed data used for analysis and figure generation in the associated manuscript.

Files and variables

File: chosen_vs_rand_5.zip

Description: 

Files are stored in Matlab (.mat) format.
There is one .mat for each trial (e.g. JAC_1.mat is subject JAC, trial 1).

Each .mat file contains 4 fields:
“chosen_max_dh”: for all 5-step sequences taken in this trial, the maximum slope of each sequence.
“chosen_mean_dh”: for all 5-step sequences taken in this trial, the mean slope of each sequence.
“rand_max_dh”: for many random samples of possible 5-step sequences, the max slope of each sequence.
“rand_mean_dh”: for many random samples of possible 5-step sequences, the mean slope of each sequence.

Slopes are all ratios of height change/distance traveled (rise/run), not angles

File: retinalCNN_data.zip

Description: 

Files are stored in Matlab (.mat) format.
There are multiple .mat files for each trial(top-level folders e.g. JAC_1 is subject JAC, trial 1).
Each .mat corresponds to a frame of the recording.

Each .mat file contains 4 fields:
“depth”: an eye perspective depth image for that frame of data.
“this_foot_map”: a probability distribution of ground truth foot locations, constructed using Gaussians centered at actual foot locations projected into image space (“step_ii” and “step_jj” specify ground truth foot locations).
“step_ii”: array of foot location rows in image space.
“step_jj”: array of foot location columns in image space.

File: data.zip

Description: 
Contains data for all remaining figures not covered by the previous two .zip files.

pupilShadowMesh:
Contains 1 .mat file per trial, each .mat file has the following fields (some were deprecated).

Note units for any position-related variables are arbitrary (derived from a photogrammetry software reference frame, which has no reference scale):

“cens”: Nx3 array of XYZ head positions over time (30Hz).
“cens_us”: Upsampled version of cens (linearly interpolated, 120Hz).
“closestActualIndexDist”: deprecated
“closestPrefIndexDist”: deprecated
“eyeVec”: Nx3 array of gaze direction unit vector (downsampled from 120Hz)
“eyeVec_orig”: Original version eyeVec (original raw data 120Hz)
“fixBool”: N length Boolean array of fixation frames (1) vs sacadde frames (0)
“fixBool_orig”: fixBool at original 120Hz
“fixationList”: Mx2 array of M fixations, each row contains [startFrame, endFrame] of fixation, computed from fixBool
“frame_prefStepNum_XYZ”: deprecated
“gazeXYZ”: Nx3 array of gaze vector intersections with triangle mesh (obtained via raycasting in Blender)
“markerNames”: string array of marker names in shadow variable’s 2nd dimension (different body parts in mocap data)
“orig2alignedMat”: 3x3 rotation matrix rotating from triangle mesh reference frame into motion capture reference frame
“shadow”: Nx30x3 XYZ positions of each marker in mocap data (see markerNames for marker names of each of the 30 markers)
“step_plantfoot_xyz”: Wx5 array, where W is the number of steps. Columns are (footfall frame, which foot 1 being left, 2 being right, X, Y, Z)
“y_flip”: whether the data required a Y flip from the triangle mesh reference frame
“z_flip”: whether the data required a Z flip from the triangle mesh reference frame
“fakeGazeXYZ”: deprecated

straight_hod_tortuosity_rel:
Contains 1 .mat file per trial, each .mat file has the following fields:
“straight_path_hods_med”: slopes (rise/run) of many samples of possible straight paths emanating from each foothold.
“tortuosity”: tortuosity of the chosen path at each foothold

tforms:
contains text files holding 4x4 transformation matrices, which rotate each of the corresponding triangle meshes into the same reference frame. Used for any analysis requiring all meshes to be in the same reference frame

meshDist.mat:
The nearest neighbor of the point in each mesh, to every other mesh when all meshes are aligned. Distances are in cm

meshDist_median.mat:
same data as meshDist.mat, but the median instead of the entire distribution of distances for each mesh

meshDist_q90.mat:
same data as meshDist.mat, but 90th quantile instead of entire distribution of distances for each mesh

stepDist.mat:
Nearest neighbor of each step location in each mesh, to step locations computed using each other mesh. Distances are in cm

stepDist_median.mat:
same data as stepDist.mat, but the median instead of the entire distribution of distances for each mesh

stepDist_q90.mat:
same data as stepDist.mat, but 90th quantile instead of entire distribution of distances for each mesh

step-params.mat:
contains distributions of scalar values derived from each step all subjects took. Distances are in units of median subject step length:
“dist_3d”: distance in 3D of step start to step end XYZ location
“ground_dist”: distance considering only XY (not vertical) components.
“height”: change in Z component only.
“slope”: the slope of the step in degrees.
“dir”: step direction (degrees).
“goal_dir”: direction to a goal from step start location (degrees)
“dir_vs_goal”: step direction, relative to goal direction (degrees, 0 being directly towards goal)

step-params-subj.mat:
same data as in step-params.mat, but per subject

tort_analysis_data.mat:
Contains per-subject data with the following variables:
“subj”: Subject identifier (string).
“leg_cm”: Leg length of each subject in centimeters (double).
“slope”: Cell array containing slope (rise/run) data for each subject (in degrees).
“tort”: Cell array containing tortuosity data for each subject.
“mask”: Cell array of logical vectors indicating valid data points (true for valid).
“corr”: Pearson correlation coefficient between slope and tortuosity for each subject (double).
“corr_p”: P-value associated with the correlation coefficient for each subject (double).
“fit_slope”: Slope (m) of the fitted line in the tortuosity vs. slope data (double).
“fit_y0”: Y-intercept (b) of the fitted line in the tortuosity vs. slope data (double).

step-corr-coeff.csv:
correlation coefficients for all to all comparisons of step-params.mat fields

step-corr-p.csv:
correlation p values for all to all comparisons of step-params.mat fields

step-covar.csv:
covariance for all to all comparisons of step-params.mat fields

Code/software

You can download the data for this repo from the associated Dryad dataset

Place the downloaded data in the folder data

The main analysis packages are:

+step_params

Figure 7A-C – step_params.plot_hist_fig7 – Step parameter distributions help define feasible alternative paths. The histograms show the distributions of (A) step slopes, (B) step lengths, and (C) step direction relative to goal direction. These distributions define the set of feasible next steps for a given foothold, allowing the calculation of feasible alternative paths to the one actually chosen by the subject. This figure shows histograms of these quantities pooled over subjects, but note that calculations of viable paths were done based on the step parameters of each individual subject.

Figure 14 – step_params.plot_hist_per_subj_fig14 – Per-subject histograms of the step parameters are shown in Figure 7.

Figure 15 – step_params.plot_all_params_fig15 – Aggregate histograms of the step parameters defined in the methods section titled “Step analysis”. Top row, from left to right: (1) step distance (meters), (2) step ground distance (meters), (3) step height (meters), (4) step ground distance (step length; see also Figure 7B). Bottom row: (1) step slope (deg; see also Figure 7A), (2) step direction (deg), (3) goal direction (deg), (4) step direction relative to goal direction (deg; see also Figure 7C).

Figure 16 – step_params.plot_all_params_covar_fig16 – Scatter plots between all parameters defined in the methods section titled “Step analysis”. The order of the step parameters along the x- and y-subplot axes are: (1) step distance, (2) step ground distance, (3) step height, (4) step slope, (5) step direction, (6) goal direction, (7) step direction relative to goal direction, and (8) step ground distance in step lengths. The title of each subplot contains the correlation value r and its associated p-value.

+tortuosity

• Figure 9 – tortuosity.plot_slop_vs_tort_fig9_10.m – Paths chosen by walkers have a lower step slope… B. An illustration of the relationship between chosen path segment tortuosity and the slope of “straight” path segments that were simulated across the same terrain. Each subpanel depicts one subject’s data. To summarize the large amount of data per subject (317–497 path segments), we binned the data into 20 quantiles of straight path slope and averaged tortuosity per bin, generating one summary tortuosity value per slope level. These scatterplots show the average tortuosity as a function of the straight path slope quantile (cyan crosses), along with best-fit lines (black). (For scatterplots showing data per chosen path segment, see Supplementary Figure 18A.) Associated correlation values (Pearson’s r) are shown at the top of each panel.
• Figure 10 – tortuosity.plot_slop_vs_tort_fig9_10.m – Relationship between leg length and the correlation between straight path step slope and path tortuosity. Subjects’ leg lengths (in centimeters) are plotted on the horizontal axis. The correlation coefficients drawn from the analyses depicted in Figure 9B are plotted on the vertical axis. The scatterplot shows one point per subject (black crosses). The linear trendline is also shown (red line). We found that there was a statistically significant negative correlation between subjects’ leg lengths and the straight path slope vs. average path tortuosity correlations in their data (r = −.86, p = .003). (For a comparable plot showing the correlations derived from data per chosen path segment, see Supplementary Figure 18B.)

Figure 17 – tortuousity.plot_slope_and_tort_fig17 – Distributions of straight path slope and chosen path segment tortuosity. Each row presents one subject’s data. The left column contains histograms of the average slope for simulated “straight” path segments, and the right column contains histograms of the tortuosity of that subject’s chosen path segments (5-step sequences).

Figure 18 – tortuosity.plot_leg_vs_corr_fig18 – Relationship between average straight path segment slope and the tortuosity of each chosen path segment. A. Tortuosity of the chosen path segments (5-step sequences) vs. slope for the simulated “straight” path segments for each of the 9 subjects. Correlation values and corresponding p-values are indicated at the top of each panel. B. Correlation between subject leg length and the strength of the straight path slope vs. tortuosity relationship in their foothold selection data. Leg lengths (in centimeters) are plotted on the horizontal axis, and the correlation coefficients for each of the plots in panel A are plotted on the vertical axis. We found a statistically significant negative correlation between those two values (r = –.70, p = .04).

+mesh_comp

Figure 5A-B – mesh_comp.plot_aggregate_distances_fig5 – Accuracy of terrain reconstructions A. Nearest neighbor error distribution for the whole terrain (median=4.5cm, 95% quantile=20.0cm). B. Nearest neighbor error distribution for individual footholds (median=4.0cm, 95% quantile=16.8cm).

retinal CNN analysis:

dataloader.py - load the .mat files in retinalCNN_data.zip in the Dryad dataset

neural_network.py - define neural network architecture

test.py - test and evaluate the trained model on held-out data

train.py - train model to predict foothold locations given depth images

visualize.py - helper code to visualize results in the video

visualize_single_example.py - helper code to visualize a single frame of results

zz_chosen_v_random_fig.m (Figure 8):

create histograms of chosen vs random path slopes, relies on chosen_vs_rand_5.zip .mat files