Dataset DOI: 10.5061/dryad.2bvq83c34

Description of the data and file structure

This dataset contains trial-level data from three rhesus macaques performing brain–computer interface (BCI) navigation tasks in Unity-based virtual environments. Data are provided as Python pickle files (.pkl) containing trial structures with timestamps, behavioral trajectories/velocities, and multiunit activity (spike threshold crossings) from Utah arrays implanted in motor cortical areas. For many sessions, synchronized eye-tracking and surface EMG recordings are also included as separate .pkl files. Spike detection thresholds used during each session are provided as .ccf files.

Files and variables

File: Method_paper_data.zip

A compressed archive containing all session data organized by subject (“Monkey 1/2/3”) and task.

Folder organization inside Method_paper_data.zip

Method_paper_data/
  Monkey 1/   (Loki)
  Monkey 2/   (Vino)
  Monkey 3/   (Maui)

Within each Monkey folder, task folders are present only for tasks performed by that animal:

• Monkey 1 (Loki): performed 2 tasks (fewer task folders).
• Monkey 2 (Vino): performed 5 tasks.
• Monkey 3 (Maui): performed 6 tasks (includes first-person perspective continuous navigation).

Method_paper_data/
  Monkey 1/   (Loki)
    Center-out/
    Continuous Navigation/
  Monkey 2/   (Vino)
    Center-out/
    Continuous Navigation/
    3D Center-out/
    Respawn/
    Obstacle/
  Monkey 3/   (Maui)
    Center-out/
    Continuous Navigation/
    3D Center-out/
    Respawn/
    Obstacle/
    Continuous Navigation (first person perspective)/

Structure inside each task folder

Each task folder has the following structure:

Task_Name/
  trainingFiles/
     navtraining_<Subject>_<YYYYMMDD>_<HHMM>_<SessionLetter>_trials.pkl
     <Subject>_<YYYYMMDD>_<HHMM>_<SessionLetter>.ccf
  navdecoding_<Subject>_<YYYYMMDD>_<HHMM>_<SessionLetter>_trials.pkl

Always present

Contains files required for decoder training:

trainingFiles/
  navtraining*_trials.pkl — training trial data (passive observation phase)
  *.ccf channel configuration file (spike thresholds) used during that session

Located directly inside the task folder:

• navdecoding*_trials.pkl — closed-loop BCI trial data
• *.ccf channel configuration file (spike thresholds) used during that session

There are multiple navdecoding*.pkl, navtraining*.pkl, and .ccf files within each task folder — one set per recording session.

Optional additional folders (session-dependent)

Some task folders may also contain:

eyeMovementData/
offlineAnalysisPerArea/
emgData/

• eyeMovementData/ — continuous eye position recordings.
• offlineAnalysisPerArea/ — area-specific offline decoding results (e.g., M1, PMd, PMv).
• emgData/ — continuous sEMG recordings.

These folders are present only when that modality or analysis was collected/performed. Task folders contain multiple sessions (different dates/times/session letters), each with its own training/decoding files.

Tasks and corresponding filename prefixes

Each session typically includes:

• a training file (navtraining…) used to train the decoder from passive observation
• a decoding file (navdecoding…) containing closed-loop online BCI trials

Task names used in the paper correspond to these filename prefixes:

File naming convention (recommended for reuse)

Most data files follow:

<prefix>_<Subject>_<YYYYMMDD>_<HHMM>_<SessionLetter>_<suffix>.pkl

Examples:

• navdecoding_Maui_20240304_1131_A_trials.pkl
• navtrainingsphere_Vino_20240206_0857_A_trials.pkl

Where:

• Subject is one of: Loki, Vino, Maui
• YYYYMMDD is the recording date (YearMonthDay)
• HHMM is local start time in 24h format (HourMinute)
• SessionLetter is a session label (A, B, C, …) used internally during acquisition
• suffix indicates content (see below)

File types included

(1) Trial data (.pkl)

• *_trials.pkl : training or decoding trial structures (core dataset)
• *_emg.pkl : surface EMG data (available for a subset of sessions)
• *_eyeMovementX.pkl and *_eyeMovementY.pkl : eye position traces (available for a subset of sessions)

(2) Spike threshold configuration (.ccf)

• *.ccf: spike detection thresholds and electrode metadata used during that session

(3) Offline analysis results

• offlineAnalysisPerArea/decoding_results.pkl (folder structure may repeat per monkey/task/date)
  Contains offline decoding predictions using different electrode selections (per area or combinations).

Number of sessions per task and monkey

The number of recording sessions (i.e., number of navdecoding*_trials.pkl files) per task and per animal is:

• Center-out task
  • Monkey 1 (Loki): 5 sessions
  • Monkey 2 (Vino): 10 sessions
  • Monkey 3 (Maui): 19 sessions
• Continuous Navigation task
  • Monkey 1 (Loki): 6 sessions
  • Monkey 2 (Vino): 7 sessions
  • Monkey 3 (Maui): 15 sessions
• 3D Center-out task
  • Monkey 2 (Vino): 12 sessions
  • Monkey 3 (Maui): 10 sessions
• Respawn task
  • Monkey 2 (Vino): 11 sessions
  • Monkey 3 (Maui): 9 sessions
• Obstacle task
  • Monkey 2 (Vino): 10 sessions
  • Monkey 3 (Maui): 11 sessions
• Continuous Navigation task (first person perspective)
  • Monkey 3 (Maui): 3 sessions

Each decoding session corresponds to one navdecoding_trials.pkl file and one associated .ccf file.
Training sessions are stored separately as navtraining_trials.pkl files. A single training session may be used to train a decoder that is applied to multiple decoding sessions.

This archive contains 128 decoding files, 65 training files, and 128 .ccf files. The same .ccf file is stored inside trainingFiles/ and duplicated in the task root directory for convenience.

Usage notes: how to open and view the files

Opening .pkl files (Python)

All .pkl files were generated in Python and can be loaded using:

import pickle

with open("filename.pkl", "rb") as f:
    obj = pickle.load(f)

The loaded object is typically:

• a list of trials, or
• a dictionary containing per-trial arrays and metadata (offline analysis).

If you prefer a safer/modern loader, joblib may also work depending on how the file was saved, but pickle is the default.

Viewing .ccf files

.ccf files are XML-based channel configuration files exported by the Blackrock/Cerebus acquisition system. They contain per-channel metadata used during that recording session, including electrode labels, acquisition settings, and spike detection threshold levels.

They can be:

• opened in any text editor (e.g., VS Code, Notepad++, etc.), or
• parsed programmatically using an XML parser in Python (e.g., xml.etree.ElementTree).

Each .ccf file includes:

• Electrode/channel labels (e.g., elec1)
• Channel and bank identifiers
• Analog-to-digital scaling parameters (units typically µV)
• Hardware filter settings
• Spike detection threshold values (stored as integer level codes in the XML)

Key fields relevant for this dataset are described below under “Variables”.

Variables and data dictionaries (with units)

A) Training and decoding *_trials.pkl files

Each *_trials.pkl file is a 2-tuple:

• session_config (dict): task/session parameters (e.g., task, threeDimensions, targetWindowRadius, timeToStayInTargetWindow, dataDirectory, isTraining, …)
• trials (list of dicts): one dictionary per trial

Below describes the per-trial dictionary keys (exact presence depends on task).

Session configuration (session_config)

Per-session task parameters exported from Unity. Keys may vary by task/version.

• task (str): task variant (e.g., movingCamera, fixedCamera). "movingCamera" is for the Continuous Navigation and Continuous Navigation (first person perspective) task, the rest is labeled "fixedCamera".
• threeDimensions (bool): if True, movement/control is 3D; if False, movement is constrained (typically planar) even if Unity stores 3D coordinates.
• targetWindowRadius (float, Unity units): radius of the target acceptance region used for success detection.
• timeToStayInTargetWindow (int, ms): dwell time required inside the target window to count as target acquisition.
• curvedPath (bool): whether the task enforces a curved/trajectory constraint during the training (task-specific).
• isTraining (bool): whether the session/trials were run in training mode (as set in Unity).

Logged for reproducibility:

• dataDirectory (str): acquisition path on the recording computer.
• useDebugLogLevel (bool): Unity logging verbosity.

Per-trial dictionary

• trial (int): trial index.
• start (float, ms): trial start time relative to the start of the recording file.
• stop (float, ms): trial end time relative to the start of the recording file.
• answer (int): trial outcome code.
  • 1 = correct
  • 3 = incorrect: target went off-screen
  • 5 = incorrect: target not reached within max trial time
  • 6= incorrect: sphere/avatar went off-screen
  • other values = aborted trials

• targetPosition (np.array, shape (3,), Unity units): target position (x, y, z) in the local task coordinate frame, expressed relative to the avatar’s start position at (0, 0, 0) at the beginning of the trial.
• targetOnset (float, ms): time the target appeared.
• photoEvents (np.array, ms): timestamps when the target was visible on screen (photodiode / screen-sync events).

Task-dependent fields:

• targetJumpPosition (np.array, shape (3,), Unity units): respawn/jump target position (Respawn task only; otherwise typically [nan, nan, nan]).
• unityTargetJumpTime (float, ms; often nan): target jump/respawn time recorded from Unity (Respawn task only).
• unityTargetPosition (np.array of dict, dtype=object; Unity units): target position as logged by Unity in absolute world coordinates (often a 1-element array containing {'x','y','z'}).
• obstaclePosition (np.array, shape (3,), Unity units): obstacle position (Obstacle task only; otherwise typically [nan, nan, nan]).

• avatarTrajectory (dict): sphere/avatar position time series
  • time (np.array, ms)
  • x, y, z (np.array, Unity units) in the same local coordinate frame as targetPosition
  • may include avatarRotation (np.array, shape (T,), degrees): avatar yaw angle around the vertical axis (Unity Euler Y), sampled at the same timestamps as x,y,z.
• avatarVelocity (dict): sphere/avatar velocity time series
  • time (np.array, ms)
  • vx, vy, vz (np.array, Unity units/s)

• muaA (dict): multiunit threshold crossings per electrode/channel
  • keys like elec1, elec2, …
  • each value is an array of spike timestamps (ms, relative to recording start)
• contA(dict): local field potential (LFP) signal per electrode.
  • each channel stores:
    • samplingRate (float, Hz; typically 1000.0)
    • data(np.array, µV): continuous LFP signal

Optional / task- or session-dependent:

• muaB, contB: same structure as above for a second bank/array.

Unity uses arbitrary Unity units for spatial quantities. In this dataset:

• positions are stored in Unity units
• velocities are stored in Unity units/s
• timestamps in trial dicts and behavior time series are in ms, all relative to the start of the recording file.

All timestamps in *_trials.pkl (trial events, behavior samples, and neural event times) are expressed in milliseconds relative to the start of the recording file, enabling direct alignment. Continuous signals (LFP, EMG, eye) are aligned to trials using their samplingRate and the trial start/stop times. Spike times and event timestamps are stored in milliseconds relative to recording start. While some logged Unity fields (e.g., unityTargetPosition) are spatial metadata rather than time-synchronized traces.

B) .ccf files (spike threshold configuration)

Files are XML-based Blackrock Cerebus Channel Configuration Files (CCF, Version 12) containing per-channel acquisition parameters used during that recording session.

Each file includes:

• Electrode/channel labels (e.g., elec1)
• Channel ID and bank information
• Analog-to-digital scaling parameters (including units, typically µV)
• Hardware filter settings
• Spike detection threshold levels
• Sampling and display parameters

Spike thresholds are stored in the acquisition system’s digital units, together with scaling parameters that define their relationship to physical units (µV).

These files can be opened with any text editor.

C) sEMG data *_emg.pkl

Each file contains a list of trials, each trial holding a dictionary:

• data (np.array, shape (T,), µV): EMG voltage over time (microvolts)
• sample rate(float, Hz): sampling rate 

EMG channels correspond to biceps and thumb placements as described in the related manuscript.

EMG recordings are available for a subset of decoding sessions (60 files).

D) Eye data *eyeMovementX.pkl and *eyeMovementY.pkl

Each file contains a list of trials, each trial holding a dictionary:

• data (np.array, shape (T,), volts): raw eye position voltage from the acquisition system.
  • X coordinate in _eyeMovementX 
  • Y coordinate in _eyeMovementY 
• sampling_rate(float, Hz): typically 10000.

122 pairs for each movement files (X and Y) are available.

E) Offline analysis (offlineAnalysisPerArea/)

Decoding results

Where it lives:
offlineAnalysisPerArea//<electrode_set>/decoding_results.pkl

with <electrode_set> one of: All, M1, PMd, PMv, M1+PMd, M1+PMv, PMv+PMd, and  Online_simulation.

Availability:
Offline analysis folders are present only for:

• Monkey 1 and Monkey 3
• Center-out and Continuous Navigation tasks

For these sessions, one decoding_results.pkl file is provided per electrode set for each corresponding decoding session. In total, 360 offline decoding files are included.

What it is:
decoding_results.pkl contains the output of the offline decoding pipeline: offline-predicted velocity commands for each online trial, using a model trained with the electrode subset corresponding to that folder (e.g., M1-only).

Each folder (e.g., M1/, PMd/, All/) contains predictions generated using only that specific electrode subset.

File format (Python dict)

Top-level keys:

• trial_answers (list of int; length = #trials)
  Outcome codes for the corresponding trials (same convention as in *_trials.pkl, e.g. 1=correct, 5=timeout, etc.).
• allPredictions (list; length = #trials)
  Offline velocity predictions aligned to each trial.
  allPredictions[i] is a list over time bins for trial i. Each time bin is a dict:
  • x : np.array shape (1,) — predicted velocity along Unity X (Unity units/s)
  • y : np.array shape (1,) — predicted velocity along Unity Y (Unity units/s)
  • z : np.array shape (1,) — predicted velocity along Unity Z (Unity units/s)
  • Notes: In 2D tasks, y is often always zero.
  • The number of time bins per trial depends on trial duration and the binning used in the offline pipeline (typically the same bin size as decoding, e.g. 50 ms).
• allRewards (list; often empty)
  Legacy field; usually unused in this dataset.

Electrode selection metadata

Where it lives:

offlineAnalysisPerArea/electrode_selection.pkl

What it is

Dictionary mapping each decoding session to the electrode subsets used for area-specific offline decoding. It specifies which electrode labels belong to: M1, PMd, PMv for each session.

This file defines the electrode grouping used to generate the All, M1, PMd, PMv, and combined electrode-set folders.

File format (Python dict)

Top-level keys are decoding session identifiers matching decoding filenames without "_trials.pkl".

Example structure:

{
"navdecodingsphere_Loki_20230825_1023_A": {
"M1": ["A_elec67", "A_elec139", ...],
"PMd": ["A_elec215", "A_elec218", ...],
"PMv": ["A_elec15", "A_elec17", ...]
},
...
}

For each session:

• Keys: "M1", "PMd", "PMv"
• Values: list of electrode labels used for that area (typically 27 electrodes per area)

The All condition corresponds to the union of M1, PMd, and PMv for that session. Electrode assignments are session-specific and may vary slightly across dates. A total of four electrode_selection.pkl files are included, one per Monkey/Task combination for which offline analysis is provided

Missing data, abbreviations, and notes

• Missing data codes: Missing values are typically stored as NaN (e.g., fields not applicable to a task).
• Some modalities are session-dependent:
  • EMG and eye tracking are not available for all animals/sessions.
• Abbreviations:
  • mua= multiunit activity (threshold crossings)
  • fpp = first-person perspective

Code/Software

• Data files were generated and analyzed using Python.
• To load .pkl files, use Python’s pickle module (example above).
• Unity tasks were implemented in Unity 2021.3.x (used during data collection).
• Related code to reproduce offline analyses and figures: Zenodo https://zenodo.org/records/15791187

Access information

Other publicly accessible locations of the data:

• None

Data was derived from the following sources:

• N/A