Less is more: selection from a small set of options improves BCI velocity control
Data files
Jul 30, 2024 version files 4.39 GB
-
centeroutdata3.mat
4.39 GB
-
CenterOutTaskTable.csv
1.32 MB
-
DemographicsData.mat
902 B
-
DemographicsTable.csv
1.81 KB
-
README.md
13.63 KB
Abstract
We designed the discrete direction selection (DDS) decoder for intracortical brain computer interface (iBCI) cursor control and showed that it outperformed currently used decoders in a human-operated real-time iBCI simulator and in monkey iBCI use. Unlike virtually all existing decoders that map between neural activity and continuous velocity commands, DDS uses neural activity to select among a small menu of preset cursor velocities. We compared closed-loop cursor control across four visits by each of 48 naïve, able-bodied human subjects using either DDS or one of three common continuous velocity decoders: direct regression with assist (an affine map from neural activity to cursor velocity), ReFIT, and the velocity Kalman Filter. DDS outperformed all three by a substantial margin. Subsequently, a monkey using an iBCI also had substantially better performance with DDS than with the Wiener filter decoder (direct regression decoder that includes time history). Discretizing the decoded velocity with DDS effectively traded high resolution velocity commands for less tortuous and lower noise trajectories, highlighting the potential benefits of simplifying online iBCI control.
Link to BioRxiv post of study describing the full study for which these data were collected: https://doi.org/10.1101/2024.06.03.596241
Healthy human subjects operated an invasive brain-computer interface (BCI) simulator (see DOI 10.1088/1741-2552/ac97c3 for technical details), wherein their finger movements were used to computationally synthesize a simulation of cortical neural firing rates. The synthetic firing rates were processed by different neural decoding algorithms (depending on the experiment group to which the subject belonged) to generate the velocity of a computer cursor, as is typical in invasive brain-computer interface studies. Subjects performed a center-out target acquisition task using the BCI simulator where they attempted to piloted the cursor to and stay within the a displayed target to register a successful hit, before returning to the workspace center to trigger the subsequent trial.
The dataset contains a table of basic summary data describing each trial, such as trial number, subject, decoder, trial times, etc.. The dataset also contains the full complex low-level data (that does not permit of a tabular format) in a .mat format (MATLAB data file), which includes time series of cursor kinematics, synthetic neural firing rates, finger kinematics, time stamps, decoder calibrations, etc..
Note that this dataset contains only a subset of all the data in the original study: the human subjects engaging in the center-out task using the invasive BCI simulator. It does not contain decoder calibration trials or the virtual keyboard task, nor does it contain single-unit motor cortex recordings from the non-human primate subject.
Description of the data and file structure
There are four main data files. The .csv files contain high level data for each trial and subject, such as which subject performed the trial, where the target was, or whether or not the trial was successful. The .mat files contain the full, low-level data for each trial and subject, such as time series of finger kinematics, the calibration for the decoder being used, and emulated neural data. A separate directory of MATLAB scripts contains the code to reproduce figures in the manuscript from the low-level .mat data. Below we describe the contents of each of the four data files.
>> CenteroutTaskTable.csv
This table contains 23683 rows, each representing a trial one trial in the center-out task (each is either piloting the cursor from the center to a peripheral target, which is timed, or returning the cursor to the center after a trial, which is untimed). There are 10 columns representing data value about the given trial, which are:
1) Trial Number: Sequentially numbered from 1 to 23683, representing each trial across all subjects and visits.
2) SubjectID: An 8-digit automatically generated number to de-identify the subject and indicate who performed the trial.
3) Decoder Group: The type of decoder used in the trial (ReFIT, vKF, DR-A, or DDS).
4) Visit Number: The visit during which the trial was performed (1-4).
5) TargetID: Indicates which target was active during the trial (0 for center, 1-8 for peripheral targets which are east, northeast, north, northwest, west, southwest, south, and south east, respectively).
6) Hit: Indicates if the subject successfully remained within the target continuously for 0.5 seconds (1 for success, 0 for failure). For trials to TargetID 0 the Hit indicator will almost always be 1 because subjects were required to return to and hold within the center to trigger the subsequent trial (with the only exception being if the task terminated during a return trial prior to the subject hitting the center target).
7) Subject-Visit Target Number: The subject-visit (sv) trial number for tracking trials within a subject's visit, ranging from 1 to 64. Note that unlike "Trial Number" above (#1) this counts both center-out and the return to center target together as one trial.
8) Average Velocity: The average velocity of the cursor during the trial.
9) Total PathLength: The total workspace distance the cursor moved during the trial. (The workspace is a square of side length 2 arbitrary units.)
10) Time Taken in Trial: The time taken (seconds) to complete the trial (which is equal to 20 if the trial times out for peripheral targets).
>> DemographicTable.csv
This table contains the demographic data recorded at the beginning of each subject's first visit. The table is a 48 rows (one row per subject) by 7 columns. The 7 columns for the demographic represented the "Subject ID" (linking the data here to the trial data above) and 6 questions below (with the possible responses listed parenthetically):
1) Subject ID: An 8-digit identification number unique to each subject (unidentifiable from all subjects).
2) Age?: Could have been any number from 18 to 45.
3) Handedness?: (right or left).
4) Is this your first CyberGlove Study?: (yes or no).
5) Gender?: (male, female, or other).
6) Do you participate in manual dexterous tasks?: (yes or no).
7) Do you have a nervous system or muscle disorder?: (yes or no).
>> Centerout3.mat
This .mat file holds all the low-level data for the Center-Out task (the MATLAB variable name is "CenterOutData"). It is organized as a 4-by-48 cell array with each column denoting a different subject and each row denoting each of that subject's 4 visits in order (row 1 being visit 1). Within each cell is a struct with the following fields:
1) TaskStartTime: Timestamp in datevec format indicating when the task began.
2) TaskFcn: Always @CenterOutCursorTargets for this data set. (This indicator changed when subjects performed different tasks, such as calibration.)
3) SubjID: The same unique subject identifier referenced in the .csv files.
4) VisitNum: The visit that the subject is currently in.
5) FRBin: Rate at which emulated neuron firing rates are aggregated (in seconds), and therefore the approximate time between each cursor command issued by the decoder.
6) EmulateBin: An internal code variable - The timestep within the specific artificial neural network uses to create the emulated neural firing rates.
7) DevelopmentMode: An internal code variable - 0 if not in development mode (i.e., no subject is present) and 1 if it is.
8) StudyInfo: Struct with fields indicating what study this subject was a part of (always DecoderComparisonCO for this dataset), the decoder group (linking to terminology in the study paper, vKF=vKF, ReFIT=ReFIT, PopVecReg=DR-A, MNSelect=DDS), and the task the subject is currently in (always "2 of 3" for this data set).
9) EmulatorFcn: The function to call the model emulating the neuron firing rates (always @TimeDelayANNr for this data set).
10) EmulatorPars: The weights and biases used by the model specified in EmulatorFcn.
11) DecoderFCN: The function to call each separate decoder.
12) DecoderPars: Cell array in which the first element contains a struct defining all parameters required for the decoder to execute. The fields here will depend on which decoder is being used by the subject (e.g., calibration matrix, time step, or categories). All other elements in the cell array are used only for task dynamics and updated continuously during the study (such as current cursor state) and are therefore irrelevant for this dataset.
13) TaskInfo: Struct detailing parameters of the task itself, including the size of the target, the amount of time that the cursor needs to remain in the target for a hit, the number of targets, the target's position, and the task time limit.
14) TrialData: Contains the low-level data for analysis as a table and is discussed further below. This is the primary data field.
15) GloveHardware: hardware specifications for the CyberGlove III used to record subject hand kinematics during the experiment.
16) CodeVersion: Experiment software version.
16) SystemPars: Configuration of the computer and MATLAB version used.
17) SaveTime: Timestamp in datevec format indicating when the task was terminated.
More information about the low-level data contained in the TrialData table:
Each TrialData table has a different number of rows depending on the total number of trials completed in the task (usually 128, which include 64 out targets and 64 center targets, i.e., 64 sv-trials) and 8 columns. These columns include:
1. TimeStamp: Tracks the moment in time which each new cursor command and neural data is given at approximately every FRBin seconds. All other columns containing time series will have one sample (possibly multidimensional) per column for each timestamp.
2. HandKinematics: CyberGlove sensor values at each sample and has 19 rows (one for each recorded sensor).
3. NeuralFiring: Emulated neuron firing rates created at each sample. The number of rows here represents the number of emulated neurons, which changes based on the visit.
4. ObjectState: The X position, Y position, X velocity, and Y velocity at every moment in time with rows 1-4, respectively.
5. PrevTargPos: Location (x, y position in workspace units) of the center of the active target for the given trial.
6. PrevTargNum: Running count of sv-trial number for this visit. Both acquisition of the center target and subsequent attempt at the peripheral target share the same PrevTargNum.
7. PrevTargID: Target indicator, the same field referenced as "TargetID" in the .csv files.
8. NumHits: The cumulative number of peripheral targets a subject hit (reached at stayed within continuously for 0.5 s).
>> DemographicsData.mat
Cell array of self-reported subject demographic survey data, collected at the beginning of their first visit (the MATLAB variable name is "demographics"). Each row contains data for one subject and each column conforms to the descriptions given in the DemographicTable.csv section.
Code/Software
We provide MATLAB scripts to reproduce figures from the original manuscript from the Centerout3.mat data file (in the Zipped directory "CodePackage-AlcoleaDanziger2024"). The MainCodeJNE script calls each support function (located in the MainCodeSupportFunctions directory) in separate code blocks to produced each analysis. Make sure the MainCodeJNE script and the associated MainCodeSupportFunctions directory are placed in the same directory as the data files prior to running the code. Additional information about each code block is provided as comments within MainCodeJNE, and brief descriptions of each code block are given here:
Data Loading: This section handles the initial data loading and formatting necessary for subsequent analysis (run this block BEFORE any others).
Performance Comparison: Produces the main performance graph (Figure 1C) showing how well subjects in each decoder group hit targets. It also includes additional analysis graphs previously used to visualize performance.
Histogram Time Plots: Analyzes the time taken by each decoder group to hit targets by creating a histogram comparing time (x) to the percentage of hits done over time (y), with results shown in Figures 1D and 1E of the original paper.
Average Speed Analysis: Examines the average cursor speed near the target or during successful trials, by creating 2 histograms: 1. Curser speed at the moment of target hit (x) to the percentage of times a hit occurred at that speed(y). 2. Average curser speed at throughout trials (x) to the percentage of times a successfully trial occurred at that speed(y), contributing to Figures 2A and 2B of the original paper, respectively.
Target Direction Histogram: Assesses the trail time spent directed toward the target (x) to the percentage of successful trials (y). Found in Figure 2E of the original paper.
Path Length and Tortuosity: Evaluates the path length (x) and tortuosity (y, the difficulty or amount of turns taken during the path) for each decoder group (Figure 2F of the original paper).
Representative Cursor Trajectories: Creates visuals of cursor trajectories for each decoder group, based on average performance of each group, used in Figure 1B of the original paper.
Neuron Performance Correlation: Investigates the impact of neurons on performance, finding no significant confounding effects (given as a statement in the original paper). Plot has the visits on the x axis and average performance of each decoder group on the y axis.
Target Type Performance: Compares performance on diagonal vs. cardinal targets, with results shown in Supplementary Figure 2A of the original paper. The x axis is the difference between the cardinal and diagonal target and the y axis is the percentage of subject-visits.
Demographic Correlation Analysis: Explores potential correlations between demographics and performance but finds none. Also holds the code that transforms the raw data (DemographicsData.mat) to the table (DemographicTable.csv).
Average Trials Analysis: Computes the average number of sv-trials completed by subjects in each decoder group who failed to complete all 64 center out and back trials before the task timeout period, noting DDS had no missed sv-trials (as indicated by NaN).
Within Group SD vs Between Group SSE: Compares the standard deviation(SD) between groups and within them to check how close the two, shown figure version in Supplementary Table 3 of the original paper.
Data Transformation: Transforms raw data from Centerout3.mat to a structured table format in CenteroutTaskTable.csv.
Data were collected from 48 able-bodied human participants on four separate lab visits each using the protocol for the real-time, human-in-the-loop, invasive brain-computer interface model called the jaBCI (published model details at DOI 10.1088/1741-2552/ac97c3).
Briefly, a subject wears a CyberGlove III that monitors the relative excursions of 19 different hand and finger joint angles. The same pre-trained artificial neural network used in the jaBCI validation study computes a set of emulated neural firing rates based on the preceeding 100ms of joint angle input data. To align with contemporary firing rate integration bin widths used in iBCI studies, we set the emulator to emit neural firing rates every 50 ms. At each time bin the subject’s neural decoder uses the emulated neural activity to compute the velocity of the computer cursor and the graphical display is updated accordingly. The joint angle tracking, neural emulation, and decoding all run in real-time. To mimic the phenomenon of neuron turnover in iBCI studies we used four separate neural readout modules (one for each visit) that emulated 71, 45, 70, and 82 neurons, thus requiring subjects to recalibrate their decoders at the start of each visit. These data are collected as subjects attempt to pilot the computer cursor to displayed targets in the center-out task described in the accompanying publication.
Therefore, the three low-level timeseries signals we record during closed-loop decoder use are 1) the subject's finger kinematics (in glove sensor units, 19 dimensional), 2) the corrisponding emulated neural firing rates (in average action potentials per second, c dimensional where c is the number of neurons in the given visit), 3) the state of the decoded computer cursor being controlled (a 4 dimensional vector of x position, y position, x velocity, and y velocity in arbitrary workspace units).
These low level data are used to determine all subsequent behavioral outcomes, such as how long the subject took to reach the target or whether the subject hit the displayed target.
