Xuan Ma, Fabio Rizzoglio, Stephanie N. Thacker, Christian Ethier, Matthew G. Perich, Lee E. Miller

Northwestern University

Correspondence Email: xuan.ma1@northwestern.edu

This data set includes behavioral recordings and extracellular neural recordings from the primary motor cortex of six rhesus macaque. Christian Ethier, Stephanie N. Thacker, Matthew G. Perich and Xuan Ma collected these data in the laboratory of Lee E. Miller in Northwestern University. Xuan Ma and Fabio Rizzoglio processed the data for use in [1], which utilized adversarial neural networks to "align" the neural signals over a long period of time to extend the accuracy of an iBCI decoder calibrated only on the very first day (day-0).

Please cite this article if you are using this dataset:

[1] Ma X, Rizzoglio F, Perreault EJ, Miller LE, Kennedy A. Using adversarial networks to extend brain computer interface decoding accuracy over time. eLife 2023;12:e84296. DOI: https://doi.org/10.7554/eLife.84296

Description of methods

Electrophysiology:

Depending on the task, we implanted a 96-channel Utah electrode array (Blackrock Neurotech, Inc.) in either the hand or arm representation area of the primary motor cortex (M1), contralateral to the arm being used for the task. The implant site was pre-planned and finally determined during the surgery with reference to the sulcal patterns and the muscle contractions evoked by intraoperative surface cortical stimulation. For each of monkeys J, S, G, and P, we also implanted intramuscular leads in forearm and hand muscles of the arm used for the task in a separate procedure. Electrode locations were verified during surgery by stimulating each lead.

Behavioral task:

Monkeys J and S were trained to perform an isometric wrist task, which required them to control the cursor on the screen by exerting forces on a small box placed around one of the hands. The box was padded to comfortably constrain the monkey’s hand and minimize its movement within the box, and the forces were measured by a 6 DOF load cell (JR3 Inc., CA) aligned to the wrist joint. During the task, flexion/extension force moved the cursor right and left respectively, while force along the radial/ulnar deviation axis moved the cursor up and down. Each trial started with the appearance of a center target requiring the monkeys to hold for a random time (0.2 – 1.0 s), after which one of eight possible outer targets selected in a block-randomized fashion appeared, accompanied with an auditory go cue. The monkey was allowed to move the cursor to the target within 2.0 s and hold for 0.8 s to receive a liquid reward. For both decoding and alignment analyses, we only used the data within each single trial (from ‘trial start’ to ‘trial end’). We did not do any temporal alignment with the trials, so the lengths of the trials were different from each other.

Monkeys P and G were trained to perform a grasping task, which required them to reach and grasp a gadget placed under the screen with one hand. The gadget was a cylinder for monkey P facilitating a power grasp with the palm and the fingers, while a small rectangular cuboid for monkey G facilitating a key grasp with the thumb and the index finger. A pair of force sensitive resistors (FSRs) were attached on the sides of the gadgets to measure the grasping forces the monkeys applied. The sum and the difference of the FSR outputs were used to determine the position of the cursor on the vertical axis and the horizontal axis respectively. At the beginning of each trial the monkey was required to keep the hand resting on a touch pad for a random time (0.5 – 1.0 s). A successful holding triggered the onset of one of three possible rectangular targets on the screen and an auditory go cue. The monkey was required to place the cursor into the target and hold for 0.6 s by increasing and maintaining the grasping force applied on the gadget. For this task we extracted trials from ‘gocue time’ to ‘trial end’, as the monkeys’ movements were quite random before the gocue.

Monkeys C and M were trained to perform a center-out (CO) reaching task while grasping the upright handle of a planar manipulandum, operated with the upper arm in a parasagittal plane. Monkey C performed the task with the right hand, monkey M with the left. At the beginning of each trial the monkey needed to move the hand to the center of the workspace. One of eight possible outer targets equally spaced in a circle was presented to the monkey after a random waiting period. The monkey needed to keep holding for a variable delay period until receiving an auditory go cue. To receive a liquid reward, the monkey was required to reach the outer target within 1.0 s and hold within the target for 0.5 s. For this task we extracted trials from ‘gocue time’ to ‘trial end’, since the monkeys kept static before the gocue.

Monkey M was also trained to perform a random-target (RT) task, reaching a sequence of three targets presented in random locations on the screen to complete a single trial. The RT task used the same apparatus as the CO reach task. At the beginning of each trial the monkey also needed to move the hand to the center of the workspace. Three targets were then presented to the monkey sequentially, and the monkey was required to move the cursor into each of them within 2.0 s after viewing each target. The positions of these targets were randomly selected, thus the cursor trajectory for each trial presented a ‘random-target’ manner. For this task we extracted trials from ‘trial start’ to ‘trial end’.

All surgical and experimental procedures were approved by the Institutional Animal Care and Use Committee (IACUC) of Northwestern University under protocol #IS00000367, and are consistent with the Guide for the Care and Use of Laboratory Animals.

Data collection and preprocessing:

M1 activity was recorded during task performance using a Cerebus system (Blackrock Neurotech, Inc.). The signals on each channel were digitalized, bandpass filtered (250 ~ 5000 Hz) and converted to spike times based on threshold crossings. The threshold was set with respect to the root-mean square (RMS) activity on each channel and kept consistent across different recording sessions (monkeys J, C and M: -5.5 x RMS; monkey S: -6.25 x RMS; monkey P: -4.75 x RMS; monkey G: -5.25 x RMS). The time stamp and a 1.6 ms snippet of each spike surrounding the time of threshold crossing were recorded. For all analyses in this study, we used multiunit threshold crossings on each channel instead of discriminating well isolated single units. We applied a Gaussian kernel (S.D. = 100 ms) to the spike counts in 50 ms, non-overlapping bins to obtain a smoothed estimate of firing rate as function of time for each channel.

The EMG signals were differentially amplified, band-pass filtered (4-pole, 50 ~ 500 Hz) and sampled at 2000 Hz. The EMGs were subsequently digitally rectified and low-pass filtered (4-pole, 10 Hz, Butterworth) and subsampled to 20 Hz. EMG channels with substantial noise were not included in the analyses. For monkeys C and M, we recorded the positions of the endpoint of the reach manipulandum at a sampling frequency of 1000 Hz using encoders in the two joints of the manipulandum.

Description of data and file structure

Summary of files and naming conventions:

This dataset contains data files from 6 monkeys and 4 types of behavioral tasks (see above). The data files from one monkey/task were compressed into a .7z file, which could be opened and extracted by '7-Zip'. The .7z files are named like 'Name_Task_XXXX.7z', where 'Name' is the full name of the monkey ('Jango': monkey J; 'Spike': monkey S; 'Greyson': monkey G; 'Pop': monkey P; 'Chewie': monkey C; 'Mihili': monkey M), 'Task' indicates the behavioral task ('ISO': isometric wrist task; 'PG': power grasp task; 'Key': key grasp task; 'CO': center-out reach task; 'RT': random-target reach task), and 'XXXX' indicates the year when the data files were recorded.

Data files could be extraxted from the .7z files once they are downloaded. Please check this document to see more details about them.

For monkeys J, S, G, C and M, neural and behavioral data are organized in a data structure called xds (cross-platform data structure) and saved in MATLAB '.mat' format. The naming convention is like 'Name_YYYYMMDD_XXX.mat', where 'Name' is the name of the monkey, 'YYYYMMDD' shows the date when the recording session was made, and 'XXX' is the serial number of the recording session on a day (like '001'). These files could be directly opened in MATLAB, so you can develop your own MATLAB codes to analyze them. At the same time, we provide **Python codes ('xds', https://github.com/limblab/xds) **to load these .mat files in Python (h5py and scipy are needed). A tutorial for xds can be found here. In summary, when analyzing these data, there is no need to write codes from scratch by yourself using h5py or scipy to load the data files , since xds has already done it for you.

Specifically, the xds where all data are encapsulated is a MATLAB struct with 20+ fields. These fields are consistent in MATLAB and Python. Some fields may not be present in some datasets because the corresponding types of data were not recorded or unavailable. Details are explained below.

'meta' is a struct containing the essential information of the recording session.

'spikes' is a 1 by N cell array containing the spike timings on each of the N cortical electrodes. Normally, N should be 96 because there are 96 electrodes on an Utah array. However, an electrode may be skipped in file writing if zero spike is detected on it in a recording session. Therefore, N is less than 96 in some data files. We've demonstrated how to do zero padding for the convinience of data analysis in this notebook.

'spike_counts' is a T by N array for the binned spike counts for the N electrodes, where T is the total number of bins determined by the length of the recording session. Apparently, 'spike_counts' is computed from 'spikes'. The default bin width is 1 ms.

'bin_width' gives the default bin width for the data files, which is 1 ms.

'spike_waveforms' is a 1 by N cell array containing the waveforms of detected spikes. Each element in the cell array corresponds to a cortical electrode. Each spike waveform has 48 data samples. In some datasets (Chewie_CO_2016 and Mihili_CO_2014) the spike waveforms are not provided so this field is not present in the data files.

'unit_names' are actually the names of the electrodes, since no spike sorting has been done to identify single units for this data set. The ordering of 'unit_names' corresponds to the columns of 'spike_counts', as well as the elements of 'spikes' and 'spike_waveforms'

'EMG' contains the EMG signals. They have been resampled to 1000 Hz following the 1 ms default bin width and synchronized with 'spike_counts'. Please check the codes for the processings steps in our data analysis here. Specifically, the data points from each EMG channel were clipped to be no larger than the mean plus 6 times the S.D. of that channel. Within each session, we removed the baseline of each EMG channel by subtracting the 2nd percentile of the amplitudes and normalized each channel to the 90th percentile. But please note in the data files you download the EMGs were not processed, no clipping, no normalizing and no baseline removal.

'EMG_names' give the names of the muscles where EMGs were recorded from, corresponding to the columns of 'EMG'.

'force' contains the force signals measured by the JR3 load cell when monkey performing the tasks. It is a 2-D array for x-axis and y-axis respectively, and has been resampled to 1000 Hz and also synchronized with 'spike_counts'.

'curs_p', 'curs_v' and 'curs_a' are the positions, velocities, and accelerations of the cursors on the screen controled by the monkey through the device during the task, also resampled to 1000 Hz and synchronized with 'spike_counts'.

'time_frame' gives the time frame of data samples. It has the same length as 'spike_counts', 'EMG', 'force' and cursors, following the 'bin_width'.

'trial_info_table' and 'trial_info_table_header' contain a collection of all trial related information. Since we extracted important items and made them as stand-alone fields in the data structure, it is not necessary to visit these two fields.

'trial_start_time', 'trial_gocue_time', 'trial_end_time', 'trial_result' provide trial informations and the names are quite self-explanatory. The unit for time is second.

'trial_target_dir' gives the direction of the target on the screen for each trial in degrees.

'trial_target_corners' provides the position of the target on the screen for each trial.

'has_EMG', 'has_cursor', 'has_force' use values 0 or 1 to indicate if the corresponding type of data is present in a data file. In 'Greyson_Key_2019' cursor data is not available during recordings so has_cursor is 0 and fields curs_p, curs_v, curs_a are absent. For 'Mihili_CO_2014', 'Mihili_RT_2013_2014', and 'Chewie_CO_2016', both has_force and has_EMG are 0 and fields EMG, EMG_names, force are absent since forces and EMGs were not recorded at all.

The timings for force onset were not provided directly in the data structure, but can be computed using the functions we provided in xds, please check the tutorial for xds for more details.

For monkey P, only trial-splitted data were provided. The data files are saved in .pkl format, and can be loaded in Python using pickle. The data files are named as 'Pop_YYYYMMDD_pg.pkl', where 'Pop' is the name of the monkey, 'YYYYMMDD' gives the date when the data file was recorded, and 'pg' indicates the behavioral task is power grasp. Please use the codes in 'load_monkey_pop_data.py' compressed with monkey P data files to load them. Specifically, when using those codes to load one data file, three lists could be obtained, which are spike_counts, EMG, and force. Each element in the lists is a numpy array corresponding to a trial. Please note the EMG data in Pop_PG_2021 were processed. The data points from each EMG channel were clipped to be no larger than the mean plus 6 times the S.D. of that channel. Within each session, we removed the baseline of each EMG channel by subtracting the 2nd percentile of the amplitudes and normalized each channel to the 90th percentile.

Besides the paper published with eLife, we also provide the codes of our method to align the neural signals for BCI stabilization, which can be found here.