Plug and play stability for intracortical brain-computer interfaces: A one-year demonstration of seamless brain-to-text communication
Data files
Nov 06, 2023 version files 3.57 GB
-
CORP_data_release.zip
3.57 GB
-
README.md
4.40 KB
Abstract
Intracortical brain-computer interfaces (iBCIs) have shown promise for restoring rapid communication to people with neurological disorders such as amyotrophic lateral sclerosis (ALS). However, to maintain high performance over time, iBCIs typically need frequent recalibration to combat changes in the neural recordings that accrue over days. In this study, we propose a method: Continual Online Recalibration with Pseudo-labels (CORP), that enables self-recalibration of communication iBCIs without interrupting the user. We evaluated CORP with one clinical trial participant. CORP achieved a stable decoding accuracy of 93.84% in an online handwriting iBCI task, significantly outperforming other baseline methods.
This dataset contains 21 sessions of recorded neural activities used for the evaluation. It has been formatted for developing and evaluating machine learning models. There 5 more sessions heldout for a planned iBCI stability competition. They will be released in the future.
We also provide a pretrained RNN seed model and a laugnage model to preproduce the results in our paper.
https://neurips.cc/virtual/2023/poster/71605
Brain-computer interfaces (BCIs) can restore communication to people who have lost the ability to move or speak. However, BCIs typically require frequent recalibration to maintain high-performance. We proposed a new method: Continual Online Recalibration with Pseudo-labels (CORP) that can automatically recalibrate handwriting BCIs without interrupting the user. We demonstrated CORP on a handwriting BCI for a person with tetraplegia for one year. The BCI was able to maintain high performance for 57 weeks without manual recalibration. This repository contains the data for reproducing the results in the paper.
Description of the data and file structure
Neural Data
The neural data are organized into two folders: seed_model_training_data, and online_evaluation_data. The seed_model_training_data folder contains the data used to train the seed model. The online_evaluation_data folder contains the data used to evaluate CORP online.
Seed Model Training Data
We collected 11 sessions of instructed delay sentence copying task data from participant T5. In each session, T5 was instructed to copy sentence on a compunter screen while we recorded his brain activity. The data were recording using two 96-channel Blackrock Utah Arrays. Data was collected in a series of ‘blocks’ (10-20 trials in each block), in between which T5 rested. On each trial, T5 first saw a red square with a sentence above it. Then, when the square turned green, T5 attempted to copy the sentence by handwriting it letter by letter.
The ‘mat’ subfolder contains the data for the instructed delay sentence copying task. Each file ‘mat’ file is one session, containing the following fields:
- `tx_feats’: list of 2D arrays. Each 2D array is the binned (20ms) threshold crossing features (-4.5 RMS) for one trial during go period. The 2D array has shape (n_timesteps, n_channels).
- `sentences’: list of strings. Each string is the sentence that T5 was instructed to copy on one trial.
- `blocks’ : list of integers. Each integer is the block number that the trial belongs to.
The `tfrecord’ folder contains the same data as the mat folder, but converted to tfrecord format for training the seed model.
Online Evaluation Data
We colected 15 sessions of instructed delay sentence copying task data from T5. The task has the same structure as the seed model training data collection task, except that T5 could see real-time decoded results while copying. In each session, we ran two kinds of blocks. In the no-recalibration block, the neural activity was decoded with a fronzen seed model. In the recalibration block, we used CORP to continually update the decoder model. The `no_recalibration and recalibration’ subfolders contain the data for each kind of block.
In the `no_recalibration’ and ‘recalibration’ folders, the data is organzied the same as the seed_model_training_data folder. Each file is one session, containing the following fields:
- `tx_feats’: list of 2D arrays. Each 2D array is the binned (20ms) threshold crossing features (-4.5 RMS) for one trial during go period. The 2D array has shape (n_timesteps, n_channels).
- `sentences’ : list of strings. Each string is the sentence that T5 was instructed to copy on one trial.
- `blocks’ : list of integers. Each integer is the block number that the trial belongs to.
- `rnn_decoded’ : list of strings. Each string is the raw RNN decoded sentence for one trial.
- `lm_decoded’ : list of strings. Each string is the language model decoded sentence for one trial.
Note that we only release the first 10 sessions in the version. The remaining 5 sessions will be used for a competition and will be released later.
Additionally, we offer two files to help reproduce the results in the paper:
Pretrained Seed Model
This folder contains a pretrained seed model in Tensorflow. See repo about its usage.
Launguage Model
This folder contains the 3-gram language model used during online evaluation. See repo about its usage.
Code/Software
This repo contains code for reproducing the results in the paper.