Restoring rapid natural bimanual typing with a neuroprosthesis after paralysis
Data files
Jan 12, 2026 version files 38.02 GB
-
Data.zip
19.48 GB
-
LanguageModels.zip
18.55 GB
-
README.md
22.14 KB
Abstract
Recognizing keyboard typing as a familiar, high information rate communication paradigm, we developed an intracortical brain computer interface (iBCI) typing neuroprosthesis providing bimanual QWERTY keyboard functionality for people with paralysis. Typing with this iBCI involves only attempted finger movements, which are decoded accurately with as few as 30 calibration sentences. Sentence decoding is improved using a 5-gram language model. This typing neuroprosthesis performed well for two iBCI clinical trial participants with tetraplegia - one with ALS and one with spinal cord injury. Typing speed is user-regulated, reaching 110 characters per minute, resulting in 22 words per minute with a word error rate of 1.6 %. This resembles able-bodied typing accuracy and provides higher throughput than current state-of-the-art hand motor iBCI decoding. In summary, a typing neuroprosthesis decoding finger movements provides an intuitive, familiar, and easy-to-learn paradigm for individuals with impaired communication due to paralysis. This dataset contains all of the neural activity recorded during these experiments, consisting of data from two BrainGate2 Clinical Trial participants. The neural activity was recorded with six microlectrode arrays implanted in the motor cortex of each participant. The dataset also contains all of the real-time outputs of the typing iBCI.
Overview
These data are reported in Jude et al. 2025 and consist of separate sessions (listed below in “Session Table”) across two BrainGate2 clinical trial participants.
All data files are .mat files that can be loaded with MATLAB or Python (using the scipy function scipy.io.loadmat).
Accompanying code is available at https://github.com/justin-jude/typingBCI.
Trial-by-trial neural features and corresponding ground truth sentence pairs can be extracted from .mat files using the following function: https://github.com/justin-jude/typingBCI/blob/main/utils/mat_to_tfrecord_analysis.py
Blocks run in each session are listed below per participant:
Participant T17 Typing
t17.2024.04.24 (Trial day 55)
| Block Number | Block Description | Number of Trials |
|---|---|---|
| 1 | Isolated Finger Movements | 62 |
| 2 | Isolated Finger Movements | 62 |
| 3 | Isolated Finger Movements | 62 |
| 6 | Isolated Finger Movements | 62 |
| 7 | Isolated Finger Movements | 62 |
| 8 | Isolated Finger Movements | 62 |
| 9 | Isolated Finger Movements | 62 |
| 10 | Isolated Finger Movements | 62 |
t17.2024.04.25 (Trial day 56)
| Block Number | Block Description | Number of Trials |
|---|---|---|
| 1 | Calibration Sentences | 20 |
| 3 | Calibration Sentences | 20 |
| 4 | Calibration Sentences | 20 |
| 5 | Calibration Sentences | 20 |
t17.2024.06.03 (Trial day 95)
| Block Number | Block Description | Number of Trials |
|---|---|---|
| 1 | Isolated Finger Movements | 62 |
| 2 | Isolated Finger Movements | 62 |
| 3 | Isolated Finger Movements | 62 |
| 4 | Calibration Sentences | 20 |
| 5 | Calibration Sentences | 20 |
| 7 | Calibration Sentences | 20 |
| 8 | Calibration Sentences | 20 |
| 9 | Calibration Sentences | 20 |
| 11 | Calibration Sentences | 20 |
| 12 | Sentence Copy Sentences | 20 |
t17.2024.06.04 (Trial day 96)
| Block Number | Block Description | Number of Trials |
|---|---|---|
| 1 | Calibration Sentences | 20 |
| 2 | Calibration Sentences | 20 |
| 3 | Calibration Sentences | 20 |
| 5 | Calibration Sentences | 20 |
| 6 | Sentence Copy Sentences | 20 |
| 8 | Free Typing Sentences | 7 |
| 9 | Free Typing Sentences | 4 |
| 10 | Free Typing Sentences | 11 |
| 11 | Free Typing Sentences | 12 |
t17.2024.06.05 (Trial day 97)
| Block Number | Block Description | Number of Trials |
|---|---|---|
| 1 | Calibration Sentences | 20 |
| 2 | Sentence Copy Sentences | 20 |
| 3 | Free Typing Sentences | 11 |
| 5 | Calibration Sentences | 20 |
| 6 | Calibration Sentences | 20 |
t17.2024.06.13 (Trial day 105)
| Block Number | Block Description | Number of Trials |
|---|---|---|
| 7 | Calibration Sentences | 20 |
| 8 | Calibration Sentences | 20 |
t17.2024.07.09 (Trial day 131)
| Block Number | Block Description | Number of Trials |
|---|---|---|
| 1 | Calibration Sentences | 20 |
| 2 | Calibration Sentences | 20 |
| 3 | Calibration Sentences | 20 |
| 4 | Free Typing Sentences | 2 |
| 6 | Calibration Sentences | 20 |
| 7 | Free Typing Sentences | 8 |
| 8 | Free Typing Sentences | 4 |
| 10 | Free Typing Sentences | 4 |
| 12 | Free Typing Sentences | 4 |
| 13 | Free Typing Sentences | 6 |
t17.2024.07.10 (Trial day 132)
| Block Number | Block Description | Number of Trials |
|---|---|---|
| 1 | Calibration Sentences | 20 |
| 2 | Calibration Sentences | 20 |
| 3 | Free Typing Sentences | 8 |
| 4 | Free Typing Sentences | 7 |
t17.2024.07.22 (Trial day 144)
| Block Number | Block Description | Number of Trials |
|---|---|---|
| 1 | Calibration Sentences | 20 |
| 2 | Calibration Sentences | 20 |
| 3 | Calibration Sentences | 20 |
| 4 | Free Typing Sentences | 11 |
| 5 | Free Typing Sentences | 5 |
Participant T18 Typing
t18.2024.12.03 (Trial day 43)
| Block Number | Block Description | Number of Trials |
|---|---|---|
| 1 | Isolated Finger Movements | 62 |
| 2 | Isolated Finger Movements | 62 |
| 3 | Isolated Finger Movements | 62 |
| 4 | Isolated Finger Movements | 62 |
| 5 | Isolated Finger Movements | 62 |
| 6 | Isolated Finger Movements | 62 |
t18.2024.12.04 (Trial day 44)
| Block Number | Block Description | Number of Trials |
|---|---|---|
| 2 | Isolated Finger Movements | 62 |
| 3 | Isolated Finger Movements | 62 |
| 5 | Calibration Sentences | 20 |
| 6 | Calibration Sentences | 20 |
| 7 | Calibration Sentences | 20 |
t18.2024.12.05 (Trial day 45)
| Block Number | Block Description | Number of Trials |
|---|---|---|
| 1 | Calibration Sentences | 20 |
| 2 | Calibration Sentences | 20 |
| 3 | Sentence Copy Sentences | 20 |
| 4 | Sentence Copy Sentences | 20 |
| 5 | Free Typing Sentences | 5 |
| 6 | Sentence Copy Sentences | 20 |
t18.2025.01.14 (Trial day 85)
| Block Number | Block Description | Number of Trials |
|---|---|---|
| 1 | Calibration Sentences | 20 |
| 2 | Calibration Sentences | 20 |
| 3 | Sentence Copy Sentences | 20 |
| 4 | Sentence Copy Sentences | 20 |
| 5 | Sentence Copy Sentences | 20 |
| 6 | Sentence Copy Sentences | 20 |
| 7 | Sentence Copy Sentences | 20 |
| 8 | Sentence Copy Sentences | 20 |
| 9 | Sentence Copy Sentences | 20 |
| 10 | Sentence Copy Sentences | 20 |
| 11 | Sentence Copy Sentences | 20 |
t18.2025.01.15 (Trial day 86)
| Block Number | Block Description | Number of Trials |
|---|---|---|
| 1 | Calibration Sentences | 20 |
| 2 | Sentence Copy Sentences | 20 |
| 3 | Calibration Sentences | 20 |
| 5 | Calibration Sentences | 20 |
| 6 | Calibration Sentences | 20 |
| 7 | Sentence Copy Sentences | 20 |
| 8 | Sentence Copy Sentences | 20 |
| 9 | Sentence Copy Sentences | 20 |
| 10 | Sentence Copy Sentences | 20 |
| 11 | Sentence Copy Sentences | 20 |
| 12 | Sentence Copy Sentences | 20 |
| 13 | Sentence Copy Sentences | 20 |
| 14 | Sentence Copy Sentences | 20 |
t18.2025.01.21 (Trial day 92)
| Block Number | Block Description | Number of Trials |
|---|---|---|
| 1 | Calibration Sentences | 20 |
| 2 | Calibration Sentences | 20 |
| 3 | Calibration Sentences | 20 |
| 4 | Sentence Copy Sentences | 20 |
| 5 | Sentence Copy Sentences | 20 |
| 6 | Sentence Copy Sentences | 20 |
| 8 | Sentence Copy Sentences | 20 |
| 9 | Sentence Copy Sentences | 20 |
| 10 | Sentence Copy Sentences | 20 |
| 11 | Sentence Copy Sentences | 20 |
t18.2025.01.22 (Trial day 93)
| Block Number | Block Description | Number of Trials |
|---|---|---|
| 1 | Calibration Sentences | 20 |
| 2 | Calibration Sentences | 20 |
| 3 | Calibration Sentences | 20 |
| 5 | Sentence Copy Sentences | 20 |
| 6 | Sentence Copy Sentences | 20 |
| 8 | Sentence Copy Sentences | 20 |
| 9 | Sentence Copy Sentences | 20 |
| 10 | Sentence Copy Sentences | 20 |
| 11 | Sentence Copy Sentences | 20 |
| 12 | Sentence Copy Sentences | 20 |
| 13 | Sentence Copy Sentences | 20 |
t18.2025.02.04 (Trial day 106)
| Block Number | Block Description | Number of Trials |
|---|---|---|
| 1 | Calibration Sentences | 20 |
| 2 | Calibration Sentences | 20 |
| 3 | Calibration Sentences | 20 |
| 4 | Sentence Copy Sentences | 20 |
| 5 | Free Typing Sentences | 11 |
| 6 | Sentence Copy Sentences | 20 |
| 7 | Sentence Copy Sentences | 20 |
| 8 | Sentence Copy Sentences | 20 |
| 9 | Sentence Copy Sentences | 20 |
| 10 | Sentence Copy Sentences | 20 |
| 11 | Sentence Copy Sentences | 20 |
| 12 | Sentence Copy Sentences | 20 |
| 13 | Sentence Copy Sentences | 20 |
| 14 | Sentence Copy Sentences | 20 |
t18.2025.02.05 (Trial day 107)
| Block Number | Block Description | Number of Trials |
|---|---|---|
| 1 | Calibration Sentences | 20 |
| 2 | Calibration Sentences | 20 |
| 3 | Calibration Sentences | 20 |
| 4 | Sentence Copy Sentences | 20 |
| 5 | Sentence Copy Sentences | 20 |
| 6 | Sentence Copy Sentences | 20 |
| 8 | Sentence Copy Sentences | 20 |
| 9 | Sentence Copy Sentences | 20 |
| 10 | Sentence Copy Sentences | 20 |
| 11 | Sentence Copy Sentences | 20 |
| 12 | Sentence Copy Sentences | 20 |
| 13 | Sentence Copy Sentences | 20 |
| 14 | Sentence Copy Sentences | 20 |
| 15 | Sentence Copy Sentences | 20 |
| 16 | Sentence Copy Sentences | 20 |
Participant T18 Handwriting
t18.2025.04.01 (Trial day 163)
| Block Number | Block Description | Number of Trials |
|---|---|---|
| 3 | Calibration Sentences | 30 |
| 4 | Calibration Sentences | 10 |
| 5 | Calibration Sentences | 10 |
| 6 | Calibration Sentences | 10 |
| 7 | Calibration Sentences | 10 |
| 8 | Calibration Sentences | 10 |
| 9 | Calibration Sentences | 10 |
| 10 | Calibration Sentences | 10 |
| 11 | Calibration Sentences | 10 |
| 12 | Calibration Sentences | 10 |
| 13 | Calibration Sentences | 10 |
Dataset Description
Neural voltage time series signals were recorded and digitized (30 kHz, 16 bits per sample) using the Neuroplex-E system (Blackrock Neurotech) attached to three percutaneous connectors on each participant’s head, and transmitted via three mini-HMDI cables (one to each Neuroplex headstage), attached to two Gemini hubs (Blackrock Neurotech), prior to final processing via a Neural Signal Processor (NSP) (Blackrock Neurotech). Packets of neural data were streamed from the NSP to our processing pipeline. Signals were analog filtered (4th order Butterworth with corners at 250 Hz to 5 kHz) using the Scipy python library (scipy.signal.filtfilt).
Electrode-specific thresholds were calculated using filtered 30kHz data once these calculated references had been applied to identify spike events. Thresholds were set at -3.5 times the standard deviation of the voltage signal per electrode. The number of non-causal threshold crossings (ncTX) were computed by counting the number of times the filtered neural time series crossed these calculated thresholds (threshold crossing rates). Additionally, spike band power was computed by taking the sum of squared voltages observed during each 10ms time bin.
T17 had six 64-electrode microelectrode arrays (Blackrock Microsystems; 1.5 mm electrode length) placed in the left precentral gyrus; two arrays were placed in the dorsal precentral gyrus (area 6d), two arrays were placed in the ventral precentral gyrus (area 6v), and two arrays were placed in middle precentral gyrus (area 55b).
T18 had four 64-electrode microelectrode arrays (Blackrock Microsystems; 1.5 mm electrode length) placed in the dorsal precentral gyrus (area 6d) of the left (motor dominant) hemisphere and two 64-electrode microelectrode arrays placed in the dorsal precentral gyrus (area 6d) of the right hemisphere.
Isolated Finger Movement blocks pertain to individual trials of 30 attempted finger movements.
Calibration Sentences blocks are open-loop sentence copy trials where each trial had the participant attempt to type the whole sentence cued on-screen, without decoded output appearing on-screen.
Sentence Copy blocks are open-loop sentence copy trials where each trial had the participant attempt to type the whole sentence cued on-screen without decoded output appeared on-screen in real time.
Free Typing Sentences are unconstrained closed-loop decoding trials where the participant can type any sentence they wish. Decoding of each trial ends when the participant stops typing.
Variables in .mat files
participant: Indicated whether the session participant is T17 or T18.
session_name: Date of session.
session_description: Indicates a typing session.
block_number: Block number corresponding to those listed above.
block_description: Indicates a typing or handwriting session.
binned_neural_threshold_crossings: Number of threshold crossings observed in each 10ms bin.
binned_neural_spike_band_power: Mean averaged sum of square voltages in each 10ms bin.
binned_neural_nsp_timestamp: Timestamp in nanoseconds assigned to each 10ms bin by the Neural Signal Processor.
binned_neural_redis_clock: Timestamp in milliseconds assigned to each 10ms bin by BRAND.
cue : Ground Truth sentence in calibration sentence and sentence copy trials. Entered manually for Free Typing trials after confirming ground truth sentence with participant.
decoder_logit_output: In Sentence Copy and Free Typing trials, the RNN output probabilities for each character. The most likely of these is highlighted in green on the on-screen keyboard for real-time feedback. Predictions occur every 20 milliseconds (i.e., every other index).
decoder_signal: Flag indicating start or end of decoding RNN decoding loop.
decoder_output_redis_clock: Timestamp in milliseconds corresponding to each decoder logit output.
ngram_decoder_partial_output: Real-time 5-gram language model ouput, updated continuously as sentence is typed throughout a trial.
ngram_decoder_final_output: Final decoded sentence from 5-gram language model for each trial.
norm_channel_means: Mean values of all 768 features (T18) or 512 features (T17) used for decoding - used for online z-scoring to maintain stability.
norm_channel_stds: Standard deviations of values of all 768 features (T18) or 512 features (T17) used for decoding - used for online z-scoring to maintain stability.
norm_redis_times: Redis timestamps for z-scoring Redis stream.
delay_duration_ms: Time in milliseconds when cued sentence is shown before typing begins.
inter_trial_duration_ms: Time in milliseconds between trials.
trial_start_redis_time: Timestamp in milliseconds signalling the start of a trial before a cue is shown in the delay period, assigned by BRAND.
trial_start_nsp_neural_time: Timestamp in nanoseconds signalling the start of a trial before a cue is shown in the delay period, assigend by the Neural Signal Processor.
trial_start_nsp_analog_time: Timestamp in nanoseconds signalling the start of a trial before a cue is shown in the delay period, assigned by the analog microphone component of the Neural Signal Processor.
go_cue_redis_time: Timestamp in milliseconds signalling to the participant to start typing after the initial delay period, assigned by BRAND.
go_cue_nsp_neural_time: Timestamp in nanoseconds signalling to the participant to start typing after the initial delay period, assigned by the Neural Signal Processor.
go_cue_nsp_analog_time: Timestamp in nanoseconds signalling to the participant to start typing after the initial delay period, assigned by the analog microphone component of the Neural Signal Processor.
trial_end_redis_time: Timestamp in milliseconds indicating the end of a trial, assigned by BRAND. A trial is ended in Calibration and Sentence Copy trials through a button press. Free Typing trials are ended through successive predictions of blank tokens.
trial_end_nsp_neural_time: Timestamp in nanoseconds indicating the end of a trial, assigned by the Neural Signal Processor.
trial_end_nsp_analog_time: Timestamp in nanoseconds indicating the end of a trial, assigned by the analog component of the Neural Signal Processor.
microphone_data: Microphone audio recorded during trials (T17 only).
microphone_nsp_time: Microphone audio timestamps in nanoseconds.
Language Model
LanguageModels.zip contains the 5-gram character based language model used to produce sentences from RNN character probabilities. This is used in the analysis scripts to obtain a final word error rate for decoding of finger movements.
Install the WeNet based Kaldi language modelling software from here first (just the third step): https://github.com/fwillett/speechBCI/tree/main/LanguageModelDecoder
Extract Data.zip and LanguageModels.zip to the same folder as the software repo (https://github.com/justin-jude/typingBCI) as this is where the analysis scripts will search for both the data and the language model.
Human subjects data
No personally identifiable information (PII) is contained within this data. Only neural features and typed sentences are present. Sentences typed in independent use mode have been screened for personal information and any PII has been removed from these.