Data from: Neural dynamics of the processing of speech features: Evidence for a progression of features from acoustic to sentential processing
Data files
Jan 31, 2025 version files 52.46 GB
-
behavioral_responses.zip
13.19 KB
-
Codes.zip
11.11 MB
-
meg.zip
51.98 GB
-
predictors.zip
100.55 MB
-
README.md
6.28 KB
-
Stimuli.zip
36.05 MB
-
trfs.zip
333.28 MB
Abstract
When we listen to speech, our brain’s neurophysiological responses “track” its acoustic features, but it is less well understood how these auditory responses are enhanced by linguistic content. We recorded magnetoencephalography (MEG) responses while subjects listened to four types of continuous-speech-like passages: speech-envelope modulated noise, English-like non-words, scrambled words, and a narrative passage. Temporal response function (TRF) analysis provides strong neural evidence for the emergent features of speech processing in cortex, from acoustics to higher-level linguistics, as incremental steps in neural speech processing. Critically, we show a stepwise hierarchical progression of progressively higher order features over time, reflected in both bottom-up (early) and top-down (late) processing stages. Linguistically driven top-down mechanisms take the form of late N400-like responses, suggesting a central role of predictive coding mechanisms at multiple levels. As expected, the neural processing of lower-level acoustic feature responses is bilateral or right lateralized, with left lateralization emerging only for lexical-semantic features. Finally, our results identify potential neural markers, linguistic level late responses, derived from TRF components modulated by linguistic content, suggesting that these markers are indicative of speech comprehension rather than mere speech perception.
This dataset includes raw MEG (magnetoencephalography) data, behavioral responses, stimuli, predictors, main codes, some intermediate results (Temporal response functions (TRFs), features extracted from TRFs), and statistical analysis codes related to the above findings, paper titled "Neural Dynamics of the processing of speech features: Evidence for a Progression of Features from Acoustic to Sentential Processing"
https://doi.org/10.5061/dryad.ht76hdrpf
The four passage types used in the study are abbreviated as follows.
- ASM - Scrambled word passage
- ASN - Speech modulated noise passages
- PH - Nonword passages
- AS - Story passage
Description of the data and file structure
stimuli.zip
- wav files - stimuli (.wav) used in the study.
- transcripts - transcripts of .wav files in 'wav files' folder
- acoustic comparisons - Acoustic comparison analysis and results plotted in Figure S5
predictors.zip
predictors (Gammatone envelope (gammatonelog-8), gammatone envelope onset (gammatonelog-on-8), phoneme onsets (pho), word onsets (wor), phoneme surprisal (psr1), cohort entropy (pen), unigram word surprisal (wfr) and GPT2 word surprisal (hug)) used for TRF modeling. Each file contains the speech features their amplitudes and latencies.
Naming convention is [passagetype]_M0[stimulinumber]|[predictor].pickle
meg.zip
Each folder contains the meg data for each subject and emptyroom measurements. All files are .fiff files. The .fiff files are data recorded from MEG KIT at the University of Maryland College Park (https://linguistics.umd.edu/resources-facilities/labs/KIT-Maryland-MEG-Lab). The data can be viewed in python using mne python package (https://mne.tools/stable/generated/mne.io.show_fiff.html)
Codes.zip
Main codes (experiment_class using Eelbrain (https://eelbrain.readthedocs.io/en/stable/reference.html#module-pipeline) (exp.py), and Jupyter notebooks for TRF computation, TRF feature extraction, and prediction accuracy comparisons. Additionally, this folder includes statistical analysis codes and results figures
1. exp.py sets up the experiment pipeline for this specific dataset along with other pre-processing options, TRF modeling, epoch extraction and parcellation.
2. figure2 - Results figures for Fig 2, generated from 'prediction accuracy.ipynb'
3. prediction accuracy.html - 'prediction accuracy.ipynb' saved as .html
4. prediction accuracy.ipynb - Once the exp.py is setup properly, 'prediction accuracy.ipynb' can be run. 'prediction accuracy.ipynb' gives example codes for prediction accuracy comparisons in Table S1.
5. R analysis - R codes used for statistical analysis, plotting, and predictor comparisons between passage types
6. TRFs - Includes the features extracted from Temporal Response Functions (TRFs) (P1 : early positive polarity peak and N1 : Late negative polarity peak, peak amplitudes (normalized magnitudes, arbitrary unit) and latencies in (ms)) for each subject, brain hemisphere, speech condition, and speech feature.
Naming convention for .csv files are [speech feature]_[hemi]ftp.csv. Where speech feature is either (Gammatone envelope (ENV), gammatone envelope onset (ENVON), phoneme onsets (PHO), word onsets (wor), phoneme surprisal (PSR), cohort entropy (PEN), unigram word surprisal (WFR) and GPT2 word surprisal (HUG)), and hemi is either (left (lh) or right (rh))
The variables in tables (.csv) files are as follows
subject = Subject ID
-
[passagetype] = Includes the TRFs
-
[passagetype]_P1 = amplitude of early peak (arbitrary unit)
-
[passagetype]_P2 = amplitude of middle peak (arbitrary unit)
-
[passagetype]_N1 = amplitude of late peak (arbitrary unit)
-
[passagetype]_P1_latency = Latency of early peak (ms)
-
[passagetype]_P2_latency = Latency of middle peak (ms)
-
[passagetype]_N1_latency = Latency of late peak (ms)
-
mask = ROIS (ftp = Frontal Temporal Parietal)
-
hemi = hemisphere (rh=right hemisphere, lh = left hemisphere)
'figs' folder includes the figures layered in Fig 3,4,5 and Figures S1, S2, S3. Each folder within fig includes the figures for each speech feature (ENV, ENVON, HUG,PEN, PHO,PSR,WFR, and WOR) The naming convention for .eps files inside each of those folder is
[speech feature]_[hemi]ftp_[peak].eps. Where speech feature and hemi are same as defined above and peak is either (early (P1), middle (P2) or late (N1))peak_latency.eps includes the figure for Figure 6B in the paper
7. TRFs example.html - 'TRFs example.ipynb' saved as .html
8. TRFs example.ipynb - Example code to extract peak amplitudes and latencies and from unigram word surprisal TRFs and comparison between narrative and scrambled passages.
9. TRFs_main.py - TRF peak extraction code definition
behavioral.zip
Behavioral responses during the data recording.
The variables in tables are as follows
- MEGID = Subject ID
- Age = Age in years
- Gender = Male (M) / Female (F)
- Handedness = Right (R) / Left (L)
- All other columns are the questions associated with each passage in the stimuli set. If it is a question ex. 'what was the fellow in the canoe doing? Responses right/ wrong/ did not answer are marked as R/W/did not answer respectively. If it is a
[word(number)]ex. LELEK(1), it represents the probe word asked and the actual number of counts in the passage. Responses show the answers (numbers) given by the participants.
TRFs.zip
Estimated TRFs for different models using boosting algorithm (https://eelbrain.readthedocs.io/en/stable/generated/eelbrain.boosting.html) for each subject and passage type. The .pkl files include the boosting result for each subject. The file naming format is '{subjects}/*{passage type} nobl - 120-900 100 Hz ftp model {model number} boosting h50 l1 4ptns ss1 cv.pkl'., where Model117 = gammatonelog-8 + gammatonelog-on-8 + pho + wor + psr1 + pen + wfr + hug, Model 111 = gammatonelog-8 + gammatonelog-on-8 + pho + wor + psr1 + pen, and Model 116 = gammatonelog-8 + gammatonelog-on-8 + pho + wor + psr1 + pen + wfr
Code/Software
Require python packages are - eelbrain, mne, and trftools
Required R packages are - ggplot2, dplyr, tidyr, readr, ggpubr
Magnetoencephalography (MEG) data were recorded from young adult participants as they listened to four types of continuous-speech-like passages: speech-envelope modulated noise, English-like non-words, scrambled words, and narrative passage. All information related to experimental procedure, stimuli, preprocessing, and data anlaysis are described in the paper.
