Data from: Auditing shortcut learning and misclassification in AI-based breast cancer genomic subtyping
Data files
Sep 23, 2026 version files 199.47 MB
-
AI-BIAS_cleaned_TCGA_PAM50_model_dataset_deidentified.csv
19.52 KB
-
capstone_model_data_for_shap_export_deidentified.csv
13.37 MB
-
data_dictionary_v2.csv
154.85 KB
-
README.md
34.92 KB
-
TCGA_BRCA_clinicalMatrix_deidentified.dta
3.54 MB
-
TCGA_BRCA_clinicalMatrix_deidentified.tsv
1.91 MB
-
TCGA.BRCA.sampleMap_HiSeqV2.tsv
180.45 MB
Abstract
Objectives: To simulate shortcut learning mechanisms in AI-based breast cancer genomic subtyping and to develop an interpretable, reproducible framework capable of auditing feature over-reliance using a low-code environment.
Materials and Methods: A retrospective secondary analysis of The Cancer Genome Atlas Breast Invasive Carcinoma (TCGA-BRCA) dataset was conducted. A multinomial logistic regression model was trained on 691 complete cases using only clinical predictors (estrogen receptor [ER], progesterone receptor [PR], HER2 status, tumor stage, and age) to predict PAM50 molecular subtypes. A pseudo-SHAP (Shapley additive explanations) attribution method was implemented in Stata to estimate the marginal influence of each variable. Model performance was evaluated using McFadden’s pseudo-R² and validated by comparing pseudo-SHAP scores against canonical SHAP values derived in Python.
Results: The clinical-only model achieved a pseudo-R² of 0.396, indicating moderate explanatory power. PR and HER2 status were statistically significant predictors (P < .01) for Luminal B and HER2-enriched subtypes, respectively. The pseudo-SHAP framework identified disproportionate influence of these features (ΔP up to +0.29 for HER2-enriched). Correlation between pseudo-SHAP and SHAP values was strong (Spearman r = 0.91, P < .001), confirming method validity.
Discussion: Shortcut learning was evident, with the model over-relying on surrogate clinical biomarkers. Such behavior may distort biological interpretation and compromise clinical decision support reliability.
Conclusion: The pseudo-SHAP method provides a scalable, interpretable auditing framework for detecting shortcut learning in resource-constrained environments. Its application could enhance transparency and equity in AI models for precision oncology.
Dataset DOI: https://doi.org/10.5061/dryad.fj6q57482
This Dryad deposit contains the de-identified TCGA-BRCA clinical annotation table (age provided as ranges), the Stata multiple imputation export, the modeling table used for the Python SHAP validation, the cohort gene expression matrix, a per-column data dictionary, the Stata scripts (published at Zenodo under the MIT license), and descriptive figures. The primary article is published in JAMIA Open, 2026;9(2):ooaf177, https://doi.org/10.1093/jamiaopen/ooaf177.
Primary article: Borges J. Auditing shortcut learning and misclassification in artificial intelligence-based breast cancer genomic subtyping. JAMIA Open. 2026;9(2):ooaf177. https://doi.org/10.1093/jamiaopen/ooaf177 (PMID 41937807, PMCID PMC13044511)
Description of the data and file structure
The study is a retrospective secondary analysis of The Cancer Genome Atlas Breast Invasive Carcinoma cohort (TCGA-BRCA) obtained from the UCSC Xena platform. A multinomial logistic regression model was trained on clinical predictors only (estrogen receptor status, progesterone receptor status, HER2 status, tumor stage, and age at diagnosis) to predict PAM50 intrinsic subtypes, and a pseudo-SHAP perturbation method implemented in Stata was used to quantify the marginal influence of each predictor and to detect shortcut learning. Canonical SHAP values were computed in Python (XGBoost and the shap package) on the same predictors for validation.
The deposit contains the de-identified clinical annotation table that the Stata pipeline starts from, the multiple imputation export produced by the pipeline, the six-column modeling table used by the Python notebook, the gene expression matrix of the cohort, a complete per-column data dictionary, the Stata scripts (published at Zenodo under the MIT license) and four descriptive figures (published at Zenodo under CC BY 4.0).
All clinical tables were de-identified for this deposit as described under Human subjects and de-identification. In particular, age at diagnosis is provided only as an age range in every file, so the deposited tables reproduce the study design but not the continuous age variable used in the published model; see Reproducibility notes.
Files and variables
| File | Category | Format | Rows x columns | Content |
|---|---|---|---|---|
| TCGA_BRCA_clinicalMatrix_deidentified.tsv | Data | tab-delimited text | 1,247 x 192 | TCGA-BRCA clinical annotation matrix (UCSC Xena TCGA.BRCA.sampleMap/BRCA_clinicalMatrix), de-identified |
| TCGA_BRCA_clinicalMatrix_deidentified.dta | Data | Stata (Stata 13 format, release 117) | 1,247 x 192 | Same table with Stata-conformant variable names, the entry point of the Stata pipeline |
| capstone_model_data_for_shap_export_deidentified.csv | Data | CSV | 9,347 x 209 | Stata multiple imputation long export (mi set mlong) of the clinical matrix plus derived modeling variables |
| AI-BIAS_cleaned_TCGA_PAM50_model_dataset_deidentified.csv | Data | CSV | 1,231 x 6 | Six-column modeling table used by the Python SHAP notebook (XGBoost) |
| TCGA.BRCA.sampleMap_HiSeqV2.tsv | Data | Tab-delimited text | 20,530 x 1,219 | Gene expression matrix of the cohort (UCSC Xena TCGA.BRCA.sampleMap/HiSeqV2), unchanged |
| data_dictionary_v2.csv | Data | CSV | 600 x 7 | Per column dictionary for every tabular file: file, column_name, original_name, description, n_non_missing, n_distinct, example_values |
| capstone_step_1_to_3_data_cleaning.do | Software | Stata do file | Steps 1 to 3: inspection, multiple imputation (mi set mlong, mi impute regress, 10 imputations per registered variable), variable selection | |
| capstone_step_4_feature_engineering_final.do | Software | Stata do file | Step 4: stage grouping and categorical encoding | |
| capstone_step_5_simplified_model.do | Software | Stata do file | Step 5: binary receptor variables and the multinomial logistic regression | |
| capstone_step6_export_model.do | Software | Stata do file | Step 6.1: export of model coefficients (esttab) | |
| capstone_step6_surrogate_tree.do | Software | Stata do file | Step 6.2: surrogate CART tree (rpart) | |
| capstone_step6_subgroup_analysis.do | Software | Stata do file | Step 6.3: subgroup and stratified cross-tabulations | |
| PAM50_Capstone_Consolidated_Pipeline.do | Software | Stata do file | Consolidated version of steps 1 to 6 in one script (same code as the GitHub repository) | |
| fig1_age_distribution_histogram.png | Supplemental | PNG | Histogram of age at diagnosis (descriptive, aggregate) | |
| fig2_pam50_subtype_frequencies.png | Supplemental | PNG | Bar plot of PAM50 subtype frequencies | |
| fig3_age_by_pam50_subtype.png | Supplemental | PNG | Age distribution by PAM50 subtype | |
| fig4_outlier_detection_age_continuous.png | Supplemental | PNG | Box plot used for outlier screening of age |
TCGA_BRCA_clinicalMatrix_deidentified.tsv
One row per TCGA-BRCA sample (1,247 samples, 1,247 distinct sample barcodes) and 192 columns. The columns are the TCGA clinical data elements distributed by UCSC Xena for the cohort, in the original order, with two columns removed (days_to_birth, form_completion_date) and four columns transformed (two age columns binned, two gender columns recoded). Column families:
| Column family | Examples | Meaning |
|---|---|---|
| Sample identifiers | sampleID, _patient, bcr_patient_barcode, bcr_sample_barcode |
TCGA barcodes (public study identifiers, not personal identifiers) |
| Suffix _nature2012 | PAM50_mRNA_nature2012, ER_Status_nature2012, PR_Status_nature2012, HER2_Final_Status_nature2012, AJCC_Stage_nature2012, OS_Time_nature2012, OS_event_nature2012 | Annotations from the TCGA Network 2012 breast cancer analysis |
Prefix _PANCAN_ |
_PANCAN_Cluster_Cluster_PANCAN, _PANCAN_DNAMethyl_BRCA |
TCGA Pan-Cancer cluster assignments |
Prefix _GENOMIC_ID_ |
_GENOMIC_ID_TCGA_BRCA_exp_HiSeqV2 |
Sample barcode as present in each Xena genomic dataset (a lookup key, not a measurement) |
| TCGA BCR clinical form | pathologic_stage, pathologic_T, pathologic_N, pathologic_M, histological_type, breast_carcinoma_estrogen_receptor_status, breast_carcinoma_progesterone_receptor_status, lab_proc_her2_neu_immunohistochemistry_receptor_status, menopause_status, radiation_therapy, vital_status, days_to_death, days_to_last_followup, year_of_initial_pathologic_diagnosis | Clinical data elements from the TCGA Biospecimen Core Resource clinical and follow-up forms |
| Outcome | PAM50Call_RNAseq | PAM50 subtype call from RNA seq (Basal, Her2, LumA, LumB, Normal); 956 samples have a call |
Variables used by the published model: PAM50Call_RNAseq (outcome), ER_Status_nature2012, PR_Status_nature2012, HER2_Final_Status_nature2012, AJCC_Stage_nature2012 (grouped I, II, III) and Age_at_Initial_Pathologic_Diagnosis_nature2012 (continuous in the analysis, age range in this deposit).
De-identified columns in this file: age_at_initial_pathologic_diagnosis and Age_at_Initial_Pathologic_Diagnosis_nature2012 hold age ranges (<30, 30-39, 40-49, 50-59, 60-69, 70-79, 80+); gender and Gender_nature2012 hold numeric codes. Every column is described individually in data_dictionary_v2.csv.
TCGA_BRCA_clinicalMatrix_deidentified.dta
The same 1,247 x 192 table saved from Stata 18 in Stata 13 file format (release 117) with saveold. Stata limits variable names to 32 characters, so long TCGA names are truncated and 16 names that collided after truncation were replaced by Stata with positional names (v48, v58, v59, v115, v116, v118, v119, v121, v122, v126, v127, v141, v174, v187, v190, v192); for each of these the original TCGA column name is stored in the Stata variable label and listed in the original_name column of data_dictionary_v2.csv (for example v48 = age_at_initial_pathologic_diagnosis). Numeric variables use Stata system missing (.) for missing values; string variables use the text NA. The age variables are strings holding age ranges and the gender variables are numeric codes.
Opening the file: Stata 13 or later (use), R haven::read_dta() ReadStat-based; the older foreign package cannot read any file format newer than Stata 12 and is not expected to work), Python pandas.read_stata() or pyreadstat.read_dta(). The file was verified to open with pyreadstat 1.3.6 (ReadStat), the same C library used by R haven and by JASP.
capstone_model_data_for_shap_export_deidentified.csv
The dataset exported from Stata after steps 1 to 5 of the pipeline, in Stata multiple imputation long format (mi set mlong). It has 9,347 rows and 209 columns:
| Block | Columns | Description |
|---|---|---|
| Rows with _mi_m = 0 | 1,247 rows | The original samples; one row per sample, sampleid equals sampleID of the clinical matrix |
| Rows with _mi_m = 1 to 20 | 20 x 405 rows | Imputed copies of the 405 samples that had a missing value in a registered variable (age at diagnosis or days to last contact); two mi impute regress ... add(10) commands produced 20 imputations |
| Columns 1 to 192 | clinical matrix | The 192 clinical matrix columns in the same order as TCGA_BRCA_clinicalMatrix_deidentified.tsv, under the Stata variable names used in the scripts (for example age_at_initia_o = Age_at_Initial_Pathologic_Diagnosis_nature2012, days_to_date_t = Days_to_Date_of_Last_Contact_nature2012, v48 = age_at_initial_pathologic_diagnosis) |
| Columns 193 to 209 | derived | pam50_numeric (PAM50 label text: Basal, Her2, LumA, LumB, Normal), freq1 to freq5 (indicator columns generated by tabulate ..., generate(); freq1 = Basal, freq2 = Her2, freq3 = LumA, freq4 = LumB, freq5 = Normal), _mi_m, _mi_id, _mi_miss (Stata mi system variables), vital_status_numeric, tumor_stage_grouped (I, II, III), er_status_num, pr_status_num, her2_status_num (Stata encode codes), er_binary, pr_binary, her2_binary (1 = Positive, 0 = Negative) |
The column to original name mapping for all 209 columns is given in data_dictionary_v2.csv (column original_name). De-identified columns: age_at_initia_o and v48 hold age ranges (imputed non-integer ages were floored before binning); gender and gender_nature2012 hold numeric codes; days_to_birth and form_completion_date were removed.
AI-BIAS_cleaned_TCGA_PAM50_model_dataset_deidentified.csv
The six-column table was loaded by the Colab notebook PAM50_SHAP_Colab_Notebook.ipynb (GitHub repository) to train the XGBoost classifier and compute canonical SHAP values (SHAP value array shape 1,231 x 5 features x 5 classes).
| Column | Values | Description |
|---|---|---|
| pam50_numeric | 0 to 4 | Target class, encoded as in the notebook label_map: 0 = Basal, 1 = Her2-enriched, 2 = Luminal A, 3 = Luminal B, 4 = Normal-like |
| er_binary | 0, 1 | ER status, 1 = Positive |
| pr_binary | 0, 1 | PR status, 1 = Positive |
| her2_binary | 0, 1 | HER2 status, 1 = Positive |
| age_at_diagnosis | age range | De-identified: <30, 30-39, 40-49, 50-59, 60-69, 70-79, 80+ (continuous in the original notebook run) |
| stage_simplified | 1 to 4 | Tumor stage I to IV |
The table has 1,231 rows and no missing values. It was produced from the same TCGA-BRCA cohort during the capstone work; 767 rows are exact duplicates of another row in the original export and were retained unchanged (apart from age binning) so that the deposited table matches the notebook input.
TCGA.BRCA.sampleMap_HiSeqV2.tsv
The IlluminaHiSeq gene expression matrix of the cohort as distributed by UCSC Xena (dataset TCGA.BRCA.sampleMap/HiSeqV2): 20,530 rows (genes; first column sample holds the gene symbol) and 1,218 sample columns headed by TCGA sample barcodes; values are log2(RSEM normalized count + 1). The file is unchanged from the Xena distribution (SHA-256 9430ba08c8f3ad70f6e61cc75dc36f1f34cce1c50b6d707bf63825512f32d78a) and contains no clinical or demographic fields. It is included as the genomic reference for the cohort; the published model uses clinical predictors only, so it is not an input to the scripts.
data_dictionary_v2.csv
600 rows, one per column of every tabular file (including the .dta and the expression matrix), with: file, column_name, original_name (the TCGA or Xena name when a Stata name differs), description, n_non_missing, n_distinct, example_values (up to five observed values).
Missing data
Every empty cell in the tab delimited and CSV files was replaced by the text NA (1,247 x 192 clinical matrix: 116,419 NA cells; multiple imputation export: 1,138,576 NA cells, most of them in the imputed rows, which carry values only for the registered variables and identifiers; six column modeling table: none). In the Stata file, numeric variables use Stata system missing (.) and string variables use NA. Missingness reflects the TCGA source annotations: many _nature2012 columns are populated only for the 952 samples included in the 2012 TCGA Network analysis, and PAM50Call_RNAseq is available for 956 samples. The 691 complete cases analysed in the article are the samples with a PAM50 call andnon-missingg ER, PR, HER2, stag,e and age.
Human subjects and de-identification
The data derive from TCGA-BRCA, a public, consented, de-identified research resource accessed through UCSC Xena; no direct identifiers exist in the source. Following Dryad's Human Subjects and Identifiable Information policy, the following additional de-identification was applied uniformly to every clinical table in this deposit (rules R1 to R8):
| Rule | Field(s) | Treatment |
|---|---|---|
| R1 | age_at_initial_pathologic_diagnosis, Age_at_Initial_Pathologic_Diagnosis_nature2012, age_at_initia_o, v48, age_at_diagnosis | Exact age replaced by an age range: <30, 30-39, 40-49, 50-59, 60-69, 70-79, 80+ (ages of 90 and above are therefore aggregated) |
| R2 | days_to_birth | Removed (it encodes exact age in days) |
| R3 | form_completion_date | Removed (calendar date) |
| R4 | year_of_initial_pathologic_diagnosis | Retained (year only) |
| R5 | days_to_death, days_to_last_followup, days_to_collection, Days_to_Date_of_Last_Contact_nature2012 and the other days_to_ fields | Retained (intervals in days from diagnosis, not calendar dates) |
| R6 | gender, Gender_nature2012, gender_nature2012 | Recoded to numeric codes; per Dryad curation instructions, the code definitions are not published in the data or in this README |
| R7 | all columns | Empty cells replaced by NA |
| R8 | all other columns | Copied verbatim |
TCGA sample and patient barcodes are public study identifiers and were retained. The expression matrix contains no demographic fields. Descriptive figures are aggregate plots. No file in this deposit contains an exact age, a date of birth, a calendar date of any event, or any direct identifier.
Code and software
| Component | Software | Notes |
|---|---|---|
| Data cleaning, multiple imputation, multinomial logistic regression, pseudo-SHAP perturbation, surrogate tree, subgroup tables | Stata 18 (any edition; the scripts also run on Stata 16 and 17). capstone_step6_surrogate_tree.do calls a community-contributed rpart command that is not part of official Stata, and capstone_step6_export_model.do requires estout (ssc install estout) |
Scripts are deposited as Software and are published at Zenodo under the MIT license, identical to the GitHub repository. |
| Canonical SHAP validation | Python 3.8 or later with pandas, xgboost, shap, scikit-learn, and matplotlib; notebook PAM50_SHAP_Colab_Notebook.ipynb in the GitHub repository | Input file: AI-BIAS_cleaned_TCGA_PAM50_model_dataset (deposited here with binned age) |
| Reading the deposited files | TSV and CSV: any software. Stata file: Stata 13 or later, R haven (read_dta), Python pandas (read_stata), or pyreadstat | Verified with pyreadstat 1.3.6 (ReadStat) |
Reproducibility notes
- The scripts were written against the continuous age variable. Because the deposited tables carry age ranges (Dryad policy for human subjects data), rerunning the scripts on the deposited files will not reproduce the continuous age term of the published model. To reproduce the published estimates exactly, download the public clinical matrix TCGA.BRCA.sampleMap/BRCA_clinicalMatrix from UCSC Xena (link under Access information), import it into Stata (
import delimited) and run the scripts; the deposited de-identified table is otherwise identical to that source apart from rules R1 to R8. capstone_step_1_to_3_data_cleaning.dobegins withuse "your_datafile.dta"; replace the placeholder with the path of TCGA_BRCA_clinicalMatrix_deidentified.dta or of the Stata import of the Xena matrix. The scripts refer to the age and last contact variables under the names age_at_initia_o and days_to_date_t (renamed from age_at_initial_pathologic_diagno and days_to_date_of_last_contact_nat, respectively).- In the deposited Stata f,ile the gender variables are already numeric, so the
encode gender_nature2012, gen(gender_num)line of step 4 is not applicable (the variable is dropped again in step 5 and does not enter the model). - Multiple imputation uses random draws; set a seed (
set seed) beforemi imputefor bitwise reproducibility of the imputed rows.
Access information
Data were derived from the following public sources:
| Source | Location |
|---|---|
| UCSC Xena, TCGA Breast Cancer (BRCA) cohort, clinical matrix (TCGA.BRCA.sampleMap/BRCA_clinicalMatrix) and gene expression Illumina HiSeq (TCGA.BRCA.sampleMap/HiSeqV2) | https://xenabrowser.net/datapages/?cohort=TCGA%20Breast%20Cancer%20(BRCA) |
| NCI Genomic Data Commons, project TCGA-BRCA | https://portal.gdc.cancer.gov/projects/TCGA-BRCA |
| TCGA Network. Comprehensive molecular portraits of human breast tumours. Nature 2012;490:61-70 (source of the _nature2012 annotations) | https://doi.org/10.1038/nature11412 |
Other locations of related material:
| Item | Location | License |
|---|---|---|
| Analytic code and notebook (GitHub) | https://github.com/julian-borges-md/breast-cancer-ai-misclassification- | MIT |
| Version 1 code and data release cited in the article | https://doi.org/10.5281/zenodo.15237131 | As stated on the record |
| Supplemental creative works (documentation, figures, slides) | https://doi.org/10.5281/zenodo.18175665 | CC BY 4.0 |
Licensing
| Component | License |
|---|---|
| Data files hosted by Dryad (all TSV and CSV files, the Stata data file and the data dictionary) | CC0 1.0 Universal public domain dedication |
| Software files (Stata scripts, published at Zenodo through Dryad) | MIT License, the same license as the GitHub repository |
| Supplemental files (figures, published at Zenodo through Dryad) | CC BY 4.0 |
The author confirms that publication of the data files under CC0 is intended and accepted, and that the MIT license of the code in the GitHub repository is not affected by it.
Contact
Julian Yin Vieira Borges, MD, MS. Department of Computer Science, Boston University Metropolitan College, Boston, MA, United States. jyborges@bu.edu. ORCID https://orcid.org/0009-0001-9929-3135
Human subjects data
Human subjects de-identification statement. All data are derived from The Cancer Genome Atlas Breast Invasive Carcinoma cohort (TCGA-BRCA), a public research resource accessed through the UCSC Xena platform. TCGA participants provided informed consent for the collection and open sharing of de-identified clinical and molecular data under IRB-approved protocols administered by the NCI/NHGRI TCGA program, and the data are distributed by the NCI Genomic Data Commons and UCSC Xena without direct identifiers. No new data were collected from human participants, and no identifiable private information was accessed; the study is exempt under 45 CFR 46.101(b)(4). For this Dryad deposit the following additional de-identification was applied uniformly to every clinical table: exact age at diagnosis was replaced by age ranges (<30, 30-39, 40-49, 50-59, 60-69, 70-79, 80+, so that ages of 90 and above are aggregated); the field days_to_birth (exact age in days) and the field form_completion_date (a calendar date) were removed; gender was recoded to numeric codes whose definitions are not published; empty cells were set to NA. Retained fields are TCGA sample and patient barcodes (public study identifiers), year of diagnosis (year only), and interval fields expressed in days from diagnosis, which are not calendar dates. The gene expression matrix contains no demographic fields, and the deposited figures are aggregate plots. No file in this deposit contains an exact age, a date of birth, a calendar date of any event, a name, an address, or any other direct or indirect identifier. The full rule set (R1 to R8) and the per-column data dictionary are provided in the README.
