Data and code from: Comparing the outputs of intramural and extramural grants funded by National Institutes of Health
Data files
Jun 19, 2026 version files 3.31 GB
-
1.data_processing_high_linked.ipynb
88.20 KB
-
2.RCR_extract.ipynb
18.54 KB
-
3.1_separate_most_cost_expensive.ipynb
19.67 KB
-
3.PMID_project_crosstable.ipynb
72.47 KB
-
4.topic_clustering.ipynb
103.65 KB
-
5.PI_attributes.ipynb
108.83 KB
-
6._regression_analysis.ipynb
420.01 KB
-
all_publist_rcr.csv
154.84 MB
-
cluster_desc.csv
3.14 KB
-
cluster_master_labels.csv
13.56 MB
-
descriptive.ipynb
20.12 MB
-
GDP_Price_Index_Annual_and_Cumulative_Values_for_1950_through_2028.xlsx
323.54 KB
-
Master.csv
2.89 GB
-
PI_allyear_prod_rcr.csv
20.72 MB
-
plot.ipynb
37.99 MB
-
pmid_project_map.csv
164.94 MB
-
README.md
6.91 KB
-
word_freq.zip
2.06 MB
Abstract
Funding agencies use a variety of mechanisms to fund research. The National Institutes of Health in the United States, for example, employs scientists to perform research at its own laboratories (intramural research), and it also awards grants to pay for research at external institutions such as universities (extramural research). Here, using data from 1594 intramural grants and 97054 extramural grants funded between 2009 and 2019, we compare the scholarly outputs from these two funding mechanisms in terms of number of publications, relative citation ratio and clinical metrics. We find that extramural awards more cost-effectively fund outputs commonly used for academic review such as number of publications and citations per dollar, while intramural awards are more cost-effective at generating work that influences future clinical work, more closely in line with agency health goals. These findings provide evidence that institutional incentives associated with different funding mechanisms drive their comparative strengths.
Dataset DOI: 10.5061/dryad.gb5mkkx5b
Description of the data and file structure
These data were collected for a comparative cost-effectiveness analysis of NIH Extramural vs. Intramural funding mechanisms with respect to research outputs.
Files and variables
File: 1.data_processing_high_linked.ipynb
Description: Data processing code
File: 2.RCR_extract.ipynb
Description: Extraction of Relative Citation Ratio and related metrics for comparative analysis of funding outputs
File: 3.1_separate_most_cost_expensive.ipynb
Description: Data quality control to identify and separate non-research grants that nevertheless appear in the datasets
File: 3.PMID_project_crosstable.ipynb
Description: Crosstabulation of NIH project numbers with research outputs in the form of PubMed IDs
File: 4.topic_clustering.ipynb
Description: Topic clustering code for research projects
File: cluster_desc.csv
Description: Code to generate attributes of Principal Investigators for use in regression analysis
Variables
- cluster_id: Word2Vec cluster ID
- dispersion_order: Cluster dispersion order
- counts: Counts
- avg_centroid_distance: Average distance to centroid
- intra_document_dispersion: Intra-cluster document dispersion
- word2vec_description: K-nearest neighbor terms in Word2Vec space
- gpt_label: ChatGPT generated human-readable label derived from the k-nearest neighbor terms
File: 5.PI_attributes.ipynb
Description: Code to generate attributes of Principal Investigators for use in regression analysis
File: 6._regression_analysis.ipynb
Description: Code to conduct regression analysis of funding mechanism cost effectiveness
File: GDP_Price_Index_Annual_and_Cumulative_Values_for_1950_through_2028.xlsx
Description: Price index used for inflation adjustment of funding amounts
File: cluster_master_labels.csv
Description: Mapping of article PubMed IDs to Word2Vec topic clusters
Variables
- _ref: Internal reference ID number
- cluster_id: Cluster ID
- dispersion_order: Cluster dispersion order
- PMID: PubMed ID
File: descriptive.ipynb
Description:
File: PI_allyear_prod_rcr.csv
Description: Code for descriptive analysis of project inputs and outputs
Variables
- : Python index number
- AND_ID: PubMed Knowledge Graph Author ID
- pub_num: Publication count
- pub_num_fl: Publication count (first/last author only)
- avg_rcr: Average Relative Citation Ratio
- avg_rcr_fl: Average Relative Citation Ratio (first/last author only)
- PROJECT_NUMBER: Number of funded projects
- FY_start: Fiscal Year of first grant
- first_pub_yr: Calendar year of first publication
- pub_yr_len: Number of years of publications
- collab_num: Number of collaborators
- PI_num: Number of Principal Investigators on grants
- past_proj_num: Number of past projects
- proj_yr_len: Number of years of funded projects
- clin_num: Number of clinical articles
- mean_human: Mean Human score of articles
- mean_animal: Mean Animal score of articles
- mean_molecular: Mean Molecular/Cellular score of articles
File: plot.ipynb
Description: Data visualizations for comparative analysis
File: all_publist_rcr.csv
Description: Article-level metrics for associated PubMed IDs from the NIH iCite database: Relative Citation Ratio, Approximate Potential to Translate (APT), Is Clinical flag, Cited by clinical article binary indicator, Human, Animal, and Molecular/Cellular Biology, and Clinical citation count. Detailed definitions available at https://icite.od.nih.gov
Variables
- : Python index number
- PMID: PubMed ID
- relative_citation_ratio: Relative Citation Ratio
- apt: Approximate Potential to Translate
- is_clinical: Is clinical article flag
- cited_by_clin: Cited by clinical article flag
- human: Human score
- animal: Animal score
- molecular_cellular: Molecular/Cellular Biology score
- clin_cite_num: Number of clinical citations
File: pmid_project_map.csv
Description: Mapping of PI projects to publications
Variables
- PMID: PubMed ID
- PROJECT_NUMBER: Core project number
- year: Publication year
File: Master.csv
Description: Compiled NIH grant metadata into a single dataset. Data fields from NIH ExPORTER https://reporter.nih.gov/exporter. Some fields were not used, and are marked "Not used"
Variables
- [no header] : Python index number
- APPLICATION_ID: Application ID
- ACTIVITY: Activity code
- ADMINISTERING_IC: Administering IC
- APPLICATION_TYPE: Application type code
- ARRA_FUNDED: ARRA funding flag
- AWARD_NOTICE_DATE: Notice of award date
- BUDGET_START: Budget start date
- BUDGET_END: Budget end date
- CFDA_CODE: Not used
- CORE_PROJECT_NUM: Core project number
- ED_INST_TYPE: Not used
- FOA_NUMBER: Funding opportunity announcement number
- FULL_PROJECT_NUM: Full project number
- SUBPROJECT_ID: Subproject ID number
- FUNDING_ICs: Funding ICs
- FY: Fiscal Year
- IC_NAME: Full IC name
- NIH_SPENDING_CATS: Not used
- ORG_CITY: Not used
- ORG_COUNTRY: Not used
- ORG_DEPT: Not used
- ORG_DISTRICT: Not used
- ORG_DUNS: Organization DUNS ID number
- ORG_FIPS: Not used
- ORG_NAME: Not used
- ORG_STATE: Not used
- ORG_ZIPCODE: Not used
- PHR: Not used
- PI_IDS: Principal Investigator ID numbers
- PI_NAMEs: Principal Investigator names
- PROGRAM_OFFICER_NAME: Program Officer name
- PROJECT_START: Project start date
- PROJECT_END: Project end date
- PROJECT_TERMS: Not used
- PROJECT_TITLE: Project title
- SERIAL_NUMBER: Project serial number
- STUDY_SECTION: Study section abbreviation
- STUDY_SECTION_NAME: Study section full name
- SUFFIX: Project suffix number
- SUPPORT_YEAR: Support year
- TOTAL_COST: Total costs (direct plus indirect costs)
- TOTAL_COST_SUB_PROJECT: Subproject total costs
- FUNDING_MECHANISM: Funding mechanism
- FUNDING_Ics: Funding ICs
- DIRECT_COST_AMT: Direct costs
- INDIRECT_COST_AMT: Indirect costs
- ORG_IPF_CODE: Not used
File: word_freq.zip
Description: Data used in word frequency analysis
Code/software
Python 3.9 or later
Packages: pandas, re, plotly, os, warnings, pysmatch, stata_setup, pystata, Jupyter notebook
Software workflow is enumerated in the file description and file name numbering.
Access information
Data used for this analysis are publicly available at:
Article data:
https://doi.org/10.35092/yhjc.c.4586573
Grant data:
https://reporter.nih.gov/exporter
Author data: