Objective: Trauma quality improvement programs and registries improve care and outcomes for injured patients. Designated trauma centers calculate injury scores using dedicated trauma registrars; however, many injuries arrive at non-trauma centers, leaving a substantial amount of data uncaptured. We propose automated methods to identify severe chest injury using machine learning (ML) and natural language processing (NLP) methods from the electronic health record (EHR) for quality reporting.

Materials and Methods: A level I trauma center was queried for patients presenting after injury between 2014 and 2018. Prediction modeling was performed to classify severe chest injury using a reference dataset labeled by certified registrars. Clinical documents from trauma encounters were processed into concept unique identifiers for inputs to ML models: logistic regression with elastic net regularization (EN), extreme gradient boosted machines (XGB), and convolutional neural networks (CNN). The optimal model was identified by examining predictive and face validity metrics using global explanations.

Results: Of 8,952 encounters, 542 (6.1%) had a severe chest injury. CNN and EN had the highest discrimination, with an area under the receiver operating characteristic curve of 0.93 and calibration slopes between 0.88 and 0.97. CNN had better performance across risk thresholds with fewer discordant cases. Examination of global explanations demonstrated the CNN model had better face validity, with top features including “contusion of lung” and “hemopneumothorax.” 

Discussion: The CNN model featured optimal discrimination, calibration, and clinically relevant features selected. 

Conclusion: NLP and ML methods to populate trauma registries for quality analyses are feasible.

This is a script for R used for the analysis for our manuscript. The source data cannot be shared as it is electronic health record data that is protected under the Health Insurance Portability and Accountability Act.

This script could be used with cTAKES analysis on institutional data to generate similar results.