Objective: To develop a diagnostic test that stratifies epileptic seizure (ES) from psychogenic non-epileptic seizure (PNES) by developing a multimodal algorithm that integrates plasma concentrations of selected immune response associated proteins and patient clinical risk factors for seizure.

Methods: Daily blood samples were collected from patients evaluated in the epilepsy monitoring unit (EMU) within 24 hours after EEG confirmed ES or PNES and plasma was isolated. Levels of 51 candidate plasma proteins were quantified using an automated, multiplexed, sandwich ELISA and then integrated and analyzed using our diagnostic algorithm.

Results: A 51 protein multiplexed ELISA panel was used to determine the plasma concentrations of ES patients, PNES patients, and healthy controls. A combination of protein concentrations, TRAIL, ICAM-1, MCP-2 and TNF-R1 were identified that provided a probability that a patient recently experienced a seizure. The diagnostic algorithm yielded an AUC of 0.94 ± 0.07, sensitivity of 82.6% (95% CI: 62.9-93.0) and specificity of 91.6% (95% CI: 74.2-97.7). Further, expanding the diagnostic algorithm to include previously identified PNES risk factors enhanced diagnostic performance with AUC of 0.97 ± 0.05, sensitivity of 91.3%; (95% CI: 73.2-97.6), and specificity of 95.8%; (95% CI: 79.8-99.3).

Conclusions: We have identified four plasma proteins that could provide a rapid, cost-effective, and accurate blood-based diagnostic test to confirm recent ES or PNES.

Classification of Evidence: This study provides Class III evidence that variable levels of four plasma proteins, when analyzed by a diagnostic algorithm, can distinguish PNES from ES with sensitivity of 82.6% and specificity of 91.6%.

With the overall objective of generating a diagnostic algorithm that efficiently stratifies ES from PNES, a two-step procedure was used. First, the effect size and a random forest approach was used to select a subset of proteins that most efficiently stratified cohorts. Second, all possible combinations of the selected subset of proteins were used to refine algorithms and the most information rich, top performing algorithm was selected. Generating all combinations of proteins to include into an algorithm is computationally inefficient. To reduce the search space and prevent inclusion of proteins with limited information the effect size between cohort means and the relative importance of each protein in a bootstrap forest was considered. The effect size and 95% confidence interval were calculated¹⁸. The results from the effect size analysis was then compared to the results of a Random Forest¹⁹ importance analysis, implemented using the scikit learn python package. The random forest was generated using 1000 trials of random sampling with replacement to generate a family of classification trees. Gini criterion was used to refine the classification trees and the relative importance of each feature was assessed using the mean decrease in impurity (e.g. the number of times a feature is used to split the tree relative to the number of samples split). To challenge the approach and determine the noise floor two fake proteins data sets were included. These data set were created using a random number generator that selected values from a normal distribution with a mean of 0 and a standard deviation of 1. 

Proteins with mean concentration difference large enough that the confidence interval of the effect size did not cross zero and had a relative importance greater than the noise floor of the random forest analysis were carried forward to algorithm testing. All unique combinations of proteins were generated and used to refine a diagnostic algorithm using a logistic regression. Video-EEG event diagnosis was used as the gold standard, dependent variable that the selected set of protein concentrations was refined against to generate a diagnostic algorithm. Each algorithm was assessed using the Akaike information criterion (AIC)²⁰. Specifically algorithms were compared by the number of proteins included in the combination and a change of greater than 2 in the AIC measure²¹. To further characterize the top performing algorithms two cross-validation tests were used, first, a leave-one-out strategy was used to assess the algorithms accuracy in scoring the excluded samples and second a 10-fold cross validation was used to assess the algorithms AUC. The binomial proportional confidence intervals for the final algorithm were generated using the Wald test for area under the curve and the Wilson score interval for the sensitivity and specificity.

In end, a logistic regression was selected to use as the final classifier because it is the industry standard and it generates a probability of seizure in addition to a discreet yes/no seizure answer when compared to a selected threshold. The logistic regression was compared to multiple classification approaches including K Nearest neighbor, Naïve Bayes, Decision Tree and a random forest. The scikit learn implementation of each algorithm was used in the comparison and default parameters were used with exception of using reduced entropy rather than Gini criterion to measure the quality of a split in the decision tree and random forest classifiers. In both cases using the entropy measure resulted in improved mean accuracy and AUC. Algorithm performance was compared using two cross validation tests, a leave one out approach was used to generate the mean accuracy and a ten-fold cross validation was used to compare AUC.

To assess the influence of potential confounding parameters (age, gender, comorbidities, drug prescription and time elapsed from seizure), the correlation with protein concentrations were calculated using the Pearson correlation coefficient test. The above parameters were also incorporated into an algorithm with the protein concentrations and the AIC was evaluated to determine if additional information supported their inclusion into the final algorithm.

To gain insight into changes caused by AED prescription all AEDs were sorted according to the chemical class, (Table 2).  The prevalence of each drug was calculated as a sum of the number of times a drug from the class was prescribed. Each drug class was used to refine an algorithm with the protein concentrations and analyzed as above using the coefficient p value and AIC. The carboxamide class was further evaluated to identify potential confounding effects on protein concentrations by stratifying the ES cohort patients according to prescription status and calculating Cohen’s d to quantify differences in mean concentrations. The same analysis was also conducted for nonsteroidal anti-inflammatory drugs (NSAID), which included the following drugs: Diclofenac, Celecoxib, Ketorolac, Aspirin, Naproxen, Ibuprofen and Meloxicam. For both AED and NSAIDs the drug list was generated upon admission to the EMU, dose and titrations were not considered.

AED family analysis results.