Stanford Study of the Accuracy of the Pre-Interventional Preventive Risk Assessment (PIPRA) Tool for the Prediction of ICU-Delirium in a Mixed Cardiothoracic Intensive Care Unit Population

Abstract:

Background: Delirium is a prevalent neuropsychiatric syndrome, particularly in intensive care units (ICU), affecting up to 87% of patients. While various predictive tools exist, many require manual data input, leading to significant time consumption. The PIPRA tool, designed to operate within existing electronic medical records (EMR), automates this process by providing a risk score for post-operative delirium (POD). This study evaluates the accuracy of the PIPRA tool in predicting ICU-delirium in a high-risk population.






Methods: This study retroactively applied the PIPRA tool to a cohort from a parallel study investigating EEG-based diagnosis for delirium in the adult cardiothoracic-ICU at Stanford University Medical Center. The tool’s predictive score was compared against a DSM-5 based neuropsychiatric assessment conducted by the CLP team, including cases of subsyndromal delirium, using a threshold of 35%, indicating “high-risk for delirium” (Dodsworth, 2023).






Results: Results from 139 patients revealed 55 (39.5%) of these developed ICU-delirium. The PIPRA tool resulted in an area under the receiver operating curve of 0.459 (95% CI: 0.347-0.572), demonstrating a sensitivity of 63.6% and specificity of 19% at the 35% cutoff threshold.






Discussion: Each year, numerous individuals undergo ICU admission only to experience ICU-Delirium and endure its associated negative outcomes. The developers of the PIPRA tool estimated that the following 9 items would be the most influential risk factors for the development of POD: sex, age, body mass index, history of delirium, history of cognitive impairment, presenting comorbidities, number of medications, serum C-reactive protein level, ASA physical status, type and duration of surgery/anesthesia.





In our sample, the PIPRA tool failed to accurately predict delirium. Several factors may shed light on this finding. As detailed in their original study, the PIPRA-artificial intelligence algorithm was trained on a population of elderly (age > 60) patients undergoing elective non-cardiac and non-intracranial surgery. The authors informed us that their studies failed to demonstrate predictive accuracy in this population. Our sample included adult patients 18 years and older (only 60% of which were >60 y/o) and a mixed population of both elective and non-elective cardiothoracic surgical cases (about 85% of the total sample), as well as nonoperative cardiopulmonary (medical) patients (e.g., pulmonary embolism).






Conclusion: The data suggests that the PIPRA tool might provide a novel approach for determining the risk of delirium in most types of surgeries, when applied to the correct population (i.e., excluding cardiac and intracranial). Several reasons may account for the PIPRA tool failing to provide an accurate assessment in our study population. Our sample dramatically deviated from that included in the original study and what the model was trained to detect. Our results do not take away from the potential for this tool to accurately predict the development of POD when applied to the correct sample, based on the indications provided by the developer. For reference, please see the results of Gunther et al, also discussed during this conference which demonstrated promising results in elderly, orthopedic surgical cases.






References:


Dodsworth, B. T. et al. (2023). PMID: 37290122


Sadeghirad, B et al. (2023). PMID: 37819663


Presentation Eligibility: "Not previously published or presented"

Diversity, Equity, and Inclusion: Our study on delirium risk assessment highlights disparities in predictive accuracy, potentially affecting marginalized groups who may experience worsened outcomes. The PIPRA tool's failure to predict delirium in our study population demonstrated the need for inclusive research, encompassing diverse patient populations. Our commitment to diversity, equity, and inclusion involves advocating for inclusive research practices and acknowledging the diverse needs of all patients, regardless of background. Our diverse authorship from Stanford University reflects our dedication to fostering inclusivity in academic discourse.