The Deterioration Risk Index: Developing and Piloting a Machine Learning Algorithm to Reduce Pediatric Inpatient Deterioration.
Journal
Pediatric critical care medicine : a journal of the Society of Critical Care Medicine and the World Federation of Pediatric Intensive and Critical Care Societies
ISSN: 1529-7535
Titre abrégé: Pediatr Crit Care Med
Pays: United States
ID NLM: 100954653
Informations de publication
Date de publication:
01 04 2023
01 04 2023
Historique:
medline:
11
4
2023
pubmed:
4
2
2023
entrez:
3
2
2023
Statut:
ppublish
Résumé
Develop and deploy a disease cohort-based machine learning algorithm for timely identification of hospitalized pediatric patients at risk for clinical deterioration that outperforms our existing situational awareness program. Retrospective cohort study. Nationwide Children's Hospital, a freestanding, quaternary-care, academic children's hospital in Columbus, OH. All patients admitted to inpatient units participating in the preexisting situational awareness program from October 20, 2015, to December 31, 2019, excluding patients over 18 years old at admission and those with a neonatal ICU stay during their hospitalization. We developed separate algorithms for cardiac, malignancy, and general cohorts via lasso-regularized logistic regression. Candidate model predictors included vital signs, supplemental oxygen, nursing assessments, early warning scores, diagnoses, lab results, and situational awareness criteria. Model performance was characterized in clinical terms and compared with our previous situational awareness program based on a novel retrospective validation approach. Simulations with frontline staff, prior to clinical implementation, informed user experience and refined interdisciplinary workflows. Model implementation was piloted on cardiology and hospital medicine units in early 2021. The Deterioration Risk Index (DRI) was 2.4 times as sensitive as our existing situational awareness program (sensitivities of 53% and 22%, respectively; p < 0.001) and required 2.3 times fewer alarms per detected event (121 DRI alarms per detected event vs 276 for existing program). Notable improvements were a four-fold sensitivity gain for the cardiac diagnostic cohort (73% vs 18%; p < 0.001) and a three-fold gain (81% vs 27%; p < 0.001) for the malignancy diagnostic cohort. Postimplementation pilot results over 18 months revealed a 77% reduction in deterioration events (three events observed vs 13.1 expected, p = 0.001). The etiology of pediatric inpatient deterioration requires acknowledgement of the unique pathophysiology among cardiology and oncology patients. Selection and weighting of diverse candidate risk factors via machine learning can produce a more sensitive early warning system for clinical deterioration. Leveraging preexisting situational awareness platforms and accounting for operational impacts of model implementation are key aspects to successful bedside translation.
Identifiants
pubmed: 36735282
doi: 10.1097/PCC.0000000000003186
pii: 00130478-202304000-00007
doi:
Types de publication
Journal Article
Langues
eng
Sous-ensembles de citation
IM
Pagination
322-333Commentaires et corrections
Type : CommentIn
Informations de copyright
Copyright © 2023 by the Society of Critical Care Medicine and the World Federation of Pediatric Intensive and Critical Care Societies.
Déclaration de conflit d'intérêts
Dr. Gorham received funding from LDA Research. The remaining authors have disclosed that they do not have any potential conflicts of interest.
Références
Sandquist M, Tegtmeyer K: No more pediatric code blues on the floor: Evolution of pediatric rapid response teams and situational awareness plans. Transl Pediatr 2018; 7:291–298
Jayaram N, Chan ML, Tang F, et al.: Frequency of medical emergency team activation prior to pediatric cardiopulmonary resuscitation. Resuscitation 2017; 115:110–115
Failure to Rescue. Agency for Healthcare Research and Quality. 2019. Available at: https://psnet.ahrq.gov/primer/failure-rescue . Accessed November 20, 2020
da Silva YS, Hamilton MF, Horvat C, et al.: Evaluation of electronic medical record vital sign data versus a commercially available acuity score in predicting need for critical intervention at a tertiary children’s hospital. Pediatr Crit Care Med 2015; 16:644–651
Pearson GA, Ward-Platt M, Harnden A, et al.: Why children die: Avoidable factors associated with child deaths. Arch Dis Child 2011; 96:927–931
Kotsakis A, Lobos AT, Parshuram C, et al.; Ontario Pediatric Critical Care Response Team Collaborative: Implementation of a multicenter rapid response system in pediatric academic hospitals is effective. Pediatrics 2011; 128:72–78
Brilli RJ, Gibson R, Luria JW, et al.: Implementation of a medical emergency team in a large pediatric teaching hospital prevents respiratory and cardiopulmonary arrests outside the intensive care unit. Pediatr Crit Care Med 2007; 8:236–246
Brady PW, Muething S, Kotagal U, et al.: Improving situation awareness to reduce unrecognized clinical deterioration and serious safety events. Pediatrics 2013; 131:e298–e308
Ben-Israel D, Jacobs WB, Casha S, et al.: The impact of machine learning on patient care: A systematic review. Artif Intell Med 2020; 103:101785
Escobar GJ, LaGuardia JC, Turk BJ, et al.: Early detection of impending physiologic deterioration among patients who are not in intensive care: Development of predictive models using data from an automated electronic medical record. J Hosp Med 2012; 7:388–395
Winter MC, Kubis S, Bonafide CP: Beyond Reporting Early Warning Score Sensitivity: The temporal relationship and clinical relevance of “true positive” alerts that precede critical deterioration. J Hosp Med 2019; 14:138–143
Gorham TJ, Rust S, Rust L, et al.: The vitals risk index-retrospective performance analysis of an automated and objective pediatric early warning system. Pediatr Qual Saf 2020; 5:e271
Sharek PJ, Parast LM, Leong K, et al.: Effect of a rapid response team on hospital-wide mortality and code rates outside the ICU in a children’s hospital. JAMA 2007; 298:2267–2274
Bonafide CP, Roberts KE, Weirich CM, et al.: Beyond statistical prediction: Qualitative evaluation of the mechanisms by which pediatric early warning scores impact patient safety. J Hosp Med 2013; 8:248–253
Chapman SM, Maconochie IK: Early warning scores in paediatrics: An overview. Arch Dis Child 2019; 104:395–399
Lambert V, Matthews A, MacDonell R, et al.: Paediatric early warning systems for detecting and responding to clinical deterioration in children: A systematic review. BMJ Open 2017; 7:e014497
Parshuram CS, Dryden-Palmer K, Farrell C, et al.; Canadian Critical Care Trials Group and the EPOCH Investigators: Effect of a pediatric early warning system on all-cause mortality in hospitalized pediatric patients: The EPOCH randomized clinical trial. JAMA 2018; 319:1002–1012
Monaghan A: Detecting and managing deterioration in children. Paediatr Nurs 2005; 17:32–35
Nuss KE, Kunar JS, Ahrens EA: Plan-Do-Study-Act methodology: Refining an inpatient pediatric sepsis screening process. Pediatr Qual Saf 2020; 5:e338
Hayes LW, Dobyns EL, DiGiovine B, et al.: A multicenter collaborative approach to reducing pediatric codes outside the ICU. Pediatrics 2012; 129:e785–e791
Jha AK: The promise of electronic records: Around the corner or down the road? JAMA 2011; 306:880–881
Lyren A, Brilli RJ, Zieker K, et al.: Children’s hospitals’ solutions for patient safety collaborative impact on hospital-acquired harm. Pediatrics 2017; 140:e20163494
Lau H, Litman KC: Saving lives by studying deaths: Using standardized mortality reviews to improve inpatient safety. Jt Comm J Qual Patient Saf 2011; 37:400–408
Fleming S, Thompson M, Stevens R, et al.: Normal ranges of heart rate and respiratory rate in children from birth to 18 years of age: A systematic review of observational studies. Lancet 2011; 377:1011–1018
Friedman J, Hastie T, Tibshirani R: Regularization paths for generalized linear models via coordinate descent. J Stat Softw 2010; 33:1–22
Tibshirani R: Regression shrinkage and selection via the lasso. J R Statist Soc B 1996; 58:267–288
Zhai H, Brady P, Li Q, et al.: Developing and evaluating a machine learning based algorithm to predict the need of pediatric intensive care unit transfer for newly hospitalized children. Resuscitation 2014; 85:1065–1071
Mayampurath A, Jani P, Dai Y, et al.: A vital sign-based model to predict clinical deterioration in hospitalized children. Pediatr Crit Care Med 2020; 21:820–826
Kipnis P, Turk BJ, Wulf DA, et al.: Development and validation of an electronic medical record-based alert score for detection of inpatient deterioration outside the ICU. J Biomed Inform 2016; 64:10–19
Wellner B, Grand J, Canzone E, et al.: Predicting unplanned transfers to the intensive care unit: A machine learning approach leveraging diverse clinical elements. JMIR Med Inform 2017; 5:e45
Rothman MJ, Rothman SI, Beals J: Development and validation of a continuous measure of patient condition using the electronic medical record. J Biomed Inform 2013; 46:837–848
Hussain FS, Sosa T, Ambroggio L, et al.: Emergency transfers: An important predictor of adverse outcomes in hospitalized children. J Hosp Med 2019; 14:482–485
Roland D: Beyond mortality: Improving outcomes for children who deteriorate in inpatient settings. J Hosp Med 2019; 14:512–513
Bonafide CP, Holmes JH, Nadkarni VM, et al.: Development of a score to predict clinical deterioration in hospitalized children. J Hosp Med 2012; 7:345–349
Brady PW, Goldenhar LM: A qualitative study examining the influences on situation awareness and the identification, mitigation and escalation of recognised patient risk. BMJ Qual Saf 2014; 23:153–161
McLellan MC, Gauvreau K, Connor JA: Validation of the Cardiac Children’s Hospital Early Warning Score: An early warning scoring tool to prevent cardiopulmonary arrests in children with heart disease. Congenit Heart Dis 2014; 9:194–202
Brady PW, Wheeler DS, Muething SE, et al.: Situation awareness: A new model for predicting and preventing patient deterioration. Hosp Pediatr 2014; 4:143–146
Braithwaite J, Wears RL, Hollnagel E: Reconciling Work-as-Imagined and Work-as-Done. Boca Raton, FL, CRC Press, Taylor & Francis Group, 2017
Leisman DE, Harhay MO, Lederer DJ, et al.: Development and reporting of prediction models: Guidance for authors from editors of respiratory, sleep, and critical care journals. Crit Care Med 2020; 48:623–633