Predicting outcomes following lower extremity open revascularization using machine learning.
Journal
Scientific reports
ISSN: 2045-2322
Titre abrégé: Sci Rep
Pays: England
ID NLM: 101563288
Informations de publication
Date de publication:
05 Feb 2024
05 Feb 2024
Historique:
received:
11
04
2023
accepted:
25
01
2024
medline:
6
2
2024
pubmed:
6
2
2024
entrez:
5
2
2024
Statut:
epublish
Résumé
Lower extremity open revascularization is a treatment option for peripheral artery disease that carries significant peri-operative risks; however, outcome prediction tools remain limited. Using machine learning (ML), we developed automated algorithms that predict 30-day outcomes following lower extremity open revascularization. The National Surgical Quality Improvement Program targeted vascular database was used to identify patients who underwent lower extremity open revascularization for chronic atherosclerotic disease between 2011 and 2021. Input features included 37 pre-operative demographic/clinical variables. The primary outcome was 30-day major adverse limb event (MALE; composite of untreated loss of patency, major reintervention, or major amputation) or death. Our data were split into training (70%) and test (30%) sets. Using tenfold cross-validation, we trained 6 ML models. Overall, 24,309 patients were included. The primary outcome of 30-day MALE or death occurred in 2349 (9.3%) patients. Our best performing prediction model was XGBoost, achieving an area under the receiver operating characteristic curve (95% CI) of 0.93 (0.92-0.94). The calibration plot showed good agreement between predicted and observed event probabilities with a Brier score of 0.08. Our ML algorithm has potential for important utility in guiding risk mitigation strategies for patients being considered for lower extremity open revascularization to improve outcomes.
Identifiants
pubmed: 38316811
doi: 10.1038/s41598-024-52944-1
pii: 10.1038/s41598-024-52944-1
doi:
Types de publication
Journal Article
Langues
eng
Sous-ensembles de citation
IM
Pagination
2899Subventions
Organisme : CIHR
ID : Canada Graduate Scholarships
Pays : Canada
Informations de copyright
© 2024. The Author(s).
Références
Zemaitis, M. R., Boll, J. M. & Dreyer, M. A. Peripheral Arterial Disease. in StatPearls (StatPearls Publishing, 2021).
Fowkes, F. G. R. et al. Comparison of global estimates of prevalence and risk factors for peripheral artery disease in 2000 and 2010: A systematic review and analysis. Lancet Lond. Engl. 382, 1329–1340 (2013).
doi: 10.1016/S0140-6736(13)61249-0
Agnelli, G., Belch, J. J. F., Baumgartner, I., Giovas, P. & Hoffmann, U. Morbidity and mortality associated with atherosclerotic peripheral artery disease: A systematic review. Atherosclerosis 293, 94–100 (2020).
pubmed: 31606132
doi: 10.1016/j.atherosclerosis.2019.09.012
Kim, M., Kim, Y., Ryu, G. W. & Choi, M. Functional status and health-related quality of life in patients with peripheral artery disease: A Cross-sectional study. Int. J. Environ. Res. Public. Health 18, 10941 (2021).
pubmed: 34682683
pmcid: 8535998
doi: 10.3390/ijerph182010941
Kohn, C. G., Alberts, M. J., Peacock, W. F., Bunz, T. J. & Coleman, C. I. Cost and inpatient burden of peripheral artery disease: Findings from the national inpatient sample. Atherosclerosis 286, 142–146 (2019).
pubmed: 31170647
doi: 10.1016/j.atherosclerosis.2019.05.026
Farber, A. et al. Surgery or endovascular therapy for chronic limb-threatening ischemia. N. Engl. J. Med. https://doi.org/10.1056/NEJMoa2207899 (2022).
doi: 10.1056/NEJMoa2207899
pubmed: 36342173
Liang, P. et al. In-hospital versus postdischarge major adverse events within 30 days following lower extremity revascularization. J. Vasc. Surg. 69, 482–489 (2019).
pubmed: 30301689
doi: 10.1016/j.jvs.2018.06.207
Conte, M. S. et al. Global vascular guidelines on the management of chronic limb-threatening ischemia. J. Vasc. Surg. 69, 3S-125S.e40 (2019).
pubmed: 31159978
pmcid: 8365864
doi: 10.1016/j.jvs.2019.02.016
Perkins, Z. B. et al. Predicting the outcome of limb revascularization in patients with lower-extremity arterial trauma: Development and external validation of a supervised machine-learning algorithm to support surgical decisions. Ann. Surg. 272, 564–572 (2020).
pubmed: 32657917
doi: 10.1097/SLA.0000000000004132
Miyata, T. et al. Risk prediction model for early outcomes of revascularization for chronic limb-threatening ischaemia. Br. J. Surg. 108, 941–950 (2021).
pubmed: 33693591
doi: 10.1093/bjs/znab036
Biancari, F. et al. Risk-scoring method for prediction of 30-day postoperative outcome after infrainguinal surgical revascularization for critical lower-limb ischemia: A Finnvasc registry study. World J. Surg. 31, 217–225 (2007).
pubmed: 17171494
doi: 10.1007/s00268-006-0242-y
Bennett, K. M., Levinson, H., Scarborough, J. E. & Shortell, C. K. Validated prediction model for severe groin wound infection after lower extremity revascularization procedures. J. Vasc. Surg. 63, 414–419 (2016).
pubmed: 26526055
doi: 10.1016/j.jvs.2015.08.094
Bilimoria, K. Y. et al. Development and evaluation of the universal ACS NSQIP surgical risk calculator: A decision aid and informed consent tool for patients and surgeons. J. Am. Coll. Surg. 217, 833–842 (2013).
pubmed: 24055383
pmcid: 3805776
doi: 10.1016/j.jamcollsurg.2013.07.385
Bertges, D. et al. The vascular study group of new england cardiac risk index (VSG-CRI) predicts cardiac complications more accurately than the Revised Cardiac Risk Index in vascular surgery patients. J. Vasc. Surg. 52, (2010).
Sharma, V. et al. Adoption of clinical risk prediction tools is limited by a lack of integration with electronic health records. BMJ Health Care Inform. 28, e100253 (2021).
pubmed: 33608259
pmcid: 7898839
doi: 10.1136/bmjhci-2020-100253
Baştanlar, Y. & Özuysal, M. Introduction to machine learning. Methods Mol. Biol. 1107, 105–128 (2014).
pubmed: 24272434
doi: 10.1007/978-1-62703-748-8_7
Shah, P. et al. Artificial intelligence and machine learning in clinical development: A translational perspective. NPJ Digit. Med. 2, 69 (2019).
pubmed: 31372505
pmcid: 6659652
doi: 10.1038/s41746-019-0148-3
Bonde, A. et al. Assessing the utility of deep neural networks in predicting postoperative surgical complications: A retrospective study. Lancet Digit. Health 3, e471–e485 (2021).
pubmed: 34215564
doi: 10.1016/S2589-7500(21)00084-4
World Medical Association. World Medical Association Declaration of Helsinki: Ethical principles for medical research involving human subjects. JAMA 310, 2191–2194 (2013).
doi: 10.1001/jama.2013.281053
Tfaily, M. A., Ghanem, P., Farran, S. H., Dabdoub, F. & Kanafani, Z. A. The role of preoperative albumin and white blood cell count in surgical site infections following whipple surgery. Sci. Rep. 12, 19184 (2022).
pubmed: 36357432
pmcid: 9649662
doi: 10.1038/s41598-022-21849-2
Collins, G. S., Reitsma, J. B., Altman, D. G. & Moons, K. G. M. Transparent Reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): The TRIPOD statement. Ann. Intern. Med. 162, 55–63 (2015).
pubmed: 25560714
doi: 10.7326/M14-0697
ACS NSQIP. ACS https://www.facs.org/quality-programs/data-and-registries/acs-nsqip/ .
Shiloach, M. et al. Toward robust information: Data quality and inter-rater reliability in the American college of surgeons national surgical quality improvement program. J. Am. Coll. Surg. 210, 6–16 (2010).
pubmed: 20123325
doi: 10.1016/j.jamcollsurg.2009.09.031
Cohen, M. E. et al. Optimizing ACS NSQIP modeling for evaluation of surgical quality and risk: Patient risk adjustment, procedure mix adjustment, shrinkage adjustment, and surgical focus. J. Am. Coll. Surg. 217, 336-346.e1 (2013).
pubmed: 23628227
doi: 10.1016/j.jamcollsurg.2013.02.027
ACS NSQIP Participant Use Data File. ACS https://www.facs.org/quality-programs/data-and-registries/acs-nsqip/participant-use-data-file/ .
Elfanagely, O. et al. Machine learning and surgical outcomes prediction: A systematic review. J. Surg. Res. 264, 346–361 (2021).
pubmed: 33848833
doi: 10.1016/j.jss.2021.02.045
Bektaş, M., Tuynman, J. B., Costa Pereira, J., Burchell, G. L. & van der Peet, D. L. Machine learning algorithms for predicting surgical outcomes after colorectal surgery: A systematic review. World J. Surg. https://doi.org/10.1007/s00268-022-06728-1 (2022).
doi: 10.1007/s00268-022-06728-1
pubmed: 36109367
pmcid: 9636121
Senders, J. T. et al. Machine learning and neurosurgical outcome prediction: A systematic review. World Neurosurg. 109, 476-486.e1 (2018).
pubmed: 28986230
doi: 10.1016/j.wneu.2017.09.149
Chen, T. & Guestrin, C. XGBoost: A Scalable Tree Boosting System. in Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining 785–794 (2016). doi: https://doi.org/10.1145/2939672.2939785 .
Rigatti, S. J. Random forest. J. Insur. Med. N. Y. N 47, 31–39 (2017).
Zhang, Z. Naïve bayes classification in R. Ann. Transl. Med. 4, 241 (2016).
pubmed: 27429967
pmcid: 4930525
doi: 10.21037/atm.2016.03.38
Noble, W. S. What is a support vector machine?. Nat. Biotechnol. 24, 1565–1567 (2006).
pubmed: 17160063
doi: 10.1038/nbt1206-1565
Schmidhuber, J. Deep learning in neural networks: an overview. Neural Netw. Off. J. Int. Neural Netw. Soc. 61, 85–117 (2015).
doi: 10.1016/j.neunet.2014.09.003
Sperandei, S. Understanding logistic regression analysis. Biochem. Medica 24, 12–18 (2014).
doi: 10.11613/BM.2014.003
Ngiam, K. Y. & Khor, I. W. Big data and machine learning algorithms for health-care delivery. Lancet Oncol. 20, e262–e273 (2019).
pubmed: 31044724
doi: 10.1016/S1470-2045(19)30149-4
Liew, B. X. W., Kovacs, F. M., Rügamer, D. & Royuela, A. Machine learning versus logistic regression for prognostic modelling in individuals with non-specific neck pain. Eur. Spine J. Off. Publ. Eur. Spine Soc. Eur. Spine Res. Soc. 31, 2082–2091 (2022).
doi: 10.1007/s00586-022-07188-w
Shipe, M. E., Deppen, S. A., Farjah, F. & Grogan, E. L. Developing prediction models for clinical use using logistic regression: An overview. J. Thorac. Dis. 11, S574–S584 (2019).
pubmed: 31032076
pmcid: 6465431
doi: 10.21037/jtd.2019.01.25
Dobbin, K. K. & Simon, R. M. Optimally splitting cases for training and testing high dimensional classifiers. BMC Med. Genomics 4, 31 (2011).
pubmed: 21477282
pmcid: 3090739
doi: 10.1186/1755-8794-4-31
Jung, Y. & Hu, J. A K-fold averaging cross-validation procedure. J. Nonparametric Stat. 27, 167–179 (2015).
doi: 10.1080/10485252.2015.1010532
Adnan, M., Alarood, A. A. S., Uddin, M. I. & Ur Rehman, I. Utilizing grid search cross-validation with adaptive boosting for augmenting performance of machine learning models. PeerJ Comput. Sci. 8, e803 (2022).
pubmed: 35494796
pmcid: 9044349
doi: 10.7717/peerj-cs.803
Wibowo, P. & Fatichah, C. Pruning-based oversampling technique with smoothed bootstrap resampling for imbalanced clinical dataset of Covid-19. J. King Saud Univ.–Comput. Inf. Sci. 34, 7830–7839 (2022).
Hajian-Tilaki, K. Receiver operating characteristic (ROC) curve analysis for medical diagnostic test evaluation. Casp. J. Intern. Med. 4, 627–635 (2013).
Redelmeier, D. A., Bloch, D. A. & Hickam, D. H. Assessing predictive accuracy: How to compare brier scores. J. Clin. Epidemiol. 44, 1141–1146 (1991).
pubmed: 1941009
doi: 10.1016/0895-4356(91)90146-Z
Loh, W.-Y. & Zhou, P. Variable importance scores. J. Data Sci. 19, 569–592 (2021).
doi: 10.6339/21-JDS1023
Riley, R. D. et al. Calculating the sample size required for developing a clinical prediction model. BMJ m441 (2020) https://doi.org/10.1136/bmj.m441 .
Ensor, J., Martin, E. C. & Riley, R. D. Pmsampsize: Calculates the Minimum Sample Size Required for Developing a Multivariable Prediction Model. (2022).
Schafer, J. L. Multiple imputation: A primer. Stat. Methods Med. Res. 8, 3–15 (1999).
pubmed: 10347857
doi: 10.1177/096228029900800102
Ross, R. K., Breskin, A. & Westreich, D. When is a complete-case approach to missing data valid? The importance of effect-measure modification. Am. J. Epidemiol. 189, 1583–1589 (2020).
pubmed: 32601706
pmcid: 7705610
doi: 10.1093/aje/kwaa124
Hughes, R. A., Heron, J., Sterne, J. A. C. & Tilling, K. Accounting for missing data in statistical analyses: Multiple imputation is not always the answer. Int. J. Epidemiol. 48, 1294–1304 (2019).
pubmed: 30879056
pmcid: 6693809
doi: 10.1093/ije/dyz032
R Core Team (2022). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. https://www.R-project.org/ .
Kuhn, M. et al. Caret: Classification and Regression Training. (2022).
Chen, T. & Guestrin, C. XGBoost: A scalable tree boosting system. Proc. 22nd ACM SIGKDD Int. Conf. Knowl. Discov. Data Min.–KDD 16, 785–794 (2016).
doi: 10.1145/2939672.2939785
Wright, M. N., Wager, S. & Probst, P. Ranger: A Fast Implementation of Random Forests. (2022).
Naivebayes: High Performance Implementation of the Naive Bayes Algorithm version 0.9.7 from CRAN. https://rdrr.io/cran/naivebayes/ .
svm function - RDocumentation. https://www.rdocumentation.org/packages/e1071/versions/1.7-11/topics/svm .
Ripley, B. & Venables, W. Nnet: Feed-Forward Neural Networks and Multinomial Log-Linear Models. (2022).
Robin, X. et al. pROC: An open-source package for R and S+ to analyze and compare ROC curves. BMC Bioinformat. 12, 77 (2011).
doi: 10.1186/1471-2105-12-77
O’Connor, D. B. et al. An anaesthetic pre-operative assessment clinic reduces pre-operative inpatient stay in patients requiring major vascular surgery. Ir. J. Med. Sci. 180, 649–653 (2011).
pubmed: 21431923
doi: 10.1007/s11845-011-0703-7
Davis, F. M. et al. The clinical impact of cardiology consultation prior to major vascular surgery. Ann. Surg. 267, 189–195 (2018).
pubmed: 29240607
doi: 10.1097/SLA.0000000000002014
Bonaca, M. P. et al. Rivaroxaban in peripheral artery disease after revascularization. N. Engl. J. Med. 382, 1994–2004 (2020).
pubmed: 32222135
doi: 10.1056/NEJMoa2000052
Conte, M. S. et al. Society for vascular surgery practice guidelines for atherosclerotic occlusive disease of the lower extremities: Management of asymptomatic disease and claudication. J. Vasc. Surg. 61, 2S-41S (2015).
pubmed: 25638515
doi: 10.1016/j.jvs.2014.12.009
Gerhard-Herman, M. D. et al. 2016 AHA/ACC guideline on the management of patients with lower extremity peripheral artery disease: Executive summary: A report of the american college of cardiology/American heart association task force on clinical practice guidelines. Circulation 135, e686–e725 (2017).
pubmed: 27840332
Aboyans, V. et al. Editor’s choice–2017 ESC guidelines on the diagnosis and treatment of peripheral arterial diseases, in collaboration with the European society for vascular surgery (ESVS). Eur. J. Vasc. Endovasc. Surg. Off. J. Eur. Soc. Vasc. Surg. 55, 305–368 (2018).
doi: 10.1016/j.ejvs.2017.07.018
Stoltzfus, J. C. Logistic regression: A brief primer. Acad. Emerg. Med. Off. J. Soc. Acad. Emerg. Med. 18, 1099–1104 (2011).
doi: 10.1111/j.1553-2712.2011.01185.x
Kia, B. et al. Nonlinear dynamics based machine learning: Utilizing dynamics-based flexibility of nonlinear circuits to implement different functions. PloS One 15, e0228534 (2020).
pubmed: 32126089
pmcid: 7053732
doi: 10.1371/journal.pone.0228534
Chatterjee, P. et al. Nonlinear systems in healthcare towards intelligent disease prediction. Nonlinear Syst.–Theor. Asp. Recent Appl. https://doi.org/10.5772/intechopen.88163 (2019).
doi: 10.5772/intechopen.88163
Ravaut, M. et al. Predicting adverse outcomes due to diabetes complications with machine learning using administrative health data. Npj Digit. Med. 4, 1–12 (2021).
doi: 10.1038/s41746-021-00394-8
Wang, R., Zhang, J., Shan, B., He, M. & Xu, J. XGBoost machine learning algorithm for prediction of outcome in aneurysmal subarachnoid hemorrhage. Neuropsychiatr. Dis. Treat. 18, 659–667 (2022).
pubmed: 35378822
pmcid: 8976557
doi: 10.2147/NDT.S349956
Fang, Z.-G., Yang, S.-Q., Lv, C.-X., An, S.-Y. & Wu, W. Application of a data-driven XGBoost model for the prediction of COVID-19 in the USA: A time-series study. BMJ Open 12, e056685 (2022).
pubmed: 35777884
doi: 10.1136/bmjopen-2021-056685
Viljanen, M., Meijerink, L., Zwakhals, L. & van de Kassteele, J. A machine learning approach to small area estimation: Predicting the health, housing and well-being of the population of Netherlands. Int. J. Health Geogr. 21, 4 (2022).
pubmed: 35668432
pmcid: 9169293
doi: 10.1186/s12942-022-00304-5
Gianfrancesco, M. A., Tamang, S., Yazdany, J. & Schmajuk, G. Potential biases in machine learning algorithms using electronic health record data. JAMA Intern. Med. 178, 1544–1547 (2018).
pubmed: 30128552
pmcid: 6347576
doi: 10.1001/jamainternmed.2018.3763
Mazmudar, A., Vitello, D., Chapman, M., Tomlinson, J. S. & Bentrem, D. J. Gender as a risk factor for adverse intraoperative and postoperative outcomes of elective pancreatectomy. J. Surg. Oncol. 115, 131–136 (2017).
pubmed: 27813101
doi: 10.1002/jso.24488
Halsey, J. N., Asti, L. & Kirschner, R. E. The impact of race and ethnicity on surgical risk and outcomes following palatoplasty: An analysis of the nsqip pediatric database. Cleft Palate-Craniofac. J. Off. Publ. Am. Cleft Palate 52, 255. https://doi.org/10.1177/10556656221078154 (2022).
doi: 10.1177/10556656221078154
Rümenapf, G., Morbach, S., Schmidt, A. & Sigl, M. Intermittent claudication and asymptomatic peripheral arterial disease. Dtsch. Ärztebl. Int. 117, 188–193 (2020).
pubmed: 32327031
pmcid: 7191120
Shaydakov, M. E. & Tuma, F. Operative Risk. in StatPearls (StatPearls Publishing, 2022).
Bevan, G. H. & WhiteSolaru, K. T. Evidence-based medical management of peripheral artery disease. Arterioscler. Thromb. Vasc. Biol. 40, 541–553 (2020).
pubmed: 31996023
doi: 10.1161/ATVBAHA.119.312142
Biscetti, F. et al. Outcomes of lower extremity endovascular revascularization: Potential predictors and prevention strategies. Int. J. Mol. Sci. 22, 2002 (2021).
pubmed: 33670461
pmcid: 7922574
doi: 10.3390/ijms22042002
Roberts, D. J. et al. Association between neuraxial anaesthesia or general anaesthesia for lower limb revascularisation surgery in adults and clinical outcomes: Population based comparative effectiveness study. BMJ 371, m4104 (2020).
pubmed: 33239330
pmcid: 7687020
doi: 10.1136/bmj.m4104
Gillies, M. A. et al. Intensive care utilization and outcomes after high-risk surgery in Scotland: A population-based cohort study. Br. J. Anaesth. 118, 123–131 (2017).
pubmed: 28039249
doi: 10.1093/bja/aew396
Patel, P. R. & Bechmann, S. Discharge Planning. in StatPearls (StatPearls Publishing, 2022).
Perez-Riverol, Y. et al. Ten simple rules for taking advantage of git and GitHub. PLoS Comput. Biol. 12, e1004947 (2016).
pubmed: 27415786
pmcid: 4945047
doi: 10.1371/journal.pcbi.1004947
Nguyen, L. L. & Barshes, N. R. Analysis of large databases in vascular surgery. J. Vasc. Surg. 52, 768–774 (2010).
pubmed: 20598475
doi: 10.1016/j.jvs.2010.03.027
Northridge, M. E. & Metcalf, S. S. Enhancing implementation science by applying best principles of systems science. Health Res. Policy Syst. 14, 74 (2016).
pubmed: 27716275
pmcid: 5050576
doi: 10.1186/s12961-016-0146-8
Bronsert, M. et al. Identification of postoperative complications using electronic health record data and machine learning. Am. J. Surg. 220, 114–119 (2020).
pubmed: 31635792
doi: 10.1016/j.amjsurg.2019.10.009
Colquhoun, D. A. et al. Considerations for integration of perioperative electronic health records across institutions for research and quality improvement: The approach taken by the multicenter perioperative outcomes group. Anesth. Analg. 130, 1133–1146 (2020).
pubmed: 32287121
pmcid: 7663856
doi: 10.1213/ANE.0000000000004489
Batko, K. & Ślęzak, A. The use of big data analytics in healthcare. J. Big Data 9, 3 (2022).
pubmed: 35013701
pmcid: 8733917
doi: 10.1186/s40537-021-00553-4
Leung, S. N. et al. Harnessing the full potential of hospital-based data to support surgical quality improvement. BMJ Open Qual. 10, e001178 (2021).
pubmed: 34697037
pmcid: 8547498
doi: 10.1136/bmjoq-2020-001178
Eikelboom, J. W. et al. Rivaroxaban with or without aspirin in stable cardiovascular disease. N. Engl. J. Med. 377, 1319–1330 (2017).
pubmed: 28844192
doi: 10.1056/NEJMoa1709118