Segmentation metric misinterpretations in bioimage analysis.
Journal
Nature methods
ISSN: 1548-7105
Titre abrégé: Nat Methods
Pays: United States
ID NLM: 101215604
Informations de publication
Date de publication:
27 Jul 2023
27 Jul 2023
Historique:
received:
26
09
2022
accepted:
06
06
2023
medline:
28
7
2023
pubmed:
28
7
2023
entrez:
27
7
2023
Statut:
aheadofprint
Résumé
Quantitative evaluation of image segmentation algorithms is crucial in the field of bioimage analysis. The most common assessment scores, however, are often misinterpreted and multiple definitions coexist with the same name. Here we present the ambiguities of evaluation metrics for segmentation algorithms and show how these misinterpretations can alter leaderboards of influential competitions. We also propose guidelines for how the currently existing problems could be tackled.
Identifiants
pubmed: 37500758
doi: 10.1038/s41592-023-01942-8
pii: 10.1038/s41592-023-01942-8
doi:
Types de publication
Journal Article
Langues
eng
Sous-ensembles de citation
IM
Informations de copyright
© 2023. The Author(s).
Références
Maier-Hein, L. et al. Metrics reloaded: pitfalls and recommendations for image analysis validation. Preprint at arXiv:2206.01653 (2022).
Mandal, S. & Uhlmann, V. SplineDist: automated cell segmentation with spline curves. In IEEE 18th International Symposium on Biomedical Imaging (ISBI) pp. 1082–1086 (IEEE, 2021).
Haque, I., Rizwan, I. & Jeremiah, N. Deep learning approaches to biomedical image segmentation. Inform. Med. Unlocked 18, 100297 (2020).
doi: 10.1016/j.imu.2020.100297
Isensee, F. et al. nnU-Net: a self-configuring method for deep learning-based biomedical image segmentation. Nat. Methods 18.2, 203–211 (2021).
doi: 10.1038/s41592-020-01008-z
Pang, H. et al. A fully automatic segmentation pipeline of pulmonary lobes before and after lobectomy from computed tomography images. Comput. Biol. Med. 147, 105792 (2022).
doi: 10.1016/j.compbiomed.2022.105792
pubmed: 35780601
Caicedo, J. C. et al. Nucleus segmentation across imaging experiments: the 2018 Data Science Bowl. Nat. Methods 16, 1247–1253 (2019) .
doi: 10.1038/s41592-019-0612-7
pubmed: 31636459
pmcid: 6919559
Edlund, C. et al. LIVECell—a large-scale dataset for label-free live cell segmentation. Nat. Methods 18.9, 1038–1045 (2021).
doi: 10.1038/s41592-021-01249-6
Aubreville, M. et al. Mitosis domain generalization in histopathology images—the MIDOG challenge. Med. Image Anal. 84, 102699 (2023).
doi: 10.1016/j.media.2022.102699
pubmed: 36463832
Upschulte, E. et al. Contour proposal networks for biomedical instance segmentation. Med. Image Anal. 77, 102371 (2022).
doi: 10.1016/j.media.2022.102371
pubmed: 35180674
Padilla, R., Netto, S. L. & Da Silva, E. A. A survey on performance metrics for object-detection algorithms. In 2020 International Conference on Systems, Signals and Image Processing (IWSSIP) pp. 237–242 (IEEE, 2020).
Everingham, M. et al. The pascal visual object classes challenge: a retrospective. Int. J. Comput. Vis. 111.1, 98–136 (2015).
doi: 10.1007/s11263-014-0733-5
Mabon, J., Ortner, M. & Zerubia, J. CNN-based energy learning for MPP object detection in satellite images. In 2022 IEEE 32nd International Workshop on Machine Learning for Signal Processing (MLSP) pp. 1–6 (IEEE, 2022).
He, K. et al. Mask R-CNN. In Proc. International Conference on Computer Vision (ICCV) pp. 2961–2969 (2017).
He, K. et al. Deep residual learning for image recognition. In Proc. of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) pp. 770–778 (2016).
Schmidt, U. et al. Cell detection with star-convex polygons. In Proc. 21st International Conference, Medical Image Computing and Computer Assisted Intervention (MICCAI) pp. 265–273 (Springer International Publishing, 2018).
Stringer, C. et al. Cellpose: a generalist algorithm for cellular segmentation. Nat. Methods 18.1, 100–106 (2021).
doi: 10.1038/s41592-020-01018-x
Grexa, I. et al. SpheroidPicker for automated 3D cell culture manipulation using deep learning. Sci. Rep. 11, 14813 (2021).
doi: 10.1038/s41598-021-94217-1
pubmed: 34285291
pmcid: 8292460
Moshkov, N. et al. Test-time augmentation for deep learning-based cell segmentation on microscopy images. Sci. Rep. 10, 5068 (2020).
doi: 10.1038/s41598-020-61808-3
pubmed: 32193485
pmcid: 7081314
Hollandi, R. et al. nucleAIzer: a parameter-free deep learning framework for nucleus segmentation using image style transfer. Cell Syst. 10.5, 453–458 (2020).
doi: 10.1016/j.cels.2020.04.003
Barker, J. S. P. Deep learning for object detection with DIGITS. NVIDIA Developer Technical Blog https://developer.nvidia.com/blog/deep-learning-object-detection-digits/ (2016).
Lalit, M., Tomancak, P. & Jug, F. EmbedSeg: embedding-based instance segmentation for biomedical microscopy data. Med. Image Anal. 81, 102523 (2022).
doi: 10.1016/j.media.2022.102523
pubmed: 35926335
Lin, T. Y. et al. Microsoft COCO: common objects in context. Computer Vision – ECCV 2014, pp. 740–755 (Springer International Publishing, 2014).
Common objects in context. COCO dataset https://cocodataset.org/#detection-eval (n.d.).