TY  - EJOUR
AU  - Reinke, Annika
AU  - Tizabi, Minu D.
AU  - Baumgartner, Michael
AU  - Eisenmann, Matthias
AU  - Heckmann-Nötzel, Doreen
AU  - Kavur, A. Emre
AU  - Rädsch, Tim
AU  - Sudre, Carole H.
AU  - Acion, Laura
AU  - Antonelli, Michela
AU  - Arbel, Tal
AU  - Bakas, Spyridon
AU  - Benis, Arriel
AU  - Blaschko, Matthew
AU  - Büttner, Florian
AU  - Cardoso, M. Jorge
AU  - Cheplygina, Veronika
AU  - Chen, Jianxu
AU  - Christodoulou, Evangelia
AU  - Cimini, Beth A.
AU  - Collins, Gary S.
AU  - Farahani, Keyvan
AU  - Ferrer, Luciana
AU  - Galdran, Adrian
AU  - van Ginneken, Bram
AU  - Glocker, Ben
AU  - Godau, Patrick
AU  - Haase, Robert
AU  - Hashimoto, Daniel A.
AU  - Hoffman, Michael M.
AU  - Huisman, Merel
AU  - Isensee, Fabian
AU  - Jannin, Pierre
AU  - Kahn, Charles E.
AU  - Kainmueller, Dagmar
AU  - Kainz, Bernhard
AU  - Karargyris, Alexandros
AU  - Karthikesalingam, Alan
AU  - Kenngott, Hannes
AU  - Kleesiek, Jens
AU  - Kofler, Florian
AU  - Kooi, Thijs
AU  - Kopp-Schneider, Annette
AU  - Kozubek, Michal
AU  - Kreshuk, Anna
AU  - Kurc, Tahsin
AU  - Landman, Bennett A.
AU  - Litjens, Geert
AU  - Madani, Amin
AU  - Maier-Hein, Klaus
AU  - Martel, Anne L.
AU  - Mattson, Peter
AU  - Meijering, Erik
AU  - Menze, Bjoern
AU  - Moons, Karel G. M.
AU  - Müller, Henning
AU  - Nichyporuk, Brennan
AU  - Nickel, Felix
AU  - Petersen, Jens
AU  - Rafelski, Susanne M.
AU  - Rajpoot, Nasir
AU  - Reyes, Mauricio
AU  - Riegler, Michael A.
AU  - Rieke, Nicola
AU  - Saez-Rodriguez, Julio
AU  - Sánchez, Clara I.
AU  - Shetty, Shravya
AU  - van Smeden, Maarten
AU  - Summers, Ronald M.
AU  - Taha, Abdel A.
AU  - Tiulpin, Aleksei
AU  - Tsaftaris, Sotirios A.
AU  - Van Calster, Ben
AU  - Varoquaux, Gaël
AU  - Wiesenfarth, Manuel
AU  - Yaniv, Ziv R.
AU  - Jäger, Paul F.
AU  - Maier-Hein, Lena
TI  - Understanding metric-related pitfalls in image analysis validation
IS  - arXiv:2302.01790
PB  - arXiv
M1  - DKFZ-2023-01783
M1  - arXiv:2302.01790
PY  - 2023
N1  - arXiv:2302.01790 [cs.CV]  	(or arXiv:2302.01790v2 [cs.CV] for this version)  	https://doi.org/10.48550/arXiv.2302.01790
AB  - Validation metrics are key for the reliable tracking of scientific progress and for bridging the current chasm between artificial intelligence (AI) research and its translation into practice. However, increasing evidence shows that particularly in image analysis, metrics are often chosen inadequately in relation to the underlying research problem. This could be attributed to a lack of accessibility of metric-related knowledge: While taking into account the individual strengths, weaknesses, and limitations of validation metrics is a critical prerequisite to making educated choices, the relevant knowledge is currently scattered and poorly accessible to individual researchers. Based on a multi-stage Delphi process conducted by a multidisciplinary expert consortium as well as extensive community feedback, the present work provides the first reliable and comprehensive common point of access to information on pitfalls related to validation metrics in image analysis. Focusing on biomedical image analysis but with the potential of transfer to other fields, the addressed pitfalls generalize across application domains and are categorized according to a newly created, domain-agnostic taxonomy. To facilitate comprehension, illustrations and specific examples accompany each pitfall. As a structured body of information accessible to researchers of all levels of expertise, this work enhances global comprehension of a key topic in image analysis validation.
KW  - Computer Vision and Pattern Recognition (cs.CV) (Other)
KW  - FOS: Computer and information sciences (Other)
LB  - PUB:(DE-HGF)25
DO  - DOI:doi.org/10.48550/arXiv.2302.01790
UR  - https://inrepo02.dkfz.de/record/282494
ER  -