guest ::
| Home > Publications database > Understanding metric-related pitfalls in image analysis validation. > print |
| Home > Publications database > Understanding metric-related pitfalls in image analysis validation. > print |
| 001 | 288084 | ||
| 005 | 20250326132557.0 | ||
| 024 | 7 | _ | |a 10.1038/s41592-023-02150-0 |2 doi |
| 024 | 7 | _ | |a pmid:38347140 |2 pmid |
| 024 | 7 | _ | |a 1548-7091 |2 ISSN |
| 024 | 7 | _ | |a 1548-7105 |2 ISSN |
| 024 | 7 | _ | |a altmetric:159461768 |2 altmetric |
| 037 | _ | _ | |a DKFZ-2024-00338 |
| 041 | _ | _ | |a English |
| 082 | _ | _ | |a 610 |
| 100 | 1 | _ | |a Reinke, Annika |0 P:(DE-He78)97e904f47dab556a77c0149cd0002591 |b 0 |e First author |
| 245 | _ | _ | |a Understanding metric-related pitfalls in image analysis validation. |
| 260 | _ | _ | |a London [u.a.] |c 2024 |b Nature Publishing Group |
| 336 | 7 | _ | |a article |2 DRIVER |
| 336 | 7 | _ | |a Output Types/Journal article |2 DataCite |
| 336 | 7 | _ | |a Journal Article |b journal |m journal |0 PUB:(DE-HGF)16 |s 1710237361_32556 |2 PUB:(DE-HGF) |x Review Article |
| 336 | 7 | _ | |a ARTICLE |2 BibTeX |
| 336 | 7 | _ | |a JOURNAL_ARTICLE |2 ORCID |
| 336 | 7 | _ | |a Journal Article |0 0 |2 EndNote |
| 500 | _ | _ | |a #EA:E130#LA:E130#LA:E290# |
| 520 | _ | _ | |a Validation metrics are key for tracking scientific progress and bridging the current chasm between artificial intelligence research and its translation into practice. However, increasing evidence shows that, particularly in image analysis, metrics are often chosen inadequately. Although taking into account the individual strengths, weaknesses and limitations of validation metrics is a critical prerequisite to making educated choices, the relevant knowledge is currently scattered and poorly accessible to individual researchers. Based on a multistage Delphi process conducted by a multidisciplinary expert consortium as well as extensive community feedback, the present work provides a reliable and comprehensive common point of access to information on pitfalls related to validation metrics in image analysis. Although focused on biomedical image analysis, the addressed pitfalls generalize across application domains and are categorized according to a newly created, domain-agnostic taxonomy. The work serves to enhance global comprehension of a key topic in image analysis validation. |
| 536 | _ | _ | |a 315 - Bildgebung und Radioonkologie (POF4-315) |0 G:(DE-HGF)POF4-315 |c POF4-315 |f POF IV |x 0 |
| 588 | _ | _ | |a Dataset connected to CrossRef, PubMed, , Journals: inrepo02.dkfz.de |
| 700 | 1 | _ | |a Tizabi, Minu Dietlinde |0 P:(DE-He78)26651d9aa10255ad4f35610a56aa91e8 |b 1 |e First author |
| 700 | 1 | _ | |a Baumgartner, Michael |0 P:(DE-He78)6ae68adec0915a4ae2d58aff814aaceb |b 2 |
| 700 | 1 | _ | |a Eisenmann, Matthias |0 P:(DE-He78)c9d6245b17f0ab26eeed345cb00d3359 |b 3 |
| 700 | 1 | _ | |a Heckmann-Nötzel, Doreen |0 P:(DE-He78)98dd16a42af0888dd129e1cd2ee2766a |b 4 |
| 700 | 1 | _ | |a Kavur, Ali Emre |0 P:(DE-He78)64296d0922a2da68074f5de2ccf74487 |b 5 |
| 700 | 1 | _ | |a Rädsch, Tim |0 P:(DE-He78)683f71d452c83a7f5ec62969c6012466 |b 6 |
| 700 | 1 | _ | |a Sudre, Carole H |b 7 |
| 700 | 1 | _ | |a Acion, Laura |0 0000-0001-5213-6012 |b 8 |
| 700 | 1 | _ | |a Antonelli, Michela |0 0000-0002-3005-4523 |b 9 |
| 700 | 1 | _ | |a Arbel, Tal |0 0000-0001-8870-3007 |b 10 |
| 700 | 1 | _ | |a Bakas, Spyridon |0 0000-0001-8734-6482 |b 11 |
| 700 | 1 | _ | |a Benis, Arriel |0 0000-0002-9125-8300 |b 12 |
| 700 | 1 | _ | |a Büttner, Florian |0 P:(DE-He78)fdeeec93551e8f3bad68db88a1130c5d |b 13 |u dkfz |
| 700 | 1 | _ | |a Cardoso, M Jorge |0 0000-0003-1284-2558 |b 14 |
| 700 | 1 | _ | |a Cheplygina, Veronika |b 15 |
| 700 | 1 | _ | |a Chen, Jianxu |0 0000-0002-8500-1357 |b 16 |
| 700 | 1 | _ | |a Christodoulou, Evangelia |0 P:(DE-He78)8da2eca0bc6341c8681c317fe2b8e27b |b 17 |
| 700 | 1 | _ | |a Cimini, Beth A |0 0000-0001-9640-9318 |b 18 |
| 700 | 1 | _ | |a Farahani, Keyvan |b 19 |
| 700 | 1 | _ | |a Ferrer, Luciana |b 20 |
| 700 | 1 | _ | |a Galdran, Adrian |b 21 |
| 700 | 1 | _ | |a van Ginneken, Bram |b 22 |
| 700 | 1 | _ | |a Glocker, Ben |0 0000-0002-4897-9356 |b 23 |
| 700 | 1 | _ | |a Godau, Patrick |0 P:(DE-He78)77a2a5b07dcbd46277a18a32372ea154 |b 24 |
| 700 | 1 | _ | |a Hashimoto, Daniel A |0 0000-0003-4725-3104 |b 25 |
| 700 | 1 | _ | |a Hoffman, Michael M |0 0000-0002-4517-1562 |b 26 |
| 700 | 1 | _ | |a Huisman, Merel |b 27 |
| 700 | 1 | _ | |a Isensee, Fabian |0 P:(DE-He78)7ea9af59d03ec7deb982a0e0562358fa |b 28 |
| 700 | 1 | _ | |a Jannin, Pierre |0 0000-0002-7415-071X |b 29 |
| 700 | 1 | _ | |a Kahn, Charles E |0 0000-0002-6654-7434 |b 30 |
| 700 | 1 | _ | |a Kainmueller, Dagmar |b 31 |
| 700 | 1 | _ | |a Kainz, Bernhard |b 32 |
| 700 | 1 | _ | |a Karargyris, Alexandros |0 0000-0002-1930-3410 |b 33 |
| 700 | 1 | _ | |a Kleesiek, Jens |b 34 |
| 700 | 1 | _ | |a Kofler, Florian |b 35 |
| 700 | 1 | _ | |a Kooi, Thijs |b 36 |
| 700 | 1 | _ | |a Kopp-Schneider, Annette |0 P:(DE-He78)bb6a7a70f976eb8df1769944bf913596 |b 37 |
| 700 | 1 | _ | |a Kozubek, Michal |0 0000-0001-7902-589X |b 38 |
| 700 | 1 | _ | |a Kreshuk, Anna |0 0000-0003-1334-6388 |b 39 |
| 700 | 1 | _ | |a Kurc, Tahsin |b 40 |
| 700 | 1 | _ | |a Landman, Bennett A |0 0000-0001-5733-2127 |b 41 |
| 700 | 1 | _ | |a Litjens, Geert |0 0000-0003-1554-1291 |b 42 |
| 700 | 1 | _ | |a Madani, Amin |b 43 |
| 700 | 1 | _ | |a Maier-Hein, Klaus |0 P:(DE-He78)33c74005e1ce56f7025c4f6be15321b3 |b 44 |
| 700 | 1 | _ | |a Martel, Anne L |0 0000-0003-1375-5501 |b 45 |
| 700 | 1 | _ | |a Meijering, Erik |0 0000-0001-8015-8358 |b 46 |
| 700 | 1 | _ | |a Menze, Bjoern |0 0000-0003-4136-5690 |b 47 |
| 700 | 1 | _ | |a Moons, Karel G M |b 48 |
| 700 | 1 | _ | |a Müller, Henning |0 0000-0001-6800-9878 |b 49 |
| 700 | 1 | _ | |a Nichyporuk, Brennan |0 0009-0006-8087-6089 |b 50 |
| 700 | 1 | _ | |a Nickel, Felix |b 51 |
| 700 | 1 | _ | |a Petersen, Jens |b 52 |
| 700 | 1 | _ | |a Rafelski, Susanne M |0 0000-0002-1399-5970 |b 53 |
| 700 | 1 | _ | |a Rajpoot, Nasir |0 0000-0001-6760-1271 |b 54 |
| 700 | 1 | _ | |a Reyes, Mauricio |b 55 |
| 700 | 1 | _ | |a Riegler, Michael A |0 0000-0002-3153-2064 |b 56 |
| 700 | 1 | _ | |a Rieke, Nicola |0 0000-0003-0241-9334 |b 57 |
| 700 | 1 | _ | |a Saez-Rodriguez, Julio |b 58 |
| 700 | 1 | _ | |a Sánchez, Clara I |b 59 |
| 700 | 1 | _ | |a Shetty, Shravya |b 60 |
| 700 | 1 | _ | |a Summers, Ronald M |0 0000-0001-8081-7376 |b 61 |
| 700 | 1 | _ | |a Taha, Abdel A |b 62 |
| 700 | 1 | _ | |a Tiulpin, Aleksei |0 0000-0002-7852-4141 |b 63 |
| 700 | 1 | _ | |a Tsaftaris, Sotirios A |b 64 |
| 700 | 1 | _ | |a Van Calster, Ben |b 65 |
| 700 | 1 | _ | |a Varoquaux, Gaël |0 0000-0003-1076-5122 |b 66 |
| 700 | 1 | _ | |a Yaniv, Ziv R |0 0000-0003-0315-7727 |b 67 |
| 700 | 1 | _ | |a Jäger, Paul |0 P:(DE-He78)04a0b5a49db132d8f00cee326cb743ca |b 68 |e Last author |
| 700 | 1 | _ | |a Maier-Hein, Lena |0 P:(DE-He78)26a1176cd8450660333a012075050072 |b 69 |e Last author |
| 773 | _ | _ | |a 10.1038/s41592-023-02150-0 |g Vol. 21, no. 2, p. 182 - 194 |0 PERI:(DE-600)2163081-1 |n 2 |p 182 - 194 |t Nature methods |v 21 |y 2024 |x 1548-7091 |
| 856 | 4 | _ | |u https://inrepo02.dkfz.de/record/288084/files/s41592-023-02150-0.pdf |
| 856 | 4 | _ | |u https://inrepo02.dkfz.de/record/288084/files/s41592-023-02150-0.pdf?subformat=pdfa |x pdfa |
| 909 | C | O | |p VDB |o oai:inrepo02.dkfz.de:288084 |
| 910 | 1 | _ | |a Deutsches Krebsforschungszentrum |0 I:(DE-588b)2036810-0 |k DKFZ |b 0 |6 P:(DE-He78)97e904f47dab556a77c0149cd0002591 |
| 910 | 1 | _ | |a Deutsches Krebsforschungszentrum |0 I:(DE-588b)2036810-0 |k DKFZ |b 1 |6 P:(DE-He78)26651d9aa10255ad4f35610a56aa91e8 |
| 910 | 1 | _ | |a Deutsches Krebsforschungszentrum |0 I:(DE-588b)2036810-0 |k DKFZ |b 2 |6 P:(DE-He78)6ae68adec0915a4ae2d58aff814aaceb |
| 910 | 1 | _ | |a Deutsches Krebsforschungszentrum |0 I:(DE-588b)2036810-0 |k DKFZ |b 3 |6 P:(DE-He78)c9d6245b17f0ab26eeed345cb00d3359 |
| 910 | 1 | _ | |a Deutsches Krebsforschungszentrum |0 I:(DE-588b)2036810-0 |k DKFZ |b 4 |6 P:(DE-He78)98dd16a42af0888dd129e1cd2ee2766a |
| 910 | 1 | _ | |a Deutsches Krebsforschungszentrum |0 I:(DE-588b)2036810-0 |k DKFZ |b 5 |6 P:(DE-He78)64296d0922a2da68074f5de2ccf74487 |
| 910 | 1 | _ | |a Deutsches Krebsforschungszentrum |0 I:(DE-588b)2036810-0 |k DKFZ |b 6 |6 P:(DE-He78)683f71d452c83a7f5ec62969c6012466 |
| 910 | 1 | _ | |a Deutsches Krebsforschungszentrum |0 I:(DE-588b)2036810-0 |k DKFZ |b 13 |6 P:(DE-He78)fdeeec93551e8f3bad68db88a1130c5d |
| 910 | 1 | _ | |a Deutsches Krebsforschungszentrum |0 I:(DE-588b)2036810-0 |k DKFZ |b 17 |6 P:(DE-He78)8da2eca0bc6341c8681c317fe2b8e27b |
| 910 | 1 | _ | |a Deutsches Krebsforschungszentrum |0 I:(DE-588b)2036810-0 |k DKFZ |b 24 |6 P:(DE-He78)77a2a5b07dcbd46277a18a32372ea154 |
| 910 | 1 | _ | |a Deutsches Krebsforschungszentrum |0 I:(DE-588b)2036810-0 |k DKFZ |b 28 |6 P:(DE-He78)7ea9af59d03ec7deb982a0e0562358fa |
| 910 | 1 | _ | |a Deutsches Krebsforschungszentrum |0 I:(DE-588b)2036810-0 |k DKFZ |b 37 |6 P:(DE-He78)bb6a7a70f976eb8df1769944bf913596 |
| 910 | 1 | _ | |a Deutsches Krebsforschungszentrum |0 I:(DE-588b)2036810-0 |k DKFZ |b 44 |6 P:(DE-He78)33c74005e1ce56f7025c4f6be15321b3 |
| 910 | 1 | _ | |a Deutsches Krebsforschungszentrum |0 I:(DE-588b)2036810-0 |k DKFZ |b 68 |6 P:(DE-He78)04a0b5a49db132d8f00cee326cb743ca |
| 910 | 1 | _ | |a Deutsches Krebsforschungszentrum |0 I:(DE-588b)2036810-0 |k DKFZ |b 69 |6 P:(DE-He78)26a1176cd8450660333a012075050072 |
| 913 | 1 | _ | |a DE-HGF |b Gesundheit |l Krebsforschung |1 G:(DE-HGF)POF4-310 |0 G:(DE-HGF)POF4-315 |3 G:(DE-HGF)POF4 |2 G:(DE-HGF)POF4-300 |4 G:(DE-HGF)POF |v Bildgebung und Radioonkologie |x 0 |
| 914 | 1 | _ | |y 2024 |
| 915 | _ | _ | |a Nationallizenz |0 StatID:(DE-HGF)0420 |2 StatID |d 2023-08-29 |w ger |
| 915 | _ | _ | |a DEAL Nature |0 StatID:(DE-HGF)3003 |2 StatID |d 2023-08-29 |w ger |
| 915 | _ | _ | |a DBCoverage |0 StatID:(DE-HGF)0200 |2 StatID |b SCOPUS |d 2023-08-29 |
| 915 | _ | _ | |a DBCoverage |0 StatID:(DE-HGF)0300 |2 StatID |b Medline |d 2023-08-29 |
| 915 | _ | _ | |a DBCoverage |0 StatID:(DE-HGF)0199 |2 StatID |b Clarivate Analytics Master Journal List |d 2023-08-29 |
| 915 | _ | _ | |a DBCoverage |0 StatID:(DE-HGF)1050 |2 StatID |b BIOSIS Previews |d 2023-08-29 |
| 915 | _ | _ | |a WoS |0 StatID:(DE-HGF)0113 |2 StatID |b Science Citation Index Expanded |d 2023-08-29 |
| 915 | _ | _ | |a DBCoverage |0 StatID:(DE-HGF)0150 |2 StatID |b Web of Science Core Collection |d 2023-08-29 |
| 915 | _ | _ | |a DBCoverage |0 StatID:(DE-HGF)1030 |2 StatID |b Current Contents - Life Sciences |d 2023-08-29 |
| 915 | _ | _ | |a DBCoverage |0 StatID:(DE-HGF)1190 |2 StatID |b Biological Abstracts |d 2023-08-29 |
| 915 | _ | _ | |a DBCoverage |0 StatID:(DE-HGF)0160 |2 StatID |b Essential Science Indicators |d 2023-08-29 |
| 915 | _ | _ | |a JCR |0 StatID:(DE-HGF)0100 |2 StatID |b NAT METHODS : 2022 |d 2023-08-29 |
| 915 | _ | _ | |a DBCoverage |0 StatID:(DE-HGF)0600 |2 StatID |b Ebsco Academic Search |d 2023-08-29 |
| 915 | _ | _ | |a Peer Review |0 StatID:(DE-HGF)0030 |2 StatID |b ASC |d 2023-08-29 |
| 915 | _ | _ | |a IF >= 40 |0 StatID:(DE-HGF)9940 |2 StatID |b NAT METHODS : 2022 |d 2023-08-29 |
| 920 | 2 | _ | |0 I:(DE-He78)E130-20160331 |k E130 |l E130 Intelligente Medizinische Systeme |x 0 |
| 920 | 2 | _ | |0 I:(DE-He78)E290-20160331 |k E290 |l NWG Interaktives maschinelles Lernen |x 1 |
| 920 | 1 | _ | |0 I:(DE-He78)E130-20160331 |k E130 |l E130 Intelligente Medizinische Systeme |x 0 |
| 920 | 1 | _ | |0 I:(DE-He78)E230-20160331 |k E230 |l E230 Medizinische Bildverarbeitung |x 1 |
| 920 | 1 | _ | |0 I:(DE-He78)C060-20160331 |k C060 |l C060 Biostatistik |x 2 |
| 920 | 1 | _ | |0 I:(DE-He78)E290-20160331 |k E290 |l NWG Interaktives maschinelles Lernen |x 3 |
| 920 | 1 | _ | |0 I:(DE-He78)FM01-20160331 |k FM01 |l DKTK Koordinierungsstelle Frankfurt |x 4 |
| 920 | 0 | _ | |0 I:(DE-He78)E130-20160331 |k E130 |l E130 Intelligente Medizinische Systeme |x 0 |
| 980 | _ | _ | |a journal |
| 980 | _ | _ | |a VDB |
| 980 | _ | _ | |a I:(DE-He78)E130-20160331 |
| 980 | _ | _ | |a I:(DE-He78)E230-20160331 |
| 980 | _ | _ | |a I:(DE-He78)C060-20160331 |
| 980 | _ | _ | |a I:(DE-He78)E290-20160331 |
| 980 | _ | _ | |a I:(DE-He78)FM01-20160331 |
| 980 | _ | _ | |a UNRESTRICTED |
| Library | Collection | CLSMajor | CLSMinor | Language | Author |
|---|