guest ::
| Home > Publications database > Methods and open-source toolkit for analyzing and visualizing challenge results. > print |
| Home > Publications database > Methods and open-source toolkit for analyzing and visualizing challenge results. > print |
| 001 | 167255 | ||
| 005 | 20240229133534.0 | ||
| 024 | 7 | _ | |a 10.1038/s41598-021-82017-6 |2 doi |
| 024 | 7 | _ | |a pmid:33504883 |2 pmid |
| 024 | 7 | _ | |a altmetric:98848347 |2 altmetric |
| 037 | _ | _ | |a DKFZ-2021-00228 |
| 041 | _ | _ | |a eng |
| 082 | _ | _ | |a 600 |
| 100 | 1 | _ | |a Wiesenfarth, Manuel |0 P:(DE-He78)1042737c83ba70ec508bdd99f0096864 |b 0 |e First author |u dkfz |
| 245 | _ | _ | |a Methods and open-source toolkit for analyzing and visualizing challenge results. |
| 260 | _ | _ | |a [London] |c 2021 |b Macmillan Publishers Limited, part of Springer Nature |
| 336 | 7 | _ | |a article |2 DRIVER |
| 336 | 7 | _ | |a Output Types/Journal article |2 DataCite |
| 336 | 7 | _ | |a Journal Article |b journal |m journal |0 PUB:(DE-HGF)16 |s 1611931330_28564 |2 PUB:(DE-HGF) |
| 336 | 7 | _ | |a ARTICLE |2 BibTeX |
| 336 | 7 | _ | |a JOURNAL_ARTICLE |2 ORCID |
| 336 | 7 | _ | |a Journal Article |0 0 |2 EndNote |
| 500 | _ | _ | |a #EA:C060#LA:C060# |
| 520 | _ | _ | |a Grand challenges have become the de facto standard for benchmarking image analysis algorithms. While the number of these international competitions is steadily increasing, surprisingly little effort has been invested in ensuring high quality design, execution and reporting for these international competitions. Specifically, results analysis and visualization in the event of uncertainties have been given almost no attention in the literature. Given these shortcomings, the contribution of this paper is two-fold: (1) we present a set of methods to comprehensively analyze and visualize the results of single-task and multi-task challenges and apply them to a number of simulated and real-life challenges to demonstrate their specific strengths and weaknesses; (2) we release the open-source framework challengeR as part of this work to enable fast and wide adoption of the methodology proposed in this paper. Our approach offers an intuitive way to gain important insights into the relative and absolute performance of algorithms, which cannot be revealed by commonly applied visualization techniques. This is demonstrated by the experiments performed in the specific context of biomedical image analysis challenges. Our framework could thus become an important tool for analyzing and visualizing challenge results in the field of biomedical image analysis and beyond. |
| 536 | _ | _ | |a 313 - Krebsrisikofaktoren und Prävention (POF4-313) |0 G:(DE-HGF)POF4-313 |c POF4-313 |x 0 |f POF IV |
| 588 | _ | _ | |a Dataset connected to CrossRef, PubMed, |
| 700 | 1 | _ | |a Reinke, Annika |0 P:(DE-He78)97e904f47dab556a77c0149cd0002591 |b 1 |u dkfz |
| 700 | 1 | _ | |a Landman, Bennett A |b 2 |
| 700 | 1 | _ | |a Eisenmann, Matthias |0 P:(DE-He78)c9d6245b17f0ab26eeed345cb00d3359 |b 3 |u dkfz |
| 700 | 1 | _ | |a Saiz, Laura Aguilera |0 P:(DE-HGF)0 |b 4 |
| 700 | 1 | _ | |a Cardoso, M Jorge |b 5 |
| 700 | 1 | _ | |a Maier-Hein, Lena |0 P:(DE-He78)26a1176cd8450660333a012075050072 |b 6 |u dkfz |
| 700 | 1 | _ | |a Kopp-Schneider, Annette |0 P:(DE-He78)bb6a7a70f976eb8df1769944bf913596 |b 7 |e Last author |u dkfz |
| 773 | _ | _ | |a 10.1038/s41598-021-82017-6 |g Vol. 11, no. 1, p. 2369 |0 PERI:(DE-600)2615211-3 |n 1 |p 2369 |t Scientific reports |v 11 |y 2021 |x 2045-2322 |
| 909 | C | O | |o oai:inrepo02.dkfz.de:167255 |p VDB |
| 910 | 1 | _ | |a Deutsches Krebsforschungszentrum |0 I:(DE-588b)2036810-0 |k DKFZ |b 0 |6 P:(DE-He78)1042737c83ba70ec508bdd99f0096864 |
| 910 | 1 | _ | |a Deutsches Krebsforschungszentrum |0 I:(DE-588b)2036810-0 |k DKFZ |b 1 |6 P:(DE-He78)97e904f47dab556a77c0149cd0002591 |
| 910 | 1 | _ | |a Deutsches Krebsforschungszentrum |0 I:(DE-588b)2036810-0 |k DKFZ |b 3 |6 P:(DE-He78)c9d6245b17f0ab26eeed345cb00d3359 |
| 910 | 1 | _ | |a Deutsches Krebsforschungszentrum |0 I:(DE-588b)2036810-0 |k DKFZ |b 4 |6 P:(DE-HGF)0 |
| 910 | 1 | _ | |a Deutsches Krebsforschungszentrum |0 I:(DE-588b)2036810-0 |k DKFZ |b 6 |6 P:(DE-He78)26a1176cd8450660333a012075050072 |
| 910 | 1 | _ | |a Deutsches Krebsforschungszentrum |0 I:(DE-588b)2036810-0 |k DKFZ |b 7 |6 P:(DE-He78)bb6a7a70f976eb8df1769944bf913596 |
| 913 | 0 | _ | |a DE-HGF |b Gesundheit |l Krebsforschung |1 G:(DE-HGF)POF3-310 |0 G:(DE-HGF)POF3-313 |3 G:(DE-HGF)POF3 |2 G:(DE-HGF)POF3-300 |4 G:(DE-HGF)POF |v Cancer risk factors and prevention |x 0 |
| 913 | 1 | _ | |a DE-HGF |b Gesundheit |l Krebsforschung |1 G:(DE-HGF)POF4-310 |0 G:(DE-HGF)POF4-313 |3 G:(DE-HGF)POF4 |2 G:(DE-HGF)POF4-300 |4 G:(DE-HGF)POF |v Krebsrisikofaktoren und Prävention |x 0 |
| 914 | 1 | _ | |y 2021 |
| 915 | _ | _ | |a JCR |0 StatID:(DE-HGF)0100 |2 StatID |b SCI REP-UK : 2018 |d 2020-08-29 |
| 915 | _ | _ | |a DBCoverage |0 StatID:(DE-HGF)0200 |2 StatID |b SCOPUS |d 2020-08-29 |
| 915 | _ | _ | |a DBCoverage |0 StatID:(DE-HGF)0300 |2 StatID |b Medline |d 2020-08-29 |
| 915 | _ | _ | |a DBCoverage |0 StatID:(DE-HGF)0320 |2 StatID |b PubMed Central |d 2020-08-29 |
| 915 | _ | _ | |a DBCoverage |0 StatID:(DE-HGF)0501 |2 StatID |b DOAJ Seal |d 2020-08-29 |
| 915 | _ | _ | |a DBCoverage |0 StatID:(DE-HGF)0500 |2 StatID |b DOAJ |d 2020-08-29 |
| 915 | _ | _ | |a Peer Review |0 StatID:(DE-HGF)0030 |2 StatID |b DOAJ : Blind peer review |d 2020-08-29 |
| 915 | _ | _ | |a Creative Commons Attribution CC BY (No Version) |0 LIC:(DE-HGF)CCBYNV |2 V:(DE-HGF) |b DOAJ |d 2020-08-29 |
| 915 | _ | _ | |a DBCoverage |0 StatID:(DE-HGF)0600 |2 StatID |b Ebsco Academic Search |d 2020-08-29 |
| 915 | _ | _ | |a Peer Review |0 StatID:(DE-HGF)0030 |2 StatID |b ASC |d 2020-08-29 |
| 915 | _ | _ | |a DBCoverage |0 StatID:(DE-HGF)0199 |2 StatID |b Clarivate Analytics Master Journal List |d 2020-08-29 |
| 915 | _ | _ | |a DBCoverage |0 StatID:(DE-HGF)0160 |2 StatID |b Essential Science Indicators |d 2020-08-29 |
| 915 | _ | _ | |a DBCoverage |0 StatID:(DE-HGF)1050 |2 StatID |b BIOSIS Previews |d 2020-08-29 |
| 915 | _ | _ | |a DBCoverage |0 StatID:(DE-HGF)1150 |2 StatID |b Current Contents - Physical, Chemical and Earth Sciences |d 2020-08-29 |
| 915 | _ | _ | |a DBCoverage |0 StatID:(DE-HGF)1040 |2 StatID |b Zoological Record |d 2020-08-29 |
| 915 | _ | _ | |a DBCoverage |0 StatID:(DE-HGF)1190 |2 StatID |b Biological Abstracts |d 2020-08-29 |
| 915 | _ | _ | |a WoS |0 StatID:(DE-HGF)0113 |2 StatID |b Science Citation Index Expanded |d 2020-08-29 |
| 915 | _ | _ | |a DBCoverage |0 StatID:(DE-HGF)0150 |2 StatID |b Web of Science Core Collection |d 2020-08-29 |
| 915 | _ | _ | |a IF < 5 |0 StatID:(DE-HGF)9900 |2 StatID |d 2020-08-29 |
| 915 | _ | _ | |a Article Processing Charges |0 StatID:(DE-HGF)0561 |2 StatID |d 2020-08-29 |
| 915 | _ | _ | |a Fees |0 StatID:(DE-HGF)0700 |2 StatID |d 2020-08-29 |
| 920 | 1 | _ | |0 I:(DE-He78)C060-20160331 |k C060 |l C060 Biostatistik |x 0 |
| 920 | 1 | _ | |0 I:(DE-He78)E130-20160331 |k E130 |l E130 Computer-assistierte med. Interventionen |x 1 |
| 980 | _ | _ | |a journal |
| 980 | _ | _ | |a VDB |
| 980 | _ | _ | |a I:(DE-He78)C060-20160331 |
| 980 | _ | _ | |a I:(DE-He78)E130-20160331 |
| 980 | _ | _ | |a UNRESTRICTED |
| Library | Collection | CLSMajor | CLSMinor | Language | Author |
|---|