000167255 001__ 167255
000167255 005__ 20240229133534.0
000167255 0247_ $$2doi$$a10.1038/s41598-021-82017-6
000167255 0247_ $$2pmid$$apmid:33504883
000167255 0247_ $$2altmetric$$aaltmetric:98848347
000167255 037__ $$aDKFZ-2021-00228
000167255 041__ $$aeng
000167255 082__ $$a600
000167255 1001_ $$0P:(DE-He78)1042737c83ba70ec508bdd99f0096864$$aWiesenfarth, Manuel$$b0$$eFirst author$$udkfz
000167255 245__ $$aMethods and open-source toolkit for analyzing and visualizing challenge results.
000167255 260__ $$a[London]$$bMacmillan Publishers Limited, part of Springer Nature$$c2021
000167255 3367_ $$2DRIVER$$aarticle
000167255 3367_ $$2DataCite$$aOutput Types/Journal article
000167255 3367_ $$0PUB:(DE-HGF)16$$2PUB:(DE-HGF)$$aJournal Article$$bjournal$$mjournal$$s1611931330_28564
000167255 3367_ $$2BibTeX$$aARTICLE
000167255 3367_ $$2ORCID$$aJOURNAL_ARTICLE
000167255 3367_ $$00$$2EndNote$$aJournal Article
000167255 500__ $$a#EA:C060#LA:C060#
000167255 520__ $$aGrand challenges have become the de facto standard for benchmarking image analysis algorithms. While the number of these international competitions is steadily increasing, surprisingly little effort has been invested in ensuring high quality design, execution and reporting for these international competitions. Specifically, results analysis and visualization in the event of uncertainties have been given almost no attention in the literature. Given these shortcomings, the contribution of this paper is two-fold: (1) we present a set of methods to comprehensively analyze and visualize the results of single-task and multi-task challenges and apply them to a number of simulated and real-life challenges to demonstrate their specific strengths and weaknesses; (2) we release the open-source framework challengeR as part of this work to enable fast and wide adoption of the methodology proposed in this paper. Our approach offers an intuitive way to gain important insights into the relative and absolute performance of algorithms, which cannot be revealed by commonly applied visualization techniques. This is demonstrated by the experiments performed in the specific context of biomedical image analysis challenges. Our framework could thus become an important tool for analyzing and visualizing challenge results in the field of biomedical image analysis and beyond.
000167255 536__ $$0G:(DE-HGF)POF4-313$$a313 - Krebsrisikofaktoren und Prävention (POF4-313)$$cPOF4-313$$fPOF IV$$x0
000167255 588__ $$aDataset connected to CrossRef, PubMed,
000167255 7001_ $$0P:(DE-He78)97e904f47dab556a77c0149cd0002591$$aReinke, Annika$$b1$$udkfz
000167255 7001_ $$aLandman, Bennett A$$b2
000167255 7001_ $$0P:(DE-He78)c9d6245b17f0ab26eeed345cb00d3359$$aEisenmann, Matthias$$b3$$udkfz
000167255 7001_ $$0P:(DE-HGF)0$$aSaiz, Laura Aguilera$$b4
000167255 7001_ $$aCardoso, M Jorge$$b5
000167255 7001_ $$0P:(DE-He78)26a1176cd8450660333a012075050072$$aMaier-Hein, Lena$$b6$$udkfz
000167255 7001_ $$0P:(DE-He78)bb6a7a70f976eb8df1769944bf913596$$aKopp-Schneider, Annette$$b7$$eLast author$$udkfz
000167255 773__ $$0PERI:(DE-600)2615211-3$$a10.1038/s41598-021-82017-6$$gVol. 11, no. 1, p. 2369$$n1$$p2369$$tScientific reports$$v11$$x2045-2322$$y2021
000167255 909CO $$ooai:inrepo02.dkfz.de:167255$$pVDB
000167255 9101_ $$0I:(DE-588b)2036810-0$$6P:(DE-He78)1042737c83ba70ec508bdd99f0096864$$aDeutsches Krebsforschungszentrum$$b0$$kDKFZ
000167255 9101_ $$0I:(DE-588b)2036810-0$$6P:(DE-He78)97e904f47dab556a77c0149cd0002591$$aDeutsches Krebsforschungszentrum$$b1$$kDKFZ
000167255 9101_ $$0I:(DE-588b)2036810-0$$6P:(DE-He78)c9d6245b17f0ab26eeed345cb00d3359$$aDeutsches Krebsforschungszentrum$$b3$$kDKFZ
000167255 9101_ $$0I:(DE-588b)2036810-0$$6P:(DE-HGF)0$$aDeutsches Krebsforschungszentrum$$b4$$kDKFZ
000167255 9101_ $$0I:(DE-588b)2036810-0$$6P:(DE-He78)26a1176cd8450660333a012075050072$$aDeutsches Krebsforschungszentrum$$b6$$kDKFZ
000167255 9101_ $$0I:(DE-588b)2036810-0$$6P:(DE-He78)bb6a7a70f976eb8df1769944bf913596$$aDeutsches Krebsforschungszentrum$$b7$$kDKFZ
000167255 9130_ $$0G:(DE-HGF)POF3-313$$1G:(DE-HGF)POF3-310$$2G:(DE-HGF)POF3-300$$3G:(DE-HGF)POF3$$4G:(DE-HGF)POF$$aDE-HGF$$bGesundheit$$lKrebsforschung$$vCancer risk factors and prevention$$x0
000167255 9131_ $$0G:(DE-HGF)POF4-313$$1G:(DE-HGF)POF4-310$$2G:(DE-HGF)POF4-300$$3G:(DE-HGF)POF4$$4G:(DE-HGF)POF$$aDE-HGF$$bGesundheit$$lKrebsforschung$$vKrebsrisikofaktoren und Prävention$$x0
000167255 9141_ $$y2021
000167255 915__ $$0StatID:(DE-HGF)0100$$2StatID$$aJCR$$bSCI REP-UK : 2018$$d2020-08-29
000167255 915__ $$0StatID:(DE-HGF)0200$$2StatID$$aDBCoverage$$bSCOPUS$$d2020-08-29
000167255 915__ $$0StatID:(DE-HGF)0300$$2StatID$$aDBCoverage$$bMedline$$d2020-08-29
000167255 915__ $$0StatID:(DE-HGF)0320$$2StatID$$aDBCoverage$$bPubMed Central$$d2020-08-29
000167255 915__ $$0StatID:(DE-HGF)0501$$2StatID$$aDBCoverage$$bDOAJ Seal$$d2020-08-29
000167255 915__ $$0StatID:(DE-HGF)0500$$2StatID$$aDBCoverage$$bDOAJ$$d2020-08-29
000167255 915__ $$0StatID:(DE-HGF)0030$$2StatID$$aPeer Review$$bDOAJ : Blind peer review$$d2020-08-29
000167255 915__ $$0LIC:(DE-HGF)CCBYNV$$2V:(DE-HGF)$$aCreative Commons Attribution CC BY (No Version)$$bDOAJ$$d2020-08-29
000167255 915__ $$0StatID:(DE-HGF)0600$$2StatID$$aDBCoverage$$bEbsco Academic Search$$d2020-08-29
000167255 915__ $$0StatID:(DE-HGF)0030$$2StatID$$aPeer Review$$bASC$$d2020-08-29
000167255 915__ $$0StatID:(DE-HGF)0199$$2StatID$$aDBCoverage$$bClarivate Analytics Master Journal List$$d2020-08-29
000167255 915__ $$0StatID:(DE-HGF)0160$$2StatID$$aDBCoverage$$bEssential Science Indicators$$d2020-08-29
000167255 915__ $$0StatID:(DE-HGF)1050$$2StatID$$aDBCoverage$$bBIOSIS Previews$$d2020-08-29
000167255 915__ $$0StatID:(DE-HGF)1150$$2StatID$$aDBCoverage$$bCurrent Contents - Physical, Chemical and Earth Sciences$$d2020-08-29
000167255 915__ $$0StatID:(DE-HGF)1040$$2StatID$$aDBCoverage$$bZoological Record$$d2020-08-29
000167255 915__ $$0StatID:(DE-HGF)1190$$2StatID$$aDBCoverage$$bBiological Abstracts$$d2020-08-29
000167255 915__ $$0StatID:(DE-HGF)0113$$2StatID$$aWoS$$bScience Citation Index Expanded$$d2020-08-29
000167255 915__ $$0StatID:(DE-HGF)0150$$2StatID$$aDBCoverage$$bWeb of Science Core Collection$$d2020-08-29
000167255 915__ $$0StatID:(DE-HGF)9900$$2StatID$$aIF < 5$$d2020-08-29
000167255 915__ $$0StatID:(DE-HGF)0561$$2StatID$$aArticle Processing Charges$$d2020-08-29
000167255 915__ $$0StatID:(DE-HGF)0700$$2StatID$$aFees$$d2020-08-29
000167255 9201_ $$0I:(DE-He78)C060-20160331$$kC060$$lC060 Biostatistik$$x0
000167255 9201_ $$0I:(DE-He78)E130-20160331$$kE130$$lE130 Computer-assistierte med. Interventionen$$x1
000167255 980__ $$ajournal
000167255 980__ $$aVDB
000167255 980__ $$aI:(DE-He78)C060-20160331
000167255 980__ $$aI:(DE-He78)E130-20160331
000167255 980__ $$aUNRESTRICTED