000274460 001__ 274460
000274460 005__ 20240229154934.0
000274460 0247_ $$2doi$$a10.1016/j.media.2023.102765
000274460 0247_ $$2pmid$$apmid:36965252
000274460 0247_ $$2ISSN$$a1361-8415
000274460 0247_ $$2ISSN$$a1361-8431
000274460 0247_ $$2ISSN$$a1361-8423
000274460 0247_ $$2altmetric$$aaltmetric:144388162
000274460 037__ $$aDKFZ-2023-00605
000274460 041__ $$aEnglish
000274460 082__ $$a610
000274460 1001_ $$0P:(DE-He78)47f4a97043307540977baf09618b5d3d$$aRoss, Tobias$$b0$$eFirst author$$udkfz
000274460 245__ $$aBeyond rankings: Learning (more) from algorithm validation.
000274460 260__ $$aAmsterdam [u.a.]$$bElsevier Science$$c2023
000274460 3367_ $$2DRIVER$$aarticle
000274460 3367_ $$2DataCite$$aOutput Types/Journal article
000274460 3367_ $$0PUB:(DE-HGF)16$$2PUB:(DE-HGF)$$aJournal Article$$bjournal$$mjournal$$s1679912771_19456
000274460 3367_ $$2BibTeX$$aARTICLE
000274460 3367_ $$2ORCID$$aJOURNAL_ARTICLE
000274460 3367_ $$00$$2EndNote$$aJournal Article
000274460 500__ $$a#EA:E130#LA:E130#
000274460 520__ $$aChallenges have become the state-of-the-art approach to benchmark image analysis algorithms in a comparative manner. While the validation on identical data sets was a great step forward, results analysis is often restricted to pure ranking tables, leaving relevant questions unanswered. Specifically, little effort has been put into the systematic investigation on what characterizes images in which state-of-the-art algorithms fail. To address this gap in the literature, we (1) present a statistical framework for learning from challenges and (2) instantiate it for the specific task of instrument instance segmentation in laparoscopic videos. Our framework relies on the semantic meta data annotation of images, which serves as foundation for a General Linear Mixed Models (GLMM) analysis. Based on 51,542 meta data annotations performed on 2,728 images, we applied our approach to the results of the Robust Medical Instrument Segmentation Challenge (ROBUST-MIS) challenge 2019 and revealed underexposure, motion and occlusion of instruments as well as the presence of smoke or other objects in the background as major sources of algorithm failure. Our subsequent method development, tailored to the specific remaining issues, yielded a deep learning model with state-of-the-art overall performance and specific strengths in the processing of images in which previous methods tended to fail. Due to the objectivity and generic applicability of our approach, it could become a valuable tool for validation in the field of medical image analysis and beyond.
000274460 536__ $$0G:(DE-HGF)POF4-315$$a315 - Bildgebung und Radioonkologie (POF4-315)$$cPOF4-315$$fPOF IV$$x0
000274460 588__ $$aDataset connected to CrossRef, PubMed, , Journals: inrepo02.dkfz.de
000274460 650_7 $$2Other$$aArtificial intelligence
000274460 650_7 $$2Other$$aBiomedical image analysis challenges
000274460 650_7 $$2Other$$aDeep learning
000274460 650_7 $$2Other$$aEndoscopic vision
000274460 650_7 $$2Other$$aGeneralized linear mixed models
000274460 650_7 $$2Other$$aGrand challenges
000274460 650_7 $$2Other$$aImage characteristics driven algorithm development
000274460 650_7 $$2Other$$aInstrument segmentation
000274460 650_7 $$2Other$$aMinimally invasive surgery
000274460 650_7 $$2Other$$aSurgical data science
000274460 7001_ $$0P:(DE-He78)861d46b75ffd6c1abb386ca3c5197bac$$aBruno, Pierangela$$b1$$udkfz
000274460 7001_ $$0P:(DE-He78)97e904f47dab556a77c0149cd0002591$$aReinke, Annika$$b2$$udkfz
000274460 7001_ $$0P:(DE-He78)1042737c83ba70ec508bdd99f0096864$$aWiesenfarth, Manuel$$b3$$udkfz
000274460 7001_ $$aKoeppel, Lisa$$b4
000274460 7001_ $$0P:(DE-He78)e9dc924f238fa6cc29465942875fe8f0$$aFull, Peter M$$b5$$udkfz
000274460 7001_ $$0P:(DE-He78)01006b3b56865f6bdad60eb489028403$$aPekdemir, Bünyamin$$b6
000274460 7001_ $$0P:(DE-He78)77a2a5b07dcbd46277a18a32372ea154$$aGodau, Patrick$$b7$$udkfz
000274460 7001_ $$0P:(DE-He78)3659551a3b56f897a34887b9f52fc2b8$$aTrofimova, Darya$$b8$$udkfz
000274460 7001_ $$0P:(DE-He78)7ea9af59d03ec7deb982a0e0562358fa$$aIsensee, Fabian$$b9$$udkfz
000274460 7001_ $$0P:(DE-He78)ae131915396ed2f27752c043e123897e$$aAdler, Tim J$$b10$$udkfz
000274460 7001_ $$0P:(DE-He78)96509db5798da9bcccf0d34db39f50e7$$aTran, Thuy N$$b11$$udkfz
000274460 7001_ $$aMoccia, Sara$$b12
000274460 7001_ $$aCalimeri, Francesco$$b13
000274460 7001_ $$aMüller-Stich, Beat P$$b14
000274460 7001_ $$0P:(DE-He78)bb6a7a70f976eb8df1769944bf913596$$aKopp-Schneider, Annette$$b15$$udkfz
000274460 7001_ $$0P:(DE-He78)26a1176cd8450660333a012075050072$$aMaier-Hein, Lena$$b16$$eLast author$$udkfz
000274460 773__ $$0PERI:(DE-600)1497450-2$$a10.1016/j.media.2023.102765$$gVol. 86, p. 102765 -$$p102765$$tMedical image analysis$$v86$$x1361-8415$$y2023
000274460 909CO $$ooai:inrepo02.dkfz.de:274460$$pVDB
000274460 9101_ $$0I:(DE-588b)2036810-0$$6P:(DE-He78)47f4a97043307540977baf09618b5d3d$$aDeutsches Krebsforschungszentrum$$b0$$kDKFZ
000274460 9101_ $$0I:(DE-588b)2036810-0$$6P:(DE-He78)861d46b75ffd6c1abb386ca3c5197bac$$aDeutsches Krebsforschungszentrum$$b1$$kDKFZ
000274460 9101_ $$0I:(DE-588b)2036810-0$$6P:(DE-He78)97e904f47dab556a77c0149cd0002591$$aDeutsches Krebsforschungszentrum$$b2$$kDKFZ
000274460 9101_ $$0I:(DE-588b)2036810-0$$6P:(DE-He78)1042737c83ba70ec508bdd99f0096864$$aDeutsches Krebsforschungszentrum$$b3$$kDKFZ
000274460 9101_ $$0I:(DE-588b)2036810-0$$6P:(DE-He78)e9dc924f238fa6cc29465942875fe8f0$$aDeutsches Krebsforschungszentrum$$b5$$kDKFZ
000274460 9101_ $$0I:(DE-588b)2036810-0$$6P:(DE-He78)01006b3b56865f6bdad60eb489028403$$aDeutsches Krebsforschungszentrum$$b6$$kDKFZ
000274460 9101_ $$0I:(DE-588b)2036810-0$$6P:(DE-He78)77a2a5b07dcbd46277a18a32372ea154$$aDeutsches Krebsforschungszentrum$$b7$$kDKFZ
000274460 9101_ $$0I:(DE-588b)2036810-0$$6P:(DE-He78)3659551a3b56f897a34887b9f52fc2b8$$aDeutsches Krebsforschungszentrum$$b8$$kDKFZ
000274460 9101_ $$0I:(DE-588b)2036810-0$$6P:(DE-He78)7ea9af59d03ec7deb982a0e0562358fa$$aDeutsches Krebsforschungszentrum$$b9$$kDKFZ
000274460 9101_ $$0I:(DE-588b)2036810-0$$6P:(DE-He78)ae131915396ed2f27752c043e123897e$$aDeutsches Krebsforschungszentrum$$b10$$kDKFZ
000274460 9101_ $$0I:(DE-588b)2036810-0$$6P:(DE-He78)96509db5798da9bcccf0d34db39f50e7$$aDeutsches Krebsforschungszentrum$$b11$$kDKFZ
000274460 9101_ $$0I:(DE-588b)2036810-0$$6P:(DE-He78)bb6a7a70f976eb8df1769944bf913596$$aDeutsches Krebsforschungszentrum$$b15$$kDKFZ
000274460 9101_ $$0I:(DE-588b)2036810-0$$6P:(DE-He78)26a1176cd8450660333a012075050072$$aDeutsches Krebsforschungszentrum$$b16$$kDKFZ
000274460 9131_ $$0G:(DE-HGF)POF4-315$$1G:(DE-HGF)POF4-310$$2G:(DE-HGF)POF4-300$$3G:(DE-HGF)POF4$$4G:(DE-HGF)POF$$aDE-HGF$$bGesundheit$$lKrebsforschung$$vBildgebung und Radioonkologie$$x0
000274460 9141_ $$y2023
000274460 915__ $$0StatID:(DE-HGF)0113$$2StatID$$aWoS$$bScience Citation Index Expanded$$d2022-11-18
000274460 915__ $$0StatID:(DE-HGF)0160$$2StatID$$aDBCoverage$$bEssential Science Indicators$$d2022-11-18
000274460 915__ $$0StatID:(DE-HGF)0100$$2StatID$$aJCR$$bMED IMAGE ANAL : 2022$$d2023-10-21
000274460 915__ $$0StatID:(DE-HGF)0200$$2StatID$$aDBCoverage$$bSCOPUS$$d2023-10-21
000274460 915__ $$0StatID:(DE-HGF)0300$$2StatID$$aDBCoverage$$bMedline$$d2023-10-21
000274460 915__ $$0StatID:(DE-HGF)0600$$2StatID$$aDBCoverage$$bEbsco Academic Search$$d2023-10-21
000274460 915__ $$0StatID:(DE-HGF)0030$$2StatID$$aPeer Review$$bASC$$d2023-10-21
000274460 915__ $$0StatID:(DE-HGF)0199$$2StatID$$aDBCoverage$$bClarivate Analytics Master Journal List$$d2023-10-21
000274460 915__ $$0StatID:(DE-HGF)0150$$2StatID$$aDBCoverage$$bWeb of Science Core Collection$$d2023-10-21
000274460 915__ $$0StatID:(DE-HGF)1160$$2StatID$$aDBCoverage$$bCurrent Contents - Engineering, Computing and Technology$$d2023-10-21
000274460 915__ $$0StatID:(DE-HGF)9910$$2StatID$$aIF >= 10$$bMED IMAGE ANAL : 2022$$d2023-10-21
000274460 9202_ $$0I:(DE-He78)E130-20160331$$kE130$$lE130 Intelligente Medizinische Systeme$$x0
000274460 9201_ $$0I:(DE-He78)E130-20160331$$kE130$$lE130 Intelligente Medizinische Systeme$$x0
000274460 9201_ $$0I:(DE-He78)C060-20160331$$kC060$$lC060 Biostatistik$$x1
000274460 9201_ $$0I:(DE-He78)E230-20160331$$kE230$$lE230 Medizinische Bildverarbeitung$$x2
000274460 9200_ $$0I:(DE-He78)E130-20160331$$kE130$$lE130 Intelligente Medizinische Systeme$$x0
000274460 980__ $$ajournal
000274460 980__ $$aVDB
000274460 980__ $$aI:(DE-He78)E130-20160331
000274460 980__ $$aI:(DE-He78)C060-20160331
000274460 980__ $$aI:(DE-He78)E230-20160331
000274460 980__ $$aUNRESTRICTED