000294883 001__ 294883
000294883 005__ 20241210182915.0
000294883 0247_ $$2doi$$a10.48550/ARXIV.2401.08501
000294883 037__ $$aDKFZ-2024-02593
000294883 1001_ $$0P:(DE-He78)68389eb0c9488ae62f7f47b128ef3b48$$aKahl, Kim-Celine$$b0$$eFirst author$$udkfz
000294883 245__ $$aValUES: A Framework for Systematic Validation of Uncertainty Estimation in Semantic Segmentation
000294883 260__ $$barXiv$$c2024
000294883 3367_ $$0PUB:(DE-HGF)25$$2PUB:(DE-HGF)$$aPreprint$$bpreprint$$mpreprint$$s1733821631_31422
000294883 3367_ $$2ORCID$$aWORKING_PAPER
000294883 3367_ $$028$$2EndNote$$aElectronic Article
000294883 3367_ $$2DRIVER$$apreprint
000294883 3367_ $$2BibTeX$$aARTICLE
000294883 3367_ $$2DataCite$$aOutput Types/Working Paper
000294883 500__ $$aPublished as a conference paper at ICLR 2024
000294883 520__ $$aUncertainty estimation is an essential and heavily-studied component for the reliable application of semantic segmentation methods. While various studies exist claiming methodological advances on the one hand, and successful application on the other hand, the field is currently hampered by a gap between theory and practice leaving fundamental questions unanswered: Can data-related and model-related uncertainty really be separated in practice? Which components of an uncertainty method are essential for real-world performance? Which uncertainty method works well for which application? In this work, we link this research gap to a lack of systematic and comprehensive evaluation of uncertainty methods. Specifically, we identify three key pitfalls in current literature and present an evaluation framework that bridges the research gap by providing 1) a controlled environment for studying data ambiguities as well as distribution shifts, 2) systematic ablations of relevant method components, and 3) test-beds for the five predominant uncertainty applications: OoD-detection, active learning, failure detection, calibration, and ambiguity modeling. Empirical results on simulated as well as real-world data demonstrate how the proposed framework is able to answer the predominant questions in the field revealing for instance that 1) separation of uncertainty types works on simulated data but does not necessarily translate to real-world data, 2) aggregation of scores is a crucial but currently neglected component of uncertainty methods, 3) While ensembles are performing most robustly across the different downstream tasks and settings, test-time augmentation often constitutes a light-weight alternative. Code is at: https://github.com/IML-DKFZ/values
000294883 536__ $$0G:(DE-HGF)POF4-315$$a315 - Bildgebung und Radioonkologie (POF4-315)$$cPOF4-315$$fPOF IV$$x0
000294883 588__ $$aDataset connected to DataCite
000294883 650_7 $$2Other$$aComputer Vision and Pattern Recognition (cs.CV)
000294883 650_7 $$2Other$$aFOS: Computer and information sciences
000294883 7001_ $$0P:(DE-He78)6a78e3a44a8038881d941fb467eb4e19$$aLüth, Carsten$$b1$$eFirst author$$udkfz
000294883 7001_ $$0P:(DE-He78)eafef5cb69dd3d85f1cc942c474a220f$$aZenk, Maximilian$$b2$$udkfz
000294883 7001_ $$0P:(DE-He78)33c74005e1ce56f7025c4f6be15321b3$$aMaier-Hein, Klaus$$b3$$udkfz
000294883 7001_ $$0P:(DE-HGF)0$$aJaeger, Paul F.$$b4$$eLast author
000294883 773__ $$a10.48550/ARXIV.2401.08501
000294883 909CO $$ooai:inrepo02.dkfz.de:294883$$pVDB
000294883 9101_ $$0I:(DE-588b)2036810-0$$6P:(DE-He78)68389eb0c9488ae62f7f47b128ef3b48$$aDeutsches Krebsforschungszentrum$$b0$$kDKFZ
000294883 9101_ $$0I:(DE-588b)2036810-0$$6P:(DE-He78)6a78e3a44a8038881d941fb467eb4e19$$aDeutsches Krebsforschungszentrum$$b1$$kDKFZ
000294883 9101_ $$0I:(DE-588b)2036810-0$$6P:(DE-He78)eafef5cb69dd3d85f1cc942c474a220f$$aDeutsches Krebsforschungszentrum$$b2$$kDKFZ
000294883 9101_ $$0I:(DE-588b)2036810-0$$6P:(DE-He78)33c74005e1ce56f7025c4f6be15321b3$$aDeutsches Krebsforschungszentrum$$b3$$kDKFZ
000294883 9101_ $$0I:(DE-588b)2036810-0$$6P:(DE-HGF)0$$aDeutsches Krebsforschungszentrum$$b4$$kDKFZ
000294883 9131_ $$0G:(DE-HGF)POF4-315$$1G:(DE-HGF)POF4-310$$2G:(DE-HGF)POF4-300$$3G:(DE-HGF)POF4$$4G:(DE-HGF)POF$$aDE-HGF$$bGesundheit$$lKrebsforschung$$vBildgebung und Radioonkologie$$x0
000294883 9141_ $$y2024
000294883 9201_ $$0I:(DE-He78)E290-20160331$$kE290$$lNWG Interaktives maschinelles Lernen$$x0
000294883 9201_ $$0I:(DE-He78)E230-20160331$$kE230$$lE230 Medizinische Bildverarbeitung$$x1
000294883 980__ $$apreprint
000294883 980__ $$aVDB
000294883 980__ $$aI:(DE-He78)E290-20160331
000294883 980__ $$aI:(DE-He78)E230-20160331
000294883 980__ $$aUNRESTRICTED