000141663 001__ 141663
000141663 005__ 20240229105127.0
000141663 0247_ $$2doi$$a10.1109/TPAMI.2017.2777967
000141663 0247_ $$2pmid$$apmid:29989983
000141663 0247_ $$2altmetric$$aaltmetric:14117635
000141663 037__ $$aDKFZ-2018-01934
000141663 041__ $$aeng
000141663 082__ $$a004
000141663 1001_ $$0P:(DE-He78)c79e48a0edbf2eee227450c6984615fa$$aHeim, Eric$$b0$$eFirst author$$udkfz
000141663 245__ $$aClickstream Analysis for Crowd-Based Object Segmentation with Confidence.
000141663 260__ $$aNew York, NY$$bIEEE$$c2018
000141663 3367_ $$2DRIVER$$aarticle
000141663 3367_ $$2DataCite$$aOutput Types/Journal article
000141663 3367_ $$0PUB:(DE-HGF)16$$2PUB:(DE-HGF)$$aJournal Article$$bjournal$$mjournal$$s1661346660_30652
000141663 3367_ $$2BibTeX$$aARTICLE
000141663 3367_ $$2ORCID$$aJOURNAL_ARTICLE
000141663 3367_ $$00$$2EndNote$$aJournal Article
000141663 500__ $$aIEEE Transactions on Pattern Analysis and Machine Intelligence (IEEE Trans. Pattern Anal. Mach. Intell.) = 2160-92921939-3539 (import from CrossRef, PubMed, )
000141663 520__ $$aWith the rapidly increasing interest in machine learning based solutions for automatic image annotation, the availability of reference annotations for algorithm training is one of the major bottlenecks in the field. Crowdsourcing has evolved as a valuable option for low-cost and large-scale data annotation; however, quality control remains a major issue which needs to be addressed. To our knowledge, we are the first to analyze the annotation process to improve crowd-sourced image segmentation. Our method involves training a regressor to estimate the quality of a segmentation from the annotator's clickstream data. The quality estimation can be used to identify spam and weight individual annotations by their (estimated) quality when merging multiple segmentations of one image. Using a total of 29,000 crowd annotations performed on publicly available data of different object classes, we show that (1) our method is highly accurate in estimating the segmentation quality based on clickstream data, (2) outperforms state-of-the-art methods for merging multiple annotations. As the regressor does not need to be trained on the object class that it is applied to it can be regarded as a low-cost option for quality control and confidence analysis in the context of crowd-based image annotation.
000141663 536__ $$0G:(DE-HGF)POF3-315$$a315 - Imaging and radiooncology (POF3-315)$$cPOF3-315$$fPOF III$$x0
000141663 588__ $$aDataset connected to CrossRef, PubMed,
000141663 7001_ $$0P:(DE-He78)a83df473f58a6a8ef43263ec9783ecf0$$aSeitel, Alexander$$b1$$udkfz
000141663 7001_ $$aAndrulis, Jonas$$b2
000141663 7001_ $$0P:(DE-He78)7ea9af59d03ec7deb982a0e0562358fa$$aIsensee, Fabian$$b3$$udkfz
000141663 7001_ $$0P:(DE-He78)908880209a64ea539ae8dc5fdb7e0a91$$aStock, Christian$$b4$$udkfz
000141663 7001_ $$0P:(DE-He78)47f4a97043307540977baf09618b5d3d$$aRoss, Tobias$$b5$$udkfz
000141663 7001_ $$0P:(DE-He78)26a1176cd8450660333a012075050072$$aMaier-Hein, Lena$$b6$$eLast author$$udkfz
000141663 773__ $$0PERI:(DE-600)2027336-8$$a10.1109/TPAMI.2017.2777967$$gVol. 40, no. 12, p. 2814 - 2826$$n12$$p2814 - 2826$$tIEEE transactions on pattern analysis and machine intelligence$$v40$$x0162-8828$$y2018
000141663 909CO $$ooai:inrepo02.dkfz.de:141663$$pVDB
000141663 9101_ $$0I:(DE-588b)2036810-0$$6P:(DE-He78)c79e48a0edbf2eee227450c6984615fa$$aDeutsches Krebsforschungszentrum$$b0$$kDKFZ
000141663 9101_ $$0I:(DE-588b)2036810-0$$6P:(DE-He78)a83df473f58a6a8ef43263ec9783ecf0$$aDeutsches Krebsforschungszentrum$$b1$$kDKFZ
000141663 9101_ $$0I:(DE-588b)2036810-0$$6P:(DE-He78)7ea9af59d03ec7deb982a0e0562358fa$$aDeutsches Krebsforschungszentrum$$b3$$kDKFZ
000141663 9101_ $$0I:(DE-588b)2036810-0$$6P:(DE-He78)908880209a64ea539ae8dc5fdb7e0a91$$aDeutsches Krebsforschungszentrum$$b4$$kDKFZ
000141663 9101_ $$0I:(DE-588b)2036810-0$$6P:(DE-He78)47f4a97043307540977baf09618b5d3d$$aDeutsches Krebsforschungszentrum$$b5$$kDKFZ
000141663 9101_ $$0I:(DE-588b)2036810-0$$6P:(DE-He78)26a1176cd8450660333a012075050072$$aDeutsches Krebsforschungszentrum$$b6$$kDKFZ
000141663 9131_ $$0G:(DE-HGF)POF3-315$$1G:(DE-HGF)POF3-310$$2G:(DE-HGF)POF3-300$$3G:(DE-HGF)POF3$$4G:(DE-HGF)POF$$aDE-HGF$$bGesundheit$$lKrebsforschung$$vImaging and radiooncology$$x0
000141663 9141_ $$y2018
000141663 915__ $$0StatID:(DE-HGF)0100$$2StatID$$aJCR$$bIEEE T PATTERN ANAL : 2017
000141663 915__ $$0StatID:(DE-HGF)0200$$2StatID$$aDBCoverage$$bSCOPUS
000141663 915__ $$0StatID:(DE-HGF)0300$$2StatID$$aDBCoverage$$bMedline
000141663 915__ $$0StatID:(DE-HGF)0600$$2StatID$$aDBCoverage$$bEbsco Academic Search
000141663 915__ $$0StatID:(DE-HGF)0030$$2StatID$$aPeer Review$$bASC
000141663 915__ $$0StatID:(DE-HGF)0199$$2StatID$$aDBCoverage$$bClarivate Analytics Master Journal List
000141663 915__ $$0StatID:(DE-HGF)0110$$2StatID$$aWoS$$bScience Citation Index
000141663 915__ $$0StatID:(DE-HGF)0150$$2StatID$$aDBCoverage$$bWeb of Science Core Collection
000141663 915__ $$0StatID:(DE-HGF)0111$$2StatID$$aWoS$$bScience Citation Index Expanded
000141663 915__ $$0StatID:(DE-HGF)1160$$2StatID$$aDBCoverage$$bCurrent Contents - Engineering, Computing and Technology
000141663 915__ $$0StatID:(DE-HGF)9905$$2StatID$$aIF >= 5$$bIEEE T PATTERN ANAL : 2017
000141663 9201_ $$0I:(DE-He78)E130-20160331$$kE130$$lE130 Intelligente Medizinische Systeme$$x0
000141663 9201_ $$0I:(DE-He78)E132-20160331$$kE132$$lMedizinische Bildverarbeitung$$x1
000141663 9201_ $$0I:(DE-He78)C070-20160331$$kC070$$lC070 Klinische Epidemiologie und Alternf.$$x2
000141663 980__ $$ajournal
000141663 980__ $$aVDB
000141663 980__ $$aI:(DE-He78)E130-20160331
000141663 980__ $$aI:(DE-He78)E132-20160331
000141663 980__ $$aI:(DE-He78)C070-20160331
000141663 980__ $$aUNRESTRICTED