000180603 001__ 180603
000180603 005__ 20241220120851.0
000180603 0247_ $$2doi$$a10.1002/jrsm.1589
000180603 0247_ $$2pmid$$apmid:35798691
000180603 0247_ $$2ISSN$$a1759-2879
000180603 0247_ $$2ISSN$$a1759-2887
000180603 0247_ $$2altmetric$$aaltmetric:130843578
000180603 037__ $$aDKFZ-2022-01433
000180603 041__ $$aEnglish
000180603 082__ $$a900
000180603 1001_ $$0P:(DE-He78)547386e1dd3330f9f40321e89ec05354$$aKebede, Mihiretu$$b0$$eFirst author$$udkfz
000180603 245__ $$aIn-depth evaluation of machine learning methods for semi-automating article screening in a systematic review of mechanistic literature.
000180603 260__ $$aSao Paulo$$bPrograma de Estudos Pós-Graduados em História$$c2023
000180603 3367_ $$2DRIVER$$aarticle
000180603 3367_ $$2DataCite$$aOutput Types/Journal article
000180603 3367_ $$0PUB:(DE-HGF)16$$2PUB:(DE-HGF)$$aJournal Article$$bjournal$$mjournal$$s1678876342_29492
000180603 3367_ $$2BibTeX$$aARTICLE
000180603 3367_ $$2ORCID$$aJOURNAL_ARTICLE
000180603 3367_ $$00$$2EndNote$$aJournal Article
000180603 500__ $$a#EA:C020#LA:C020# / 2023 Mar;14(2):156-172
000180603 520__ $$aWe aimed to evaluate the performance of supervised machine learning algorithms in predicting articles relevant for full-text review in a systematic review. Overall, 16,430 manually screened titles/abstracts, including 861 references identified relevant for full-text review were used for the analysis. Of these, 40% (n=6573) were sub-divided for training (70%) and testing (30%) the algorithms. The remaining 60% (n=9857) were used as a validation set. We evaluated down- and up-sampling methods and compared unigram, bigram, and singular value decomposition (SVD) approaches. For each approach, Naïve Bayes, Support Vector Machines (SVM), regularized logistic regressions, Neural Networks, random forest, Logit boost, and XGBoost were implemented using simple term frequency or Tf-Idf feature representations. Performance was evaluated using sensitivity, specificity, precision and area under the Curve. We combined predictions of best-performing algorithms (Youden Index ≥0.3 with sensitivity/specificity≥70/60%). In down sample unigram approach, Naïve Bayes, SVM/quanteda text models with Tf-Idf, and linear SVM e1071 package with Tf-Idf achieved >90% sensitivity at specificity >65%. Combining the predictions of the 10 best-performing algorithms improved the performance to reach 95% sensitivity and 64% specificity in the validation set. Crude screening burden was reduced by 61% (5979) (adjusted: 80.3%) with 5% (27) false negativity rate. All the other approaches yielded relatively poorer performances. The down sampling unigram approach achieved good performance in our data. Combining the predictions of algorithms improved sensitivity while screening burden was reduced by almost two-third. Implementing machine learning approaches in title/abstract screening should be investigated further toward refining these tools and automating their implementation. This article is protected by copyright. All rights reserved.
000180603 536__ $$0G:(DE-HGF)POF4-313$$a313 - Krebsrisikofaktoren und Prävention (POF4-313)$$cPOF4-313$$fPOF IV$$x0
000180603 588__ $$aDataset connected to CrossRef, PubMed, , Journals: inrepo02.dkfz.de
000180603 650_7 $$2Other$$aAutomated screening
000180603 650_7 $$2Other$$aCitation Screening
000180603 650_7 $$2Other$$aMachine Learning
000180603 650_7 $$2Other$$aNLP
000180603 650_7 $$2Other$$aNatural Language Processing
000180603 650_7 $$2Other$$aSystematic review
000180603 650_7 $$2Other$$aText mining
000180603 7001_ $$0P:(DE-He78)2d4cac0f4bf270325b98fe07036dc6c6$$aLe Cornet, Charlotte$$b1$$udkfz
000180603 7001_ $$0P:(DE-He78)74a6af8347ec5cbd4b77e562e10ca1f2$$aTurzanski-Fortner, Renée$$b2$$eLast author$$udkfz
000180603 773__ $$0PERI:(DE-600)2548499-0$$a10.1002/jrsm.1589$$gp. jrsm.1589$$n2$$p156-172$$tCordis$$v14$$x1759-2879$$y2023
000180603 8767_ $$82022 (V10366)$$92022-06-23$$d2024-12-19$$eHybrid-OA$$jZahlung erfolgt
000180603 909CO $$ooai:inrepo02.dkfz.de:180603$$pVDB$$pOpenAPC$$popenCost
000180603 9101_ $$0I:(DE-588b)2036810-0$$6P:(DE-He78)547386e1dd3330f9f40321e89ec05354$$aDeutsches Krebsforschungszentrum$$b0$$kDKFZ
000180603 9101_ $$0I:(DE-588b)2036810-0$$6P:(DE-He78)2d4cac0f4bf270325b98fe07036dc6c6$$aDeutsches Krebsforschungszentrum$$b1$$kDKFZ
000180603 9101_ $$0I:(DE-588b)2036810-0$$6P:(DE-He78)74a6af8347ec5cbd4b77e562e10ca1f2$$aDeutsches Krebsforschungszentrum$$b2$$kDKFZ
000180603 9131_ $$0G:(DE-HGF)POF4-313$$1G:(DE-HGF)POF4-310$$2G:(DE-HGF)POF4-300$$3G:(DE-HGF)POF4$$4G:(DE-HGF)POF$$aDE-HGF$$bGesundheit$$lKrebsforschung$$vKrebsrisikofaktoren und Prävention$$x0
000180603 9141_ $$y2022
000180603 915__ $$0StatID:(DE-HGF)3001$$2StatID$$aDEAL Wiley$$d2021-02-03$$wger
000180603 915__ $$0StatID:(DE-HGF)0160$$2StatID$$aDBCoverage$$bEssential Science Indicators$$d2021-02-03
000180603 915__ $$0StatID:(DE-HGF)0113$$2StatID$$aWoS$$bScience Citation Index Expanded$$d2021-02-03
000180603 915__ $$0StatID:(DE-HGF)0100$$2StatID$$aJCR$$bRES SYNTH METHODS : 2022$$d2023-10-26
000180603 915__ $$0StatID:(DE-HGF)0300$$2StatID$$aDBCoverage$$bMedline$$d2023-10-26
000180603 915__ $$0StatID:(DE-HGF)0600$$2StatID$$aDBCoverage$$bEbsco Academic Search$$d2023-10-26
000180603 915__ $$0StatID:(DE-HGF)0030$$2StatID$$aPeer Review$$bASC$$d2023-10-26
000180603 915__ $$0StatID:(DE-HGF)0199$$2StatID$$aDBCoverage$$bClarivate Analytics Master Journal List$$d2023-10-26
000180603 915__ $$0StatID:(DE-HGF)0150$$2StatID$$aDBCoverage$$bWeb of Science Core Collection$$d2023-10-26
000180603 915__ $$0StatID:(DE-HGF)1110$$2StatID$$aDBCoverage$$bCurrent Contents - Clinical Medicine$$d2023-10-26
000180603 915__ $$0StatID:(DE-HGF)0200$$2StatID$$aDBCoverage$$bSCOPUS$$d2023-10-26
000180603 915__ $$0StatID:(DE-HGF)9905$$2StatID$$aIF >= 5$$bRES SYNTH METHODS : 2022$$d2023-10-26
000180603 915pc $$0PC:(DE-HGF)0000$$2APC$$aAPC keys set
000180603 915pc $$0PC:(DE-HGF)0001$$2APC$$aLocal Funding
000180603 9202_ $$0I:(DE-He78)C020-20160331$$kC020$$lC020 Epidemiologie von Krebs$$x0
000180603 9201_ $$0I:(DE-He78)C020-20160331$$kC020$$lC020 Epidemiologie von Krebs$$x0
000180603 9200_ $$0I:(DE-He78)C020-20160331$$kC020$$lC020 Epidemiologie von Krebs$$x0
000180603 980__ $$ajournal
000180603 980__ $$aVDB
000180603 980__ $$aI:(DE-He78)C020-20160331
000180603 980__ $$aUNRESTRICTED
000180603 980__ $$aAPC