guest ::
| Home > Publications database > Leveraging foundation models for content-based image retrieval in radiology. |
| Home > Publications database > Leveraging foundation models for content-based image retrieval in radiology. |
| Journal Article | DKFZ-2025-01429 |
; ; ; ; ; ; ; ; ; ;
2025
Elsevier Science
Amsterdam [u.a.]
This record in other databases: 
Please use a persistent id in citations: doi:10.1016/j.compbiomed.2025.110640
Abstract: Content-based image retrieval (CBIR) has the potential to significantly improve diagnostic aid and medical research in radiology. However, current CBIR systems face limitations due to their specialization to certain pathologies, limiting their utility. On the other hand, several vision foundation models have been shown to produce general-purpose visual features. Therefore, in this work, we propose using vision foundation models as powerful and versatile off-the-shelf feature extractors for content-based image retrieval. Our contributions include: (1) benchmarking a diverse set of vision foundation models on an extensive dataset comprising 1.6 million 2D radiological images across four modalities and 161 pathologies; (2) identifying weakly-supervised models, particularly BiomedCLIP, as highly effective, achieving a P@1 of up to 0.594 (P@3: 0.590, P@5: 0.588, P@10: 0.583), comparable to specialized CBIR systems but without additional training; (3) conducting an in-depth analysis of the impact of index size on retrieval performance; (4) evaluating the quality of embedding spaces generated by different models; and (5) investigating specific challenges associated with retrieving anatomical versus pathological structures. Despite these challenges, our research underscores the vast potential of foundation models for CBIR in radiology, proposing a shift towards versatile, general-purpose medical image retrieval systems that do not require specific tuning. Our code, dataset splits and embeddings are publicly available here.
Keyword(s): Content-based image retrieval ; Foundation models ; Medical imaging ; Self-supervised learning ; Supervised learning ; Weakly-supervised learning
; BIOSIS Previews ; Biological Abstracts ; Clarivate Analytics Master Journal List ; Current Contents - Life Sciences ; Ebsco Academic Search ; Essential Science Indicators ; IF >= 5 ; JCR ; Nationallizenz
; SCOPUS ; Science Citation Index Expanded ; Web of Science Core Collection
|
The record appears in these collections: |
