TY  - JOUR
AU  - Imrie, Fergus
AU  - Denner, Stefan
AU  - Brunschwig, Lucas S
AU  - Maier-Hein, Klaus
AU  - van der Schaar, Mihaela
TI  - Automated Ensemble Multimodal Machine Learning for Healthcare.
JO  - IEEE journal of biomedical and health informatics
VL  - 29
IS  - 6
SN  - 2168-2194
CY  - New York, NY
PB  - IEEE
M1  - DKFZ-2025-01558
SP  - 4213 - 4226
PY  - 2025
AB  - The application of machine learning in medicine and healthcare has led to the creation of numerous diagnostic and prognostic models. However, despite their success, current approaches generally issue predictions using data from a single modality. This stands in stark contrast with clinician decision-making which employs diverse information from multiple sources. While several multimodal machine learning approaches exist, significant challenges in developing multimodal systems remain that are hindering clinical adoption. In this paper, we introduce a multimodal framework, AutoPrognosis-M, that enables the integration of structured clinical (tabular) data and medical imaging using automated machine learning. AutoPrognosis-M incorporates 17 imaging models, including convolutional neural networks and vision transformers, and three distinct multimodal fusion strategies. In an illustrative application using a multimodal skin lesion dataset, we highlight the importance of multimodal machine learning and the power of combining multiple fusion strategies using ensemble learning. We have open-sourced our framework as a tool for the community and hope it will accelerate the uptake of multimodal machine learning in healthcare and spur further innovation.
KW  - Humans
KW  - Machine Learning
KW  - Neural Networks, Computer
KW  - Image Interpretation, Computer-Assisted: methods
LB  - PUB:(DE-HGF)16
C6  - pmid:40031027
DO  - DOI:10.1109/JBHI.2025.3530156
UR  - https://inrepo02.dkfz.de/record/303207
ER  -