PuSH - Publication Server of Helmholtz Zentrum München

Ortega, J.V. ; Lamm, L. ; Eckardt, F.* ; Schworm, B.* ; Jasnin, M. ; Peng, T.

Randomized-MLP Regularization Improves Domain Adaptation and Interpretability in DINOv2.

In: (39th Conference on Neural Information Processing Systems, NeurIPS 2025, 02-07 December 2025, San Diego). 2025. 861-895 (Advances in Neural Information Processing Systems ; 38)
Vision Transformers (ViTs), such as DINOv2, achieve strong performance across domains but often repurpose low-informative patch tokens in ways that reduce the interpretability of attention and feature maps. This challenge is especially evident in medical imaging, where domain shifts can degrade both performance and transparency. In this paper, we introduce Randomized-MLP (RMLP) regularization, a contrastive learning-based method that encourages more semantically aligned representations. We use RMLPs when fine-tuning DINOv2 to both medical and natural image modalities, showing that it improves or maintains downstream performance while producing more interpretable attention maps. We also provide a mathematical analysis of RMLPs, offering insights into its role in enhancing ViT-based models and advancing our understanding of contrastive learning.
Additional Metrics?
[➜Log in]
Edit extra informations Login
Publication type Article: Conference contribution
ISSN (print) / ISBN 1049-5258
Conference Title 39th Conference on Neural Information Processing Systems, NeurIPS 2025
Conference Date 02-07 December 2025
Conference Location San Diego
Quellenangaben Volume: 38, Issue: , Pages: 861-895 Article Number: , Supplement: ,
Institute(s) Helmholtz Artificial Intelligence Cooperation Unit (HAI)
Helmholtz Pioneer Campus (HPC)