PuSH - Publication Server of Helmholtz Zentrum München

Dalmonte, F.* ; Bayar, E.* ; Akbas, E. ; Georgescu, M.-I.

Q-Former Autoencoder: A Modern Framework for Medical Anomaly Detection.

In: (Proceedings 2026 IEEE Cvf Winter Conference on Applications of Computer Vision Wacv 2026, 6-10 March 2026, Tucson). 2026. 7985-7995 (Proceedings 2026 IEEE Cvf Winter Conference on Applications of Computer Vision Wacv 2026)
DOI
Anomaly detection in medical images is an important yet challenging task due to the diversity of possible anomalies and the practical impossibility of collecting comprehensively annotated data sets. In this work, we tackle unsupervised medical anomaly detection proposing a modernized autoencoder-based framework, the Q-Former Autoencoder, that leverages state-of-the-art pretrained vision foundation models, such as DINO, DINOv2 and Masked Autoencoder. Instead of training encoders from scratch, we directly utilize frozen vision foundation models as feature extractors, enabling rich, multi-stage, high-level representations without domain-specific fine-tuning. We propose the usage of the Q-Former architecture as the bottleneck, which enables the control of the length of the reconstruction sequence, while efficiently aggregating multi-scale features. Additionally, we incorporate a perceptual loss computed using features from a pretrained Masked Autoencoder, guiding the reconstruction towards semantically meaningful structures. Our framework is evaluated on four diverse medical anomaly detection benchmarks, achieving state-of-the-art results on BraTS2021, RESC, and RSNA. Our results highlight the potential of vision foundation model encoders, pretrained on natural images, to generalize effectively to medical image analysis tasks without further fine-tuning. We release the code and models at https://github.com/emirhanbayar/QFAE.
Altmetric
Additional Metrics?
[➜Log in]
Edit extra informations Login
Publication type Article: Conference contribution
Keywords Anomaly Detection ; Autoencoders ; Medical Images ; Unsupervised Learning
ISSN (print) / ISBN [9798331555115]
Conference Title Proceedings 2026 IEEE Cvf Winter Conference on Applications of Computer Vision Wacv 2026
Conference Date 6-10 March 2026
Conference Location Tucson
Quellenangaben Volume: , Issue: , Pages: 7985-7995 Article Number: , Supplement: ,
Institute(s) Helmholtz Artifical Intelligence Cooperation Unit (HAICU)
Institute of Environmental Medicine (IEM)