PuSH - Publikationsserver des Helmholtz Zentrums München

Schulze Buschoff, L.M. ; Akata, E. ; Bethge, M.* ; Schulz, E.

Visual cognition in multimodal large language models.

Nat. Mach. Intell. 7, 96-106 (2025)
Verlagsversion Forschungsdaten DOI
Open Access Gold
Creative Commons Lizenzvertrag
A chief goal of artificial intelligence is to build machines that think like people. Yet it has been argued that deep neural network architectures fail to accomplish this. Researchers have asserted these models’ limitations in the domains of causal reasoning, intuitive physics and intuitive psychology. Yet recent advancements, namely the rise of large language models, particularly those designed for visual processing, have rekindled interest in the potential to emulate human-like cognitive abilities. This paper evaluates the current state of vision-based large language models in the domains of intuitive physics, causal reasoning and intuitive psychology. Through a series of controlled experiments, we investigate the extent to which these modern models grasp complex physical interactions, causal relationships and intuitive understanding of others’ preferences. Our findings reveal that, while some of these models demonstrate a notable proficiency in processing and interpreting visual data, they still fall short of human capabilities in these areas. Our results emphasize the need for integrating more robust mechanisms for understanding causality, physical dynamics and social cognition into modern-day, vision-based language models, and point out the importance of cognitively inspired benchmarks.
Altmetric
Weitere Metriken?
[➜Einloggen]
Zusatzinfos bearbeiten [➜Einloggen]
Publikationstyp Artikel: Journalartikel
Dokumenttyp Wissenschaftlicher Artikel
Korrespondenzautor
ISSN (print) / ISBN 2522-5839
e-ISSN 2522-5839
Zeitschrift Nature machine intelligence
Quellenangaben Band: 7, Heft: 1, Seiten: 96-106 Artikelnummer: , Supplement: ,
Verlag Springer
Verlagsort [London]
Nichtpatentliteratur Publikationen
Begutachtungsstatus Peer reviewed
Institut(e) Institute of AI for Health (AIH)