PuSH - Publikationsserver des Helmholtz Zentrums München

Hedrich, N.L.* ; Schulz, E. ; Hall-McMaster, S.* ; Schuck, N.W.*

An inductive bias for slowly changing features in human reinforcement learning.

PLoS Comput. Biol. 20, 30:e1012568 (2024)
Postprint DOI PMC
Open Access Gold
Identifying goal-relevant features in novel environments is a central challenge for efficient behaviour. We asked whether humans address this challenge by relying on prior knowledge about common properties of reward-predicting features. One such property is the rate of change of features, given that behaviourally relevant processes tend to change on a slower timescale than noise. Hence, we asked whether humans are biased to learn more when task-relevant features are slow rather than fast. To test this idea, 295 human participants were asked to learn the rewards of two-dimensional bandits when either a slowly or quickly changing feature of the bandit predicted reward. Across two experiments and one preregistered replication, participants accrued more reward when a bandit's relevant feature changed slowly, and its irrelevant feature quickly, as compared to the opposite. We did not find a difference in the ability to generalise to unseen feature values between conditions. Testing how feature speed could affect learning with a set of four function approximation Kalman filter models revealed that participants had a higher learning rate for the slow feature, and adjusted their learning to both the relevance and the speed of feature changes. The larger the improvement in participants' performance for slow compared to fast bandits, the more strongly they adjusted their learning rates. These results provide evidence that human reinforcement learning favours slower features, suggesting a bias in how humans approach reward learning.
Altmetric
Weitere Metriken?
[➜Einloggen]
Zusatzinfos bearbeiten [➜Einloggen]
Publikationstyp Artikel: Journalartikel
Dokumenttyp Wissenschaftlicher Artikel
Korrespondenzautor
Schlagwörter Model Selection; Representations; Statistics; Attention; Slowness
ISSN (print) / ISBN 1553-734X
e-ISSN 1553-7358
Zeitschrift PLoS Computational Biology
Quellenangaben Band: 20, Heft: 11, Seiten: 30, Artikelnummer: e1012568 Supplement: ,
Verlag Public Library of Science (PLoS)
Verlagsort 1160 Battery Street, Ste 100, San Francisco, Ca 94111 Usa
Nichtpatentliteratur Publikationen
Begutachtungsstatus Peer reviewed
Institut(e) Helmholtz Artifical Intelligence Cooperation Unit (HAICU)
Förderungen BMBF (Excellence Strategy of the Federal Government and the Laender)
Max Planck Society
European Research Council Starting grant
Alexander von Humboldt Fellowship
Einstein Centre for Neurosciences PhD Fellowship