PuSH - Publication Server of Helmholtz Zentrum München

Hedrich, N.L.* ; Schulz, E. ; Hall-McMaster, S.* ; Schuck, N.W.*

An inductive bias for slowly changing features in human reinforcement learning.

PLoS Comput. Biol. 20, 30:e1012568 (2024)
Postprint DOI PMC
Open Access Gold
Identifying goal-relevant features in novel environments is a central challenge for efficient behaviour. We asked whether humans address this challenge by relying on prior knowledge about common properties of reward-predicting features. One such property is the rate of change of features, given that behaviourally relevant processes tend to change on a slower timescale than noise. Hence, we asked whether humans are biased to learn more when task-relevant features are slow rather than fast. To test this idea, 295 human participants were asked to learn the rewards of two-dimensional bandits when either a slowly or quickly changing feature of the bandit predicted reward. Across two experiments and one preregistered replication, participants accrued more reward when a bandit's relevant feature changed slowly, and its irrelevant feature quickly, as compared to the opposite. We did not find a difference in the ability to generalise to unseen feature values between conditions. Testing how feature speed could affect learning with a set of four function approximation Kalman filter models revealed that participants had a higher learning rate for the slow feature, and adjusted their learning to both the relevance and the speed of feature changes. The larger the improvement in participants' performance for slow compared to fast bandits, the more strongly they adjusted their learning rates. These results provide evidence that human reinforcement learning favours slower features, suggesting a bias in how humans approach reward learning.
Altmetric
Additional Metrics?
[➜Log in]
Edit extra informations Login
Publication type Article: Journal article
Document type Scientific Article
Corresponding Author
Keywords Model Selection; Representations; Statistics; Attention; Slowness
ISSN (print) / ISBN 1553-734X
e-ISSN 1553-7358
Journal PLoS Computational Biology
Quellenangaben Volume: 20, Issue: 11, Pages: 30, Article Number: e1012568 Supplement: ,
Publisher Public Library of Science (PLoS)
Publishing Place 1160 Battery Street, Ste 100, San Francisco, Ca 94111 Usa
Non-patent literature Publications
Reviewing status Peer reviewed
Institute(s) Helmholtz Artifical Intelligence Cooperation Unit (HAICU)
Grants BMBF (Excellence Strategy of the Federal Government and the Laender)
Max Planck Society
European Research Council Starting grant
Alexander von Humboldt Fellowship
Einstein Centre for Neurosciences PhD Fellowship