PuSH - Publication Server of Helmholtz Zentrum München

Modirshanechi, A. ; Lin, W.H.* ; Xu, H.A.* ; Herzog, M.H.* ; Gerstner, W.*

Novelty as a drive of human exploration in complex stochastic environments.

Proc. Natl. Acad. Sci. U.S.A. 122:e2502193122 (2025)
Publ. Version/Full Text Research data DOI PMC
Open Access Hybrid
Creative Commons Lizenzvertrag
In order to find extrinsic rewards, humans explore their environment even if exploration requires several intermediate, reward-free decisions. It has been hypothesized that intrinsic rewards, such as novelty, surprise, or information gain, guide this reward-free exploration. However, in artificial agents, different intrinsic reward signals induce exploration strategies that respond differently to stochasticity. In particular, some strategies are vulnerable to the "noisy TV problem," i.e., an attraction to irrelevant stochastic stimuli. Here, we ask whether humans exhibit a similar attraction to reward-free stochasticity. We design a multistep decision-making paradigm in which participants search for rewarding states in a complex environment containing a highly stochastic but reward-free subregion. We show that i) participants persistently explore the stochastic subregion, and ii) their decisions are best explained by a novelty-driven exploration strategy, compared to alternatives driven by information gain or surprise. Our findings suggest that novelty and extrinsic rewards jointly control human exploration in complex environments.
Altmetric
Additional Metrics?
[➜Log in]
Edit extra informations Login
Publication type Article: Journal article
Document type Scientific Article
Keywords Exploration ; Human Behavior ; Information-seeking ; Reinforcement Learning; Bayesian Model Selection; Information; Curiosity; Exploitation; Uncertainty; Systems; Reward
ISSN (print) / ISBN 0027-8424
e-ISSN 1091-6490
Journal Proceedings of the National Academy of Sciences of the United States of America
Quellenangaben Volume: 122, Issue: 39, Pages: , Article Number: e2502193122 Supplement: ,
Publisher National Academy of Sciences
Publishing Place 2101 Constitution Ave Nw, Washington, Dc 20418 Usa
Reviewing status Peer reviewed
Institute(s) Human-Centered AI (HCA)
Grants European Union
Swiss NSF