Contributions of expected learning progress and perceptual novelty to curiosity-driven exploration

Bibliographic Data

ID21306979
AuthorsFrancesco Poli (0000-0001-9841-0056, Radboud University Nijmegen, corresponding author), Marlene Meyer (0000-0003-2229-6933, Radboud University Nijmegen), Rogier B Mars (0000-0001-6302-8631, Radboud University Nijmegen), Sabine Hunnius (0000-0001-5510-4032, Radboud University Nijmegen)
Year2022
Volume225
Pages105119
Publication date2022-08-01
Peer ReviewedYes
Open AccessYes
TypeARTICLE
VenueCognition (JOURNAL)
Journal identifiersISSN: 0010-0277 • E-ISSN: 1873-7838
PublisherElsevier BV (PUBLISHER)
DOI10.1016/j.cognition.2022.105119
PMID35421742
OpenAlexW4224016353
LanguageEN
Citations received9
References cited35

Exploration is curiosity-driven when it relies on the intrinsic motivation to know rather than on extrinsic rewards. Recent evidence shows that artificial agents perform better on a variety of tasks when their learning is curiosity-driven, and humans often engage in curiosity-driven learning when sampling information from the environment. However, which mechanisms underlie curiosity is still unclear. Here, we let participants freely explore different unknown environments that contained learnable sequences of events with varying degrees of noise and volatility. A hierarchical reinforcement learning model captured how participants were learning in these different kinds of unknown environments, and it also tracked the errors they expected to make and the learning opportunities they were planning to seek. With this computational approach, we show that participants' exploratory behavior is guided by learning progress and perceptual novelty. Moreover, we demonstrate an overall tendency of participants to avoid extreme forms of uncertainty. These findings elucidate the cognitive mechanisms that underlie curiosity-driven exploration of unknown environments. Implications of this novel way of quantifying curiosity within a reinforcement learning framework are discussed

Cognition · Cognitive psychology · Cognitive science · Curiosity · Novelty · Perception · Reinforcement learning · Variety (cybernetics) · Artificial Intelligence · Child and Animal Learning Development · Computer Science · Face Recognition and Perception · Neuroscience · Psychological and Educational Research Studies · Psychology · Social Psychology

Tools

Open DOIOpen Access