关键词: 好奇, 内部动机, 强化学习, 信息寻求, 好奇满足

Abstract:

The curiosity feedback loop model considers the nature of curiosity as intrinsic motivation, regards information-seeking behaviors as an evoked outcome of curiosity, emotions as a concomitant product of curiosity, and emphasizes the dynamic and changing nature of curiosity. The model incorporates the expected value of control model and Bayesian reinforcement learning framework, and integrates research evidence from multiple functional brain systems such as the monitoring system, reward system, and control system. The model provides new ideas for understanding the neurophysiological mechanisms of curiosity.

The curiosity feedback loop model decomposes a curious event into the following six processes: perceived information gap, curiosity generation, value assessment of control, information seeking, curiosity satisfaction, and information integration. Specifically, an individual develops a desire for information (reward) as a result of perceived information gap, and thus curiosity occurs. Subsequently, the individual performs a value assessment of control based on the current activity state or the expected outcome of potential behavior (e.g., information seeking). The decision to initiate information seeking is made by assessing how much control is needed to be exerted for the next behavior. Information seeking changes the information input, which in turn changes the individual state, and the new state becomes a cue for the value assessment of control, influencing a new round of information-seeking behavior. Information seeking corresponds to different outcomes: curiosity satisfied or curiosity unsatisfied. Curiosity satisfaction implies acquisition of information reward, which tends to increase the individual's estimate of the expected value of the new information and the validity of the behavior, which in turn reinforces information-seeking behavior. The further integration of information leads to the expansion of prior knowledge, and the expanded prior knowledge makes it easier for individuals to realize new information gap and stimulates new information-seeking behavior. This process creates a positive feedback loop that contributes to sustainable knowledge acquisition. Conversely, if information acquisition fails, the positive feedback loop will be interrupted.

This dynamic loop of curiosity is embedded in the lifelong development of the individual, changing with the accumulation of experience and the development of brain. On the one hand, it is influenced and limited by various physiological changes in life development; on the other hand, the repeated consolidation of the curiosity loop also causes physiological changes that in turn affect an individual’s life development. In simple terms, the two curiosity-related systems - the dopaminergic and noradrenergic systems - contribute to short-term attention, memory, and information-seeking behavior, and are also beneficial for the long-term maintenance and improvement of cognitive function.

Curiosity studies become increasingly multidisciplinary and cross-cutting, and therefore a unified concept and framework is needed as a basis for further scientific discussion. Regarding future curiosity research, increased attention could be given to: (1) strengthening the focus on curiosity satisfaction; (2) improving existing paradigms for curiosity research; (3) enhancing comparisons of the use of internal and external rewards; and (4) valuing the developmental aspects of curiosity research.

Key words: curiosity, intrinsic motivation, reinforcement learning, information seeking, curiosity relief