Abstract:

Category learning in multisensory environments, which is a fundamental human cognitive ability, has significant implications for understanding cross-modal knowledge representation. This study systematically examines the cognitive characteristics and neural mechanisms of multisensory category learning by integrating event-related potential (ERP) techniques and drift-diffusion modeling (DDM). We established three experimental groups— the early-stage, middle-state and later-stage groups—in which participants acquired the ability to discriminate four categories of multisensory stimuli through corrective feedback. During the learning process, we simultaneously recorded electroencephalographic (EEG) data and employed a multimodal analytical approach integrating neural oscillation with computational modeling by using the DDM. This combined methodology enabled us to systematically examine how varying degrees of learning proficiency modulate the neurocomputational mechanisms underlying multisensory category acquisition. From a behavioral perspective, the middle- and later-stage learning groups demonstrated significantly greater accuracy, reaction time and drift rates than the early-stage learning group, along with a decision threshold bias toward correct responses. At the neural level, middle- and later-stage learning elicited amplified amplitudes in the N1, P1, and LPC components while decreasing the amplitude of the N250-FSP complex. Time?frequency analyses demonstrated significant power reductions in the theta, alpha, and delta frequency bands. Regression analyses identified distinct neural predictors: variations in drift rates were jointly explained by reductions in N250-FSP amplitude and theta oscillations, whereas decision threshold biases were predicted by coordinated activity in early perceptual processing (P1), feature discrimination (N250-FSP), and memory retrieval (LPC) components. These findings reveal a dual-mechanism framework through which learning sufficiency optimizes decision efficiency. (1) Enhanced information accumulation rates are associated with reduced N250-FSP amplitudes and theta-band reorganization, reflecting streamlined feature integration and conflict resolution. (2) Decision threshold shifts result from the synergistic interplay of sensory encoding (P1), categorical feature discrimination (N250-FSP), and postretrieval monitoring (LPC). Notably, the dissociation between theta-mediated drift rate modulation and fronto-posterior ERP dynamics in threshold adjustment offers compelling evidence for parallel neural pathways that govern distinct decision parameters. This study advances multisensory learning theories by elucidating the neurocognitive mechanisms underlying learning optimization, thereby providing insights regarding the development of targeted interventions in adaptive learning systems and cross-modal training paradigms. These findings highlight the pivotal role of learning duration in shaping both the neurocomputational architecture of decision-making processes and the efficiency of cross-modal knowledge consolidation.

Key words: multisensory, category learning, drift-diffusion model, EEG