Abstract: Although prior research has investigated how unisensory training modulates perceptual sensitivity and decision criteria, the computational mechanisms and neural underpinnings associated with category learning through coordinated auditory-visual experience remain insufficiently characterized. To address this gap, the present study systematically examines how multisensory category learning reconfigures the temporal dynamics of feature processing and category selection. We combined high-temporal-resolution electroencephalography (EEG) with trial-by-trial hierarchical Bayesian drift-diffusion modeling (HDDM) to jointly characterize neural response trajectories and latent cognitive processes.
The participants first completed a multisensory category-learning phase. Four arbitrary categories were defined by orthogonal auditory (pure-tone frequency) and visual (vehicle shape) morph continua. Each category corresponded to a distinct quadrant in the resulting two-dimensional stimulus space. Participants received accuracy feedback after every response. In the subsequent test phase, participants performed a two-alternative odd-ball task; on each trial, they judged on whether a briefly presented multisensory stimulus belonged to a prespecified target category or to any of the three nontarget categories. Critically, the stimuli were varied parametrically along both the auditory and visual continua. Half of the trials featured prototypical exemplars (0% morph distance from the category centroid), whereas the other half featured high-variance exemplars (±49% morph distance). This finding fully crossed, orthogonal manipulation of perceptual typicality (prototype vs. deviant) and decision relevance (target vs. nontarget) enabled a functional dissociation between neural processes supporting early feature processing and those subserving category selection information.
The behavioral results revealed that prototypical stimuli were classified with significantly higher accuracy and shorter response times than deviant stimuli were. In contrast, nontargeted decisions were both more accurate and faster than target decisions were, whereas nontarget trials comprised a heterogeneous set of within- and between-category deviants. To isolate the latent cognitive mechanisms underlying these behavioral dissociations, we applied hierarchical Bayesian drift?diffusion modeling (HDDM). With respect to stimulus typicality, prototypes were associated with higher drift rates (v), more conservative decision thresholds (a), and stronger starting-point biases (z) toward the correct response boundary. With respect to decision relevance, target decisions exhibited lower drift rates, more liberal thresholds, and attenuated starting-point biases than nontarget decisions did. Neurophysiological analyses extended these computational insights. Prototypical stimuli elicited larger N250 amplitudes over bilateral occipital-temporal electrodes, followed by enhanced frontal selection positivity (FSP) and a late positive component (LPC) over fronto-central and parietal electrodes. Time-frequency analyses further revealed concomitant increases in alpha and beta power over the parietal and occipital regions. In contrast, target decisions were associated with the larger FSP and dual-peaked LPC complexes over frontoparietal sites. They were also accompanied by widespread suppression of alpha and beta oscillations. Importantly, multivariate ridge regression indicated that trial-level FSP amplitudes and alpha/beta power positively predicted drift rates for deviant stimuli. With respect to target categorization, both FSP/LPC amplitudes and alpha/beta power positively predicted drift rates. Moreover, alpha power exerted a robust negative influence on decision thresholds for both the deviant and target conditions.
Collectively, these findings support a dual-pathway architecture dynamically shaped by multisensory category learning. A rapid feed-forward pathway, indexed by the FSP and increased alpha/beta synchrony over parieto-occipital regions, supports the automatic extraction of diagnostic multisensory features, operating independently of current task demands. A slower, memory-dependent pathway—characterized by functional coupling between the FSP and the late positive complex (LPC), alongside desynchronization of alpha/beta oscillations—is engaged during active rule retrieval and conflict monitoring, particularly under conditions of decision uncertainty.

Key words: feature processing, category selection, drift diffusion model, event-related potentials, neurocomputational mechanism