Abstract: The subject-performed task (SPT) effect refers to the finding that action encoding (physically performing described actions) yields better recall of verbal instructions than verbal encoding (merely listening to them). Multimodal theory proposes that this advantage arises because action encoding recruits additional sensory channels beyond the verbal modality, including kinesthetic (motor execution), visual (e.g., observing objects and actions), and tactile input (e.g., touching object surfaces), thereby enriching memory traces and enhancing their retrievability.
Building on this framework, the present study examined how the conventionality of action-object pairings in verbal instructions moderates the multimodal dependencies of the SPT effect, with the aim of distinguishing critical from auxiliary sensory contributions. Three experiments were conducted using familiar everyday objects.
In Experiment 1 (N = 200), we manipulated instruction conventionality (conventional, e.g., “sharpen a pencil” non-conventional, e.g., “poke a pencil”) and the presence versus absence of object pictures (providing visual cues without physical interaction). Results showed that conventional instructions elicited a SPT effect (i.e., performance in the action encoding condition was superior to that in the verbal encoding condition) regardless of whether pictures were present. In contrast, non-conventional instructions failed to produce the SPT effect even when object pictures were available. We hypothesized that static visual input alone may be insufficient to support the online integration of verbs and nouns for non-conventional pairings, and that physical interaction with objects might be necessary. To test this hypothesis, Experiment 2 (N = 58) replaced pictures with real, manipulable objects, enabling participants to physically interact with them during encoding. Under these conditions, non-conventional instructions yielded a significant SPT effect when objects were present but not when absent, indicating that sensorimotor interaction, rather than mere visibility, is essential for the SPT effect with non-conventional instructions. However, because real objects concurrently engage both vision and touch, Experiment 2 could not isolate which sensory channel drives this effect. Experiment 3 (N = 54) addressed this by orthogonally manipulating visual access (blindfolded vs. unblindfolded) and fine tactile fidelity (wearing gloves vs. bare-handed) during object interaction, with a verbal-only baseline. Results showed that the SPT effect emerged only when vision was available, irrespective of fine tactile condition. Specifically, recall performance was significantly better in both “vision + touch” and “vision + no-fine-touch” action conditions than in the verbal baseline (ps < .001), whereas “no-vision + touch” and “no-vision + no-fine-touch” conditions did not differ from verbal encoding (ps > .999). These findings demonstrate that vision, rather than fine tactile input, is the necessary channel for the SPT effect with non-conventional instructions.
Collectively, these findings reveal that the sensory channel dependence of the SPT effect varies with instruction conventionality. For conventional instructions, kinesthetic input alone, without visual or tactile support, suffices to drive the SPT effect. For non-conventional instructions, the SPT effect requires the coordination of kinesthetic and visual input, while fine tactile information plays no independent role. These results refine multimodal theory by demonstrating that channel contributions are not simply additive, where more channels yield greater benefit, but are instead flexibly determined by whether pre-existing action-object associations are available. This advances the understanding of multimodal memory and offers practical guidance for memory support strategies in populations with sensory or neurodevelopmental differences (e.g., autism spectrum disorder), where optimizing visual-motor integration may enhance instruction memory without relying on tactile precision.

Key words: SPT effect, instruction memory, multimodal processing theory