Abstract:

Virtual reality creates an immersive experience for users by providing visual, auditory, and tactile information. However, tactile feedback still faces many technical bottlenecks, which limit natural interaction in the virtual reality environment. Pseudo-haptic technology based on multi-sensory illusion can enhance and enrich tactile perception with the help of information from other channels, which is one of the effective ways to optimize tactile perception in the virtual reality environment. In order to explore the problem in a more targeted way, the study focuses on roughness perception among different dimensions of tactile perception. We first discuss the relationship between visual, auditory, and tactile perception in roughness information integration and further analyze visual and auditory factors affecting roughness perception.

In the process of roughness perception, the channel with less noise and more reliable information will have a higher weight. By evaluating the roughness under every single channel (visual、auditory and tactile) condition,and each double-channel（visual-tactile and auditory-tactile) condition, previous studies have quantified the high weight of visual and auditory channels in roughness perception. It further verifies that roughness perception results from integrating tactile, visual, and auditory information. Therefore, it has excellent potential to manipulate visual and auditory factors to induce or enhance roughness perception in the virtual reality environment.

For visual factors, the texture of the surface, light or shadow, and the control display rate (CDR) may affect the perception of roughness. ①When the texture density of the object surface increase, the perceived roughness will increase accordingly; ②Since the angle of incidence influence the shadow of the surface texture, thus affecting the perception of roughness; ③ When the glossiness of surface decrease, the perceived roughness increase; ④Reducing the CDR of the virtual hand or presenting a vibratory virtual hand will increase the user's perception of the roughness.

For auditory factors, manipulating loudness and pitch/frequency can change the roughness perception: increasing loudness in the entire frequency of sound or increasing loudness in a high, medium, or low frequency band of sound can improve the roughness perception. There is a strong correlation between pitch/frequency and roughness perception. According to current research, we speculated that there might be an inverted U-shaped curve between pitch/frequency change and roughness perception.

In the future, in order to optimize the tactile roughness experience in virtual reality environment, studies can be carried out from the following aspects: Firstly, it is necessary to explore the differences between every single channel (visual, auditory, and tactile) information in the virtual reality environment and real environment respectively, since single channel information is the prerequisite for multi-sensory channel information in the virtual reality environment. Secondly, from the practical point of view, we can choose the channel more suitable to the current scene or task and provide sufficient information timely with emphasis. Thirdly, the difference of single-channel information in virtual reality will lead to information mismatch between different channels. Therefore, it is necessary to explore further how the brain integrates the originally different and mismatched multi-channel sensory information. Finally, most studies explored the usability of relevant sensory information on roughness perception from the perspective of single-channels or double-channels. Therefore, in the future, it is of significant importance to explore the information integration and processing rules of three channels in virtual reality.

Key words: roughness perception, multi-sensory channel integration, visual cues, auditory cues