Perception Of Materials In Virtual Reality Based On Their Audiovisual Properties

Abstract

This study examined the effects of cue conflicts between auditory and visual material information in a virtual environment. All combinations of impact sounds and visual textures for four materials were paired, creating sixteen conditions. Participants, wearing a VR headset, viewed the rendered target object and heard the paired sound when it was struck with a virtual metal rod. To study the effect of agency, half the trials involved an agent striking the target (agent-interaction), while in the other half, participants struck it themselves (self-interaction). Once they classified the material of the target object, their responses and response times were recorded.

Results show that participants relied largely on auditory properties when classifying materials, no significant difference was found between agent-interaction and self-interaction modes, and in four conditions, potential audiovisual illusions were observed. These findings underscore the importance of high-quality auditory cues in VR, as discordant signals can distort perceived material properties.