Touching Products Virtually: Facilitating Consumer Mental Imagery with Gesture Control and Visual Presentation

Abstract

Gesture-based interaction has greatly changed the way in which we interact with online products by allowing users to control digital systems with hand movements. This study investigates how gesture-based interaction modes, namely, mid-air gesture and touchscreen gesture, compared with mouse-based interaction, affect consumers’ virtual product experiences (VPE) by eliciting mental imagery (i.e., haptic imagery and spatial imagery). Furthermore, we explore how visual product presentation can be designed to facilitate different types of interaction modes. Through a lab experiment, we find that touchscreen gesture outper- forms mid-air gesture and mouse-based interaction in terms of eliciting haptic imagery, and this effect is mitigated when 3D presentation is used. We also find that mid-air gesture outperforms touchscreen gesture and mouse-based interaction in terms of eliciting spatial imagery when 3D presentation is used. Both haptic imagery and spatial imagery can further reduce consumers’ product uncertainty. Our results extend prior work on interactivity design of VPE and further contribute to the emerging literature on gesture-based interaction.