Touch interactions are central to many human activities, but there are few technologies for computationally augmenting free-hand interactions with real environments. Here, we describe Tactile Echoes, a finger-wearable system for augmenting touch interactions with physical objects. This system captures and processes touch-elicited vibrations in real-time in order to enliven tactile experiences.  We process these signals via a parametric signal processing network in order to generate responsive tactile and auditory feedback. Just as acoustic echoes are produced through the delayed replication and modification of sounds, so are Tactile Echoes produced through transformations of vibrotactile inputs in the skin. The echoes also reflect the contact interactions and touched objects involved. A transient tap produces discrete echoes, while a continuous slide yields sustained feedback. We also demonstrate computational methods that allow these effects to be selectively assigned to different objects or actions by optically tracking the motion of the finger. A large variety of distinct multisensory effects can be designed via ten processing parameters. We investigated how Tactile Echoes are perceived in several perceptual experiments using multidimensional scaling methods. This allowed us to deduce low-dimensional, semantically grounded perceptual descriptions.  We describe several virtual and augmented reality applications of Tactile Echoes. In a user study, we found that these effects made interactions more responsive and engaging. Our findings show how to endow a large variety of touch interactions with expressive multisensory effects.

Movie file: Tactile Echoes.mp4

The movie shows the demonstration of Tactile Echoes device which is augmenting tactile sensation in free-hand interactions with real environments. This movie also presents the application of Tactile Echoes. For example, 2D touch display application with Tactile Echoes and using Tactile Echoes feedback in the Virtual Reality. The concept of Tactile Echoes, system, and hardware configuration are also included. With the Tactile Echoes device, we measured output and conducted perceptual study and user study. How to do the user study of Tactile Echoes is also described in the video.

Web page (Appendix of “Tactile Echoes: Multisensory Augmented Reality for the Hand”): Appendix_of_Tactile_Echoes.webloc

In the corresponding paper, we conducted 3 perceptual experiments to investigate how touch interaction augmented by the Tactile Echoes were perceived and to identify a perceptual space that adequately described the perceptual similarity of different output from Tactile Echoes. On the web page, additional three results from these experiments and a data set of parameters used in these three perceptual experiments are described. First, the result from the first perceptual experiment of descriptive word harvesting is described. Second, the result from word voting is showed. The selected rate and words are presented. Third, from the third perceptual study, the all R-Square value of regression MDS perceptual study is showed. It contains the statistical result of MDS analysis. Finally, the engineering parameters of Tactile Echoes to create each output stimuli used in these experiments are described.