Skip to main content
Dryad logo

Data from: Augmented reality powers a cognitive assistant for the blind

Citation

Liu, Yang; Stiles, Noelle R.B.; Meister, Markus; Stiles, Noelle RB (2018), Data from: Augmented reality powers a cognitive assistant for the blind, Dryad, Dataset, https://doi.org/10.5061/dryad.8mb5r88

Abstract

To restore vision for the blind several prosthetic approaches have been explored that convey raw images to the brain. So far these schemes all suffer from a lack of bandwidth. An alternate approach would restore vision at the cognitive level, bypassing the need to convey sensory data. A wearable computer captures video and other data, extracts important scene knowledge, and conveys that to the user in compact form. Here we implement an intuitive user interface for such a device using augmented reality: each object in the environment has a voice and communicates with the user on command. With minimal training this system supports many aspects of visual cognition: obstacle avoidance, scene understanding, formation and recall of spatial memories, navigation. Blind subjects can traverse an unfamiliar multi-story building on their first attempt. To spur further development in this domain we developed an open-source environment for standardized benchmarking of visual assistive devices.

Usage Notes