Journal article
EchoSee: An Assistive Mobile Application for Real-Time 3D Environment Reconstruction and Sonification Supporting Enhanced Navigation for People with Vision Impairments
Bioengineering (Basel), Vol.11(8), 831
08/14/2024
DOI: 10.3390/bioengineering11080831
PMCID: PMC11351581
PMID: 39199789
Abstract
Improving the quality of life for people with vision impairments has been an important goal in the research and design of assistive devices for several decades. This paper seeks to further that goal by introducing a novel assistive technology platform that leverages real-time 3D spatial audio to promote safe and efficient navigation for people who are blind or visually impaired (PVI). The presented platform, EchoSee, uses modern 3D scanning technology on a mobile device to construct a live, digital 3D map of a user’s environment as they move about their surroundings. Spatialized, virtual audio sources (i.e., virtual speakers) are dynamically placed within the digital 3D scan of the world, providing the navigator with a real-time 3D stereo audio “soundscape.” The digital 3D map, and its resultant soundscape, are continuously updated as the user moves about their environment. The generated soundscape is played back through headphones connected to the navigator’s device. This paper details (1) the underlying technical components and how they were integrated to produce the mobile application that generates a dynamic soundscape on a consumer mobile device, (2) a methodology for analyzing navigation performance with the application, (3) the design and execution of a user study investigating the effectiveness of the presented system, and (4) a discussion of the results of that study along with a proposed future study and possible improvements. Altogether, this paper presents a novel software platform aimed at assisting individuals with vision impairments to navigate and understand spaces safely, efficiently, and independently and the results of a feasibility study analyzing the viability of the approach.
Details
- Title: Subtitle
- EchoSee: An Assistive Mobile Application for Real-Time 3D Environment Reconstruction and Sonification Supporting Enhanced Navigation for People with Vision Impairments
- Creators
- Broderick S. Schwartz - University of IowaSeth King - University of Iowa, Teaching and LearningTyler Bell - University of Iowa
- Resource Type
- Journal article
- Publication Details
- Bioengineering (Basel), Vol.11(8), 831
- Publisher
- MDPI AG
- DOI
- 10.3390/bioengineering11080831
- PMID
- 39199789
- PMCID
- PMC11351581
- ISSN
- 2306-5354
- eISSN
- 2306-5354
- Grant note
- 1I50RX003002 / Department of Veterans Affairs, Veterans Health Administration, Rehabilitation Research and Development Service
- Language
- English
- Date published
- 08/14/2024
- Academic Unit
- Electrical and Computer Engineering; Teaching and Learning; Ophthalmology and Visual Sciences
- Record Identifier
- 9984699051302771
Metrics
4 Record Views