SensoryScape: a modular system for multimodal navigation assistance on mobile devices

For people who are blind or visually impaired, navigating an unfamiliar environment can be dangerous. Stairs, curbs, and sudden elevation changes are difficult to detect in time to avoid a fall, and existing assistive tools like white canes, guide dogs, and electronic devices each have real limitations in coverage, cost, or social acceptance. This thesis presents SensoryScape, a navigation assistance system that runs on an iPhone and Apple Watch. Rather than relying on a single mode of alert, SensoryScape combines directional sound cues, wrist vibrations, and voice interaction to give users a richer, more dynamic sense of their surroundings in real time. The system is designed to complement, not replace, the mobility aids people already rely on. A key goal of SensoryScape was accessibility in the broadest sense, including affordability. By building on consumer hardware most people already own, it avoids the cost and stigma often associated with specialized assistive devices. Users can also customize the system extensively and switch between different environment settings without technical expertise. Testing of the staircase and fall hazards detection capability across 120 trials showed the system correctly identified hazards 92.5% of the time, with no false alarms, giving users an average warning distance of about two and a half feet. Detection was most reliable for taller steps and when approaching from below. Detecting fall hazards from above proved more challenging due to the physics of how the depth sensor of phones work.