Harvard app Mobilio uses smartphone sensors to guide blind walkers

Navigating beyond standard digital maps

People with low or no vision face frequent hazards when navigating unfamiliar indoor and outdoor spaces. Standard navigation tools such as Google Maps lack the fine spatial resolution required to identify immediate physical barriers, leaving users reliant on white canes or guide dogs. To address this gap, PhD researcher Raymond Liu and assistant professor of bioengineering Patrick Slade at the Harvard John A. Paulson School of Engineering and Applied Sciences designed Mobilio, an assistive innovation reported in New Atlas.

The engineers conducted trials with 14 test subjects who walked through indoor test routes using a cane paired with Mobilio. Their performances were compared against travel using a cane alongside Google Maps. According to the team's research paper in Nature Biomedical Engineering, subjects using Mobilio reached their destinations 13% faster and suffered 41% fewer collisions indoors, confirming the functional benefits of real-time route adjustment.

“Our system uses the sensors in a smartphone, GPS information, and the actual motion of the person to create what’s essentially a small autonomous vehicle plan on how to navigate them from where they are to where they want to go.”

— Patrick Slade, Assistant Professor of Bioengineering at Harvard University

Pedestrian perspectives replace automotive datasets

Before assembling the software prototype, the researchers surveyed more than 100 people with limited or no vision to determine their primary physical challenges. The user group highlighted three core requirements: continuous directional feedback along straight paths, accurate turn-by-turn guidance, and dependable detection of immediate physical obstacles, as documented by New Atlas.

Most commercial autonomous computer-vision models rely on datasets captured from vehicles driving along paved roadways. Because pedestrians navigate sidewalks, crosswalks, and indoor corridors, Liu and Slade built artificial intelligence semantic segmentation models using imagery captured directly from a pedestrian perspective. Liu, whose older brother is blind, emphasized that testing technology with affected users is essential because visual impairments vary widely from person to person.

Autonomous vehicle algorithms run on commodity hardware

Mobilio operates without requiring specialized external headsets or external computation packs. Instead, the application runs entirely on a standard consumer smartphone by processing inputs from the device camera, GPS receiver, internal motion sensors, and onboard LiDAR sensors when available, according to New Atlas.

The software computes safe walking trajectories by tracking user movement and adjusting spatial audio cues to match each individual's walking gait, leveraging principles often seen in robotics. Running localized vision models on standard smartphone chips eliminated the need for specialized external processing boards, keeping the assistive setup accessible to users without extra purchases.

Field testing inside daily routines

Following their initial laboratory evaluations, the Harvard SEAS development team plans to expand testing into varied outdoor environments and public venues. The next research phase will assess how Mobilio adapts to the varying daily habits, pacing, and route preferences of visually impaired users over longer periods of regular use, as detailed by New Atlas, marking a promising step forward for the invention.

Read also on The Patent: Trilateral patent offices meet in Vienna on Global Assignment and AI