Google Lookout App to aid the Visually Challenged


Google is coming up with an app that will enable visually challenged auditory cues as they encounter objects, text, and people around them. In March, Microsoft made a similar effort by releasing Soundscape, an app that uses auditory signals and labels to help visually impaired people navigate cities.


Home, Work & Play, Scan, and Experimental are the four modes available in the Lookout app. Depending on the mode, the app evaluates important objects in the surroundings and then shares what it deems to be relevant information. The information may include a recipe book, the location of a bathroom, an exit sign, a chair, or a person nearby, according to Google. The software also includes voiced notifications, which are said to be designed for minimal involvement and allow users to keep focused on their current duties and activities.

If a user selects the Home mode, the Lookout app will provide notifications about the location of the couch, table or dishwasher. Similarly, the Work & Play mode warns users when they're next to an elevator or stairwell, while the Scan mode is designed to read text from a notebook. There is also the Experimental model that allows users to get early access to the features that are in the pipeline.

Instead of navigating in a traditional manner, the app gives an idea of where the objects are located in an easy way. For instance, it offers "couch 3 o'clock" to let users understand that the couch is on their right.

Users are recommended to use Lookout with their device worn in a shirt pocket or hanging on a lanyard around their neck, with the camera sensor pointing away from the body. This enables the app to scan the surroundings. The app also uses machine learning to learn what users want to hear about and then accordingly deliver more appropriate results. Moreover, the core experience on Lookout is processed on the device to offer its features without requiring an Internet connection. "Accessibility will be an ongoing priority for us, and Lookout is one step in helping blind or visually impaired people gain more independence by understanding their physical surroundings," said Patrick Clary, Product Manager for Google's Central Accessibility Team, in a blog post.

Similar to Lookout, Microsoft's Soundscape also offers a bunch of features for visually challenged people. The app has the ability to call out key points of interest (including roads and intersections), place an audio beacon on a point of interest, describe the current location, and detail nearby points. However, Soundscape is so far exclusive to iOS devices, whereas Google's Lookout will be available for the Android ecosystem.