Artificial intelligence (AI) has become a dominant force in technology, and it’s encouraging to note that people with vision loss are benefiting. In August 2017, we were delighted to announce, A Gift from Microsoft, the new Seeing AI app, an intelligent camera that translates text and images into spoken words. The app is part of an ongoing research project, founded and led by software engineer, Saqib Shaikh, who also worked on the development of Bing search and the Cortana voice assistant. He well understands the needs of people with low vision or blindness, having lost his own sight at the age of seven.

It was a labor of love for Saqib, and it appeared to be an incredibly magnanimous gesture on the part of Microsoft. Seeing AI was available exclusively for Apple iOS on the App Store, and now it is also on Google Play for Android users. The app was, and remains, free and it continues to develop and improve. 

Seeing AI is now comprised of 10 channels, here is a rundown of the impressive lineup in version 5.2.

• Short Text – If you’ve never used the app, this is a good reason to start. Point the camera at text and it will read it back to you, instantly. Use it to read, just about anything including: all kinds of signs, home and building addresses, mail, labels, tags and other inaccessible text you encounter every day.

• Document – The camera guides you to photograph the entire document, then reads it back. Now, instead of reading the document from start to finish, you can ask Seeing Ai questions and get right to the information you’re looking for. For example, photograph a restaurant menu, then type or dictate a question like, “What are the chicken dishes?” Or photograph a receipt and ask for the price of a specific item.

• Product – A barcode reader with audio guidance helps you find the code, then scans it. In the grocery store, for example, the code might identify the brand and flavor on the ice cream container, or the fat content of the milk, or if the bread is wheat or white.

• Person – Use the camera to scan and tell you about the people nearby. Facial descriptions include an estimate of age and expression. Set face recognition to announce the name of someone you know when they come into view.

• Currency – Select the paper currency you wish to identify (choose from 17 currency options) and hold the camera over a single note to hear the value.

• Scene – Take a photo of a scene and hear a description. Explore the photo by moving your finger around the screen or tap button for More Info.

• World — A newly experimental channel that helps you navigate indoor environments, guided by spatial audio using headphones.

• Color – Point the camera at an item and it will identify the color.

• Handwriting – Take picture of a handwritten note, right side up. Legible handwriting and clear photos deliver best results.

• Light – For users with no light perception, this channel indicates intensity and direction of light through audible tone and pitch changes.

The Seeing AI app is available for iOS in the App Store, although it is not yet on Google Play, Lookout is.

Feedback helps improve the app, so do share your thoughts through the app menu or by email to [email protected]. Get quick help at the upper right corner on every channel.

Post originally published on Jan 19, 2018 and most recently updated Dec 6, 2023