The estimated reading time for this post is 5 minutes
One of the best uses of technology is helping people overcome challenges. Seeing AI, a free new app from Microsoft, does just that for people who are blind or visually impaired.
Seeing AI combines the phone’s camera and artificial intelligence to identify places, objects and people. The app then announces what it sees so people who are blind or visually impaired will know what is in front of them.
Using the app, people can hear a description of what is in front of them. The app’s interpretation of what it’s seeing is quite sophisticated, it can recognize emotions. It won’t just say that someone is smiling, it will tell you the person is happy. Or surprised. Or angry.
Users can take Seeing AI shopping. The app can read barcodes so people can know whether a can is soup or dog food. It will also soon be able to identify currency so users know whether they are handing over a bill that’s worth $20 versus $1.
Diners at restaurants can use Seeing AI to read menus. The app can read both food choices and prices.
Pedestrians can use the app to read signs and get directions, though the app warns against using it for navigation.
One of the more impressive features of Seeing AI is a document scanner. The app can read documents aloud and even help the user center the document on the screen.
The app looks for the edges of the document so it knows that it’s reading the entire text. After all, a user could get into trouble if the word “not” were cut off in the margins.
The app can also identify images in other apps. When you are using another app, tap the Share icon and choose “Recognize with Seeing AI.” The app can identify what is in the images as if it were a scene in front of you.
This video from Microsoft shows what Seeing AI can do:
How to Use Seeing AI
Seeing AI is divided into Channels:
- Short Text
- Document
- Product
- Person
After you download the app, screens will walk you through how to use it. There are video tutorials for each channel.
With the Document and Person channels, the app takes a picture of the object or person, then interprets it and announces what it is.
The Short Text channel can be used to read things such as signs and envelopes that have a few words that will fit on one screen. Hold the camera over the text and the app will start reading out loud automatically.
The Documents channel is for longer, printed pages. The app will instruct you how to center the camera so it can read all the text.
Try to put the document on a surface with a contrasting color, such as a white sheet of paper on a dark surface. This helps the app recognize the edges of the paper.
The Product channel reads barcodes to help users learn what the product is. The app may also provide more information about the product, such as cooking instructions.
Seeing AI will guide users with beeps to let them know when the camera is getting closer to, and then honing in on, the barcode.
When the app scans the barcode, it will read the name of the product. If more information is available about the product, a More Information button will appear in the app and be announced.
The Person channel helps users identify people, who they are and what they’re doing. The app can interpret a photo of multiple people.
Users can teach the app names of people so it can announce who they are by name.
The app will help guide users while taking a photo of people so that their faces are centered in the photo.
This channel also helps users understand the mood of people. The app will tell how the person is feeling, based on their facial expression. The app will also guess the person’s gender and age.
=> See, How Old Are You? Let Microsoft Guess!
Experimental Channels
The app also has an experimental channel called Scene. With that channel, the app guesses what is happening in the scene.
For example, in the video above, the app guesses (correctly) that the scene is of a girl throwing a Frisbee in a park.
Microsoft also has a Currency channel coming soon to the app that identifies denominations of bills.
What Else You Need to Know
Seeing AI is a new app and will not be 100% accurate. For example, just because someone is smiling doesn’t necessarily mean they are happy.
The app recognizes text but not handwriting yet. That feature may be in the future.
You need to be connected to the Internet for the app to work.
For more information about Seeing AI, including the instructional videos, check out Microsoft’s Seeing AI website.
Availability
Seeing AI is a free app available for iPhone and iPad in the iTunes App Store. Microsoft has not yet announced whether it will be available on Android in the future.
The app is available in U.S., Canada, India, Hong Kong, New Zealand, Singapore, with more countries promised soon.
Your Thoughts
Do you like the idea of Seeing AI to help blind and visually impaired people identify what is in front of them? How do you think Seeing AI will be helpful to others? What would you like to see the app be able to do?
Share your thoughts in the Comments section below!
Pennsylvania Conference for Women
The Pennsylvania Conference for Women just announced that Michelle Obama, former First Lady of the United States, will participate in a conversation at the 14th annual Conference on Thursday, October 3rd, 2017. More than 10,000 are expected to attend the event at the Pennsylvania Convention Center in Philadelphia.
Previously announced keynote speakers include Dr. Brené Brown, courage and vulnerability researcher, #1 New York Times best-selling author and one of the top five most viewed TED speakers in the world; Shawn Achor, New York Times bestselling author of The Happiness Advantage; and Carla Harris, Wall Street executive and author.
For more information, head to the Pennsylvania Conference website
Voice of America Learning English
Want to learn English while reading about tech? Check out my personal technology column at Voice of America Learning English. This week’s article is: Why You Should Avoid Public USB Charging Areas.
Wow! This is awesome! I can’t begin to imagine the challenges that the visually impaired face on a daily basis. Hopefully this app helps to keep folks safe and better aware. As amazing as it is, I’d think that they’d be able to recognize edges better. Might it be a struggle for a blind person to find a contrasting background? Nonetheless, Great article and I’ll be forwarding it to some people I know. Thanks!
Hi Dave, Good point about the contrasting background. Perhaps a sighted person could help guide the blind person to a table with a dark surface. Then the person would know in the future that is a good surface to use.
Wow, science is really helping others. AI is the future.
Hi Hamza, Yes, this is an exciting development for those who have vision challenges. This app opens up a whole new world for many people and could truly change lives.
The Pennsylvania convention for women simply announced that Michelle Obama, former First woman of america, will participate in a conversation on the 14th annual conference on Thursday, October 3rd, 2017. greater than 10,000 are expected to wait the event on the Pennsylvania conference middle in Philadelphia.
HI Micheal, Yes, the Pennsylvania Conference for Women features Michelle Obama, former First Lady of the US, as its keynote speaker. The Conference sold out in hours. I hope you got your ticket quickly!