AI, Apps

Facebook iOS App To Use AI And VoiceOver To Assist Blind Users


The Facebook iOS application comes with a very different and thoughtful update which includes AI and VoiceOver, one of the accessibility features built into iOS, to describe and narrate the contents of posts to blind users. A high percentage of posts on the social website consist of pictures with a brief caption,  therefore, this feature will definitely improve the experience for the blind users.

The Facebook iOS application, with the help of Artificial Intelligence, recognizes and identifies the contents of the picture which are then read to the user by Apple’s VoiceOver thus explaining to the user the entire scene of the picture.  The people at Facebook have named this feature ‘automatic alt text.’

“Automatic alt text, which is coming to iOS today and later to Android and the web, recognizes objects in photos using machine learning […] While still in its early stages, the technology can reliably identify concepts in categories including transportation (“car,” “boat,” “airplane”), nature (“snow,” “ocean,” “sunset”), sports (“basketball court”), and food (“sushi”). The technology can also describe people (“baby,” “smiling,” beard”), and identify a selfie.” -The Verge

According to Facebook, there were two solutions to address this problem and they chose the one they thought was befitting

“Matt King, a Facebook engineer who is blind, […] said “We could probably require people when they upload a photo: ‘please describe this for blind people.’ It would drive people nuts — that would never work at scale.” (This is the actual approach Twitter is taking to the problem, though adding descriptions is optional.).”

Keeping in view the likelihood of misidentification, the algorithm only tags photos with minimum 80% certainty level of the keywords. The social network company also plans to improve the  AI with time so that it can facilitate more blind users in the future.

“The impact of doing something like this is really telling people who are blind, your ability to participate in the social conversation that’s going on around the world is really important to us. It’s saying as a person, you matter, and we care about you. We want to include everybody — and we’ll do what it takes to include everybody.”

Previous ArticleNext Article