iDentifi: Object Recognition for Visually Impaired
Apps used to recognize objects and/or read text for people who are blind and visually impaired have increased in number. We have discussed TapTapSee recently, an others exist as well.
This post details iDentifi, a new free app that attempts to describe objects and read text for people who are blind and visually impaired.
What is iDentifi?
Anmol Tuckrel, a high school student from Toronto, Canada, began work on the app about a year ago. According to a TechCrunch article, Tuckrel was fascinated by the possibilities of machine learning and computer vision. The app uses Google Vision, CloudSight and Google Translate, all trusted resources that can distinguish objects easily. These facts indicate that iDentifi uses artificial intelligence to identify objects whereas apps like TapTapSee use crowdsourcing.
Using the App
Before attempting to use the app, please note that you must be connected to the internet to use it. The app’s layout is quite easy to comprehend. Its initial screen contains four buttons, one in each corner of the screen--“Settings” in the top left, “Instructions” in the top right, “Select photo” in the bottom left and “Take photo” in the bottom right. Of course, if you flick left and right, you will locate the same buttons in the same order. Knowing their location, however, allows you to find the button you want without extra flicks or swipes.
Each button and the area surrounding it is brightly colored with a different color included for each button or area of the screen. As a result, people with low vision can distinguish the buttons easily, and individuals who use both VoiceOver and their remaining vision benefit since the app’s functionality is excellent in both cases.
If you press the “Settings” button, you first choose the language for all interactions with the app from the list of over 25 languages. Next is the mode button where you choose from “Images low accuracy”, “Images high accuracy”, or text. The low accuracy mode provides a general description of the picture you take and returns the quickest response. The high accuracy mode gives you a more detailed description of the image and requires more time for receiving a response. In text mode, the app tries to read the text from the image you’ve taken.
The final setting is speaking rate—how fast you want the app to speak to you when it reads its results; the settings are very slow, slow, normal, fast, and very fast.
The instructions describe some of the app’s functionality and tell you the location of important buttons on the app. The instructions do not stay on the screen, but if you need to hear them again, double tap the instructions button a second time.
Selecting a photo sends that photo to the app; iDentifi then tries to determine what is in the photo. You must allow iDentifi to access your photos and also the camera. Once you hit the select photo button, you see the standard camera interface that you would use to send a photo to Facebook, include one in a message, etc.
When you double tap this button, you see a screen that mimics the standard iPhone camera screen with buttons for flash, viewfinder, camera mode, camera chooser, take picture, and cancel. If you are satisfied with the camera settings, double tap the “Take picture” button, located just above the iPhone’s home button. You will hear a sound as the phone takes the picture. You then can select “Retake” or “Use photo”, found on the bottom left and bottom right of the screen respectively. If you have usable vision and believe that your picture is not satisfactory or if you just want to use a different picture, select the retake button and start the process over.
If you tap on use photo, you hear the app say, “Loading”. At this point, the picture runs through the app for identification purposes. You can retake a picture as many times as you like, but you must hit the use photo button for the app to begin the identification process. All photos you take using this app are not saved. The identifications given by the app are not able to be reread and do not remain on the screen, but you can try the three-finger quadruple tap gesture to put the response on the clipboard and add it to a message, email, etc.
Currently iDentifi is available on the iOS platform only; the developer plans to create an Android version in the future. Because the results of the picture recognitions are not shown on the screen, individuals who are deafblind and anyone using a braille display may have problems accessing the results. The app will read text and does a good job doing so. It will not replace an OCR app like KNFB Reader, though, especially if you store files for later reading. If you don’t need to store the file or go back and read it multiple times, iDentifi will work well.
The developer hopes to increase the available languages to close to 100 and wants the app to work in video mode. He appears to be responsive and open to suggestions so send them and help improve the app.
Finally, the only other limitation, as is the case with all camera apps, is the ability of each person to take a suitable picture. Fortunately, you do not have to have the camera perfectly centered to take a usable picture.
The iDentifi app is an excellent choice for anyone who is blind and visually impaired. It identifies objects quite well and reads text reasonably well also. Remember to turn the mode to text if you want the app to read text; otherwise, it will simply tell you there is text without reading it. The results with the mode set to high accuracy are very good; its descriptions of objects and their colors are quite helpful. You may find that this app also doubles as a color identifier, at least for basic colors. Would you like to see the app in action? Watch this short video and view another one included in the TechCrunch article written about the app. For more information about iDentifi, visit the website at http://getidentifi.com/#home-section. The site discusses the numerous awards and the press coverage the app has received and tells you how to get support or make comments about the app. Get the app at the following link or search for iDentifi on the app store; remember that the d is the only capitalized letter.