Sighted People Phetch Descriptions of Graphics for the Blind

Screen readers, such as JAWS for Windows and Window Eyes are the assistive technology solutions commonly used by blind people to listen to web page content. This assistive technology uses a synthesized voice. However, screen readers are unequipped to deal with pictures without detailed captions. Therefore, pictures on most websites remain inaccessible to visually impaired people.

The online game named "Phetch" is designed to encourage normal web users to generate missing captions for pictures.

The game is played in groups of three to five people. One of the players is the "describer," who has to write a short paragraph on a randomly chosen web image given to him. The others are "seekers", who use the description given by the describer to find the correct picture on the web with the help of search engines. The first one among the seekers to find the image becomes the describer in the next round.

The descriptions given by the describers that are good enough to lead to the particular pictures will be stored as captions for respective images. The failed attempts will be discarded.

Click this link to visit the Phetch website to start playing: http://www.peekaboom.org/phetch/.

Google also has an Image Labeler game. The format is brilliantly simple: players are assigned a random (unknown) partner, with whom they need to collaborate to agree on tags to describe a particular image. It's a great way to collect metadata about an image-descriptive tags because you've got an ideal situation: a double blind test.

Fastr is another game where players assist in the creation of image tags. This game uses flickr images. It loads ten images that all share a common tag, one by one, and you guess what the tag is. When you guess right, the tag will turn blue, and you'll get points. The faster you guess, the more points you get. The points are reset every six minutes. You'll need to choose a player name before you can play. You can play using tags for multiple languages and for specific flickr groups.

Flickr is a photo sharing site where people can choose to make their photos either public or private (but more than 80 percent of the 5.5 million photos are public.) Pictures can be annotated, have captions added within the frame, or comments can be posted below other user photos. Most significantly, people can append tags to a specific category (such as "dog," "poodle" or "cute"). Then all of the photos that share a particular tag can themselves be explored as a group, which online pundits call a "folksonomy."

The result is something that approaches a vast theater of collective performance art. Various groups on Flickr are devoted to photo collections of confusing street signs, mannequins and Halloween costumes. There's a group devoted to pictures of dogs, naturally, but there's also a group devoted to pictures of dogs' noses.

The ESP Game has been running since 2003 and was created by Carnegie Mellon. Computer programs can't yet determine the contents of arbitrary images, but the ESP game provides a novel method of labeling them: players get to have fun as they help determine their contents. If the ESP game is played as much as other popular online games, it is estimated that all the images on the Web can be labeled in a matter of weeks!

Having proper labels associated to each image on the Internet would allow for very accurate image search, would improve the accessibility of the Web (by providing word descriptions of all images to visually impaired individuals), and would help users block inappropriate (e.g., pornographic) images from their computers.