Automatic image tagging: Why this could be a killer app

One of the dream media applications that I discussed at a presentation at Boston City Hall earlier this year was automatic "tagging" of images. If you're not familiar with this term, it means applying text descriptions to a Web page, computer file, image, or video, in order to make it easy to categorize and/or find on a computer. Google, Flickr, and many other sites depend on tags to categorize content, and make it easily locatable for users, but one limitation is that tagging generally has to be performed by humans. That is, Flickr won't know that an uploaded picture of a skyscraper is a building unless someone (usually the person who uploaded it) "tags" it as building.

In my presentation, I predicted an application that can auto-tag images uploaded to the 'Net. There is certainly a need for such an application, because in ten years' time most people in developed countries will be carrying around a cameraphone or similar device with high-speed connectivity, and constantly uploading pictures and video to the 'Net. Anytime there is a news event, there is a good chance that there will be a witness -- or potentially hundreds or even thousands of witnesses -- taking pictures and sharing them on the Internet. If there is a system for automatically tagging the photos, instead of waiting for humans to do it, personal news applications can quickly pull together image galleries, text feeds, and other online documentation to create rich reports on various events soon after they happen. GPS-capable cameras will help, too. These technologies will revolutionize the media business, and lead to a much different understanding of world events -- ordinary people will be able to bypass the traditional media gatekeepers (newspapers, governments, television stations) that have traditionally filtered the images and reporting coming from news events.

Anyway, that's the gist of what I said in the presentation. But it turns out that two researchers at Penn State have developed a tool that can autotag images. It's a little rough around the edges, but it is quite impressive. The Automatic Linguistic Indexing of Pictures project was developed by Jia Li and James Wang, and includes a Web interface which lets you try the autotagging function. The project and the research by Li and Wang is described here.

Roland Piquepaille discusses the autotagging technology and has a good visual example of how it works. There is also a slashdot discussion of the technology.