Why doesn't Facial Recognition Necessarily Mean Better Search?

Updated with comment from Nikolaj Nyholm, the Polar Rose CEO, you can see his comment here.

Is Polar Rose just one more Riya? Probably, it can be more successful in terms of technology but it unlikely can make image search in the Web better. It can be cool for finding a photo of someone who looks the same or pretty the same as the person on the original picture (if it works well) but cannot really help if you need to record, share, and search information about any object, and not only people, visualized on the image.

As more and more companies are joining the race for labeling and searching content within images, I think it’s a good time to make a brief overview of these products to understand whether and how they can help us manage images.

I’d like to start with the company that has already initiated a lot of hope around their product. I’m talking about Polar Rose that pre-announced a product release last month. If you still haven’t heard about them, this is what they do in a nutshell.

You surf the Internet with their plugin installed into your browser. When you see a photo, the plugin locates people or something that looks like people and adds an icon over the picture. Then you click the icon and tag the photo telling the program the name of the person. The program builds a 3D model from the photo to eliminate factors that may affect recognition, such as lighting or specific camera’s angle. Your tags are stored on a Polar Rose server so they become a collective memory for millions of other surfers. This collective intelligence is used to recognize tagged people on other photos. When the program finds another photo where the same person is captured, it recognizes the face and automatically offers the name.

Sounds cool and… too familiar. Right, we all have already heard about Riya. They also started with a lot of promises. Now they’re trying to apply their image recognition to find handbags and watches of celebrities. I don’t want to say bad things about Polar Rose, especially they really have chances to be more successful as their technology may allow more opportunities. I’m already seeing very useful applications (if it works, of course) of their technology for collections of group photos where manual tagging can require much time. It could be also great to find a photo of myself (well, my ego makes me do that!) in the Web hoping that someone found me interesting enough to shot and post my picture.

Polar Rose is planning to provide their API to those Web site owners who are interested to use facial recognition technology on their sites. According to Nikolaj Nyholm, the Polar Rose CEO, "results will be culled from across the web, either from our spidering or from the input of users and other partners." I am not sure whether they are going to build a centralized image search engine to become independent on a specific site but would love to get to know that.

Although there is still no product available for public, I'll try to understand how the product will be positioned and predict what issues thet will have to face:

• First, photos are not only about people. I know that Polar Rose or any other company could try to recognize other objects and maybe they could achieve interesting results some day. However, it will definitely take too much time to successfully apply image recognition techniques to inanimated objects. Recently, photos with content other than people are out of scope. If you need to find, for example, a photo of the Women’s Titanic Memorial in Washington, Polar Rose will be unlikely helpful.



• Second, there is something that cannot be covered: your impressions that you want to store together with photos. You could do it with paper photos by writing comments on a backside, but digital photography has changed the rules. You may want more space for your own, not machine-generated comments associated with specific objects (so tags cannot help here as it can be hard to understand to which object exactly the tag relates) that even can concern non-identification issues.

  I think Polar Rose is quite useless here.




• Third, Nikolaj Nyholm, the Polar Rose CEO said that “the core functionality is photo sharing, not photo labeling”. As there is no publicly available information on this, I don't know how exactly sharing will work. Most likely, all identification information will be stored in a database which means data about image content is separated from the image. It doesn't matter until both the image and the database are stored on the same server. However, what happens when the image is sent, say, by email or downloaded to a local computer? Will identification labels with the names of the people be lost?

The bottom line: facial recognition is cool for finding a photo of someone who looks the same or pretty the same as the person on the original picture (if it works well) but cannot really help if you need to record, share, and search information about any object, and not only people, visualized on the image.

Posted on January 8, 2007 by Alex Masycheff

See all posts