Chick Engine Helps You To Find That Dress You Saw At Pinterest

By Jonathan Allen, at ClickZ:  
What does breast cancer screening have to do with fashion? More than rubber bracelets or ribbon broaches.
Adrian Rosebrock, from Catonsville, Maryland, has put into action insights from his day job as a developer at the National Cancer Institute unit to make a visual search engine.
Working in the breast cancer screening unit, Rosebrock has been developing metrics to detect cancer in images and taking those learnings about computer 'vision', namely histology, and applied it to the problem of shape and color in visual search in the fashion vertical.

His project, Chic Engine, matches the shape and color of any image led query you input, either via a image file upload or a hosted image URL – provided you are looking for clothing matches. Currently the index of returned products comes mainly from ShopStyle but what is available so far is an impressive demonstration of how visual search could be something to look out for.

 

Barriers to Entry

Personally speaking, I have been skeptical towards the possibilities of visual search for awhile now because, like we once said of mobile, it's one of those technologies that seems to stall just before really "taking off" with users.

The main barriers to image search is that in general, it's just not that intuitive to users. Uploading an image to search for similar matches is not only more effort than typing keywords in a query box, it is also something that the majority of web users are not even able to do.
Although, dear reader, it may be a simple enough thing for you to do, the average web user is limited by the ability to understand how their own computer works, let alone the web. Furthermore, the ostensibly easier feature for power searchers –namely to search against the URL of a hosted image – is near impossible for most people to get their head around without a rather technical demonstration.

With that in mind, when Rosebrock pitched SEW the story, I asked him what his fashion search engine could do with data from the latest dazzling debutant of social media, Pinterest. The next day I got a reply. Chic Engine could now match products pinned on Pinterest.

 

The Challenges of Computer Vision

In a phone conversation, Rosebrock told Search Engine Watch that there was an interesting technology challenge at the heart of image search – namely what it is possible for a computer to see and what is scaleable for a computer to return. The more accuracy required to find matches the lesser the ability of the search engine to find them.

As Superfish also found, the sweet spot seems to be somewhere between discovering what is just identical and what is actually similar and, just like Superfish, and any "computer vision" technology, Chic Engine has struggled with 'background noise' and has to focus on the foreground of the image.

For Rosebrock, the solution comes down to three tenets for designing a good visual search engine:

• Accuracy
• Purpose 
• Scalability

For visual search to be accurate, the user expects to find the image they are looking for in the database. Nothing new here.
However, for visual search to be useful, it has to go beyond finding identical matches and expand the users visual vocabulary – and the kicker here is that every new result is possibly a new query. Now, that is pretty interesting.

Imagine a search engine that enables you to delve into an infinite chain of queries to target exactly what you want and then think about how different that is to the incumbent offerings.
Unlike Superfish, Chic Engine runs a much more limited results set, which solves the scaleability problem such that it does not need any "special sauce" to identify the difference between a flat washer and an engagement ring. However, that doesn't make it any less impressive as what Rosebrock seems to have solved is the problem of even wanting to do a visual search query in the first place.

 

Snap & Grab

Just how easy Chic Engine is to use should be a win to users with fashion sense. What makes it stand out is the ability to select a certain part of the image and search only on that portion of the image.
Particularly impressive is Chic Engine's ability to match subtle color tones and hues. The image cropping mechanism also helps to reduce the problem of background noise as the user selection serves as a "hint" as to where the clothing is – Chic Engine then automatically detects the clothing match.



Just like the majority of web users cannot find an URL, let alone copy/paste it, so the majority of people are not blessed with the ability to truly articulate the style, cut, texture and color of a piece of clothing they like.

Read the full story here