Pinterest today is announcing an update to its browser button for Google’s Chrome browser that should make it easier to save interesting content that users find on websites.
Initially when Pinterest incorporated its visual search technology for finding pins of visually similar things into its Chrome extension, the system inadvertently ended up capturing screenshots of entire browser windows, instead of one little thing in it that a user might be interested in. The extension would then show pins of browser window screenshots that had previously been saved to Pinterest so that users could add them to their boards.
Clearly, that’s not how the system should work. So Pinterest engineers went to work building a system that radically refined the area to recommend. The result is something called instant image feature detection. You can try it out by right-clicking on a blank area of a webpage and hitting “Search” next to the Pinterest logo. The resulting page will automatically highlight an interesting thing that’s visible in the current window — you can move the selector around if you wish — and then show you pins that are visually similar.
“The net effect is people are getting much better results when they’re looking at screenshots the first time it comes up, and they’re not having to do select box, loop it around,” Pinterest engineer Kent Brewster told VentureBeat in an interview.
As a result of the change, the number of repins has gone up considerably, Brewster said. “The anecdotal results we got back was, ‘Oh my God, this is so much more useful,'” he said.
Pinterest’s visual search technology was recently incorporated into Samsung’s Galaxy S8 smartphone. Meanwhile Amazon has put a “visual search” feature into the S8 as well, but its function is to spot products and then add them to your shopping cart. And last month Google was seen testing visual search in the Google App. But the data sets are different from Pinterest’s collection of pins.
Pinterest brought visual search to its core app last year and has since launched the Lens feature, which uses the technology to spot pins of objects that are captured with a smartphone or tablet’s camera. The underlying object recognition system draws on deep learning, a type of artificial intelligence that involves training artificial neural networks on lots of data and then getting them to make predictions about new data.
Brewster’s Medium post has details on the technology the company built for homing in on specific regions of a browser window screenshot.
