Facebook Can Now Find Your Face, Even When It's Not Tagged

A new Facebook tool deploys facial recognition to identify users in photos, even when they're not tagged.
Image may contain Human Person Shelf Clothing and Apparel
Facebook's new tool is the latest expansion of its facial-recognition technology.Tom Kelley Archive

Facebook just loosened the leash a little on its facial-recognition algorithms. Starting Tuesday, any time someone uploads a photo that includes what Facebook thinks is your face, you’ll be notified even if you weren’t tagged.

The new feature rolled out to most of Facebook’s more than 2 billion global users this morning. It applies only to newly posted photos, and only those with privacy settings that make an image visible to you. Facebook users in Canada and the European Union are excluded. The social network doesn’t use facial-recognition technology in those regions, due to wariness from privacy regulators.

Facebook has steadily expanded its use of facial recognition over the years. The company first offered the technology to users in late 2010, with a feature that suggests people to tag in photos. Backlash against the way users were automatically opted into that system is one reason Facebook’s algorithms are face blind in Canada and the EU today. Elsewhere, the company made new efforts to notify users, but left the feature essentially unchanged. In 2015, the company launched a photo-organization app called Moments that uses facial recognition to help you share photos with people in your snaps.

Facebook’s head of privacy, Rob Sherman, positions the new photo-notification feature as giving people more control over their image online. “We’ve thought about this as a really empowering feature,” he says. “There may be photos that exist that you don’t know about.” Informing you of their existence is also good for Facebook: more notifications flying around means more activity from users and more ad impressions. More people tagging themselves in photos adds more data to Facebook’s cache, helping to power the lucrative ad-targeting business that keeps the company afloat.

Once Facebook identifies you in a photo, it will display a notification that leads to a new Photo Review dialog. There you can choose to tag yourself in the image, message the user who posted an image, inform Facebook that the face isn’t you, or report an image for breaching the site’s rules.

Facebook

As part of the new feature, Facebook will also notify users if someone else attempts to use their photo in a profile; Facebook says it’s trying to make it harder to impersonate other people. The company is also adding facial recognition to its service for visually impaired people that describes photos from friends in text.

How good is Facebook’s facial-recognition technology? Among the best in the world. The hundreds of billions of photos stored on the company’s servers provide ample data to train machine-learning algorithms to distinguish different faces. Nipun Mathur, of Facebook’s applied-machine-learning group declines to provide any figures on the system’s accuracy. He said the system works even if it doesn’t have a full view of your face, although it can’t recognize people in 90-degree profile. In 2015, Facebook’s AI research group published a paper on a system that could recognize people even when their faces are not visible, using other cues such as clothing or body shape. Facebook says nothing from that work is in the new product.

If you don’t like the sound of all that, you may want to take advantage of a revamped privacy control Facebook also launched Tuesday. You could already opt out of Facebook’s facial-recognition-powered photo tag suggestions, but the setting’s description delicately avoided using the term facial recognition. A new version of the setting that allows you to turn off facial recognition altogether does use the phrase, perhaps making it easier for people to understand what they’re already allowing. If you opt-out of facial recognition, Facebook says it will delete the face template used to find you in photos.

Some privacy advocates say the system should require users to opt in, rather than force them to opt out. In 2015, nine organizations walked out of a Department of Commerce process intended to develop a code of conduct for commercial use of facial recognition, including at social-media companies. Jennifer Lynch, a senior staff attorney with the Electronic Frontier Foundation, says corporate refusals to make their technology opt-in was one reason she and others abandoned the process.

Lynch argues that Facebook’s current policy prevents people from being able to make decisions about privacy and risks to their personal data. The company can instantly and silently roll out sweeping new uses for face data that affect over a billion people.

Lynch says there’s a lot of interest from retailers in using face recognition to track and target shoppers in stores, an area of business Facebook might conceivably be tempted by. A recently disclosed patent application envisions Facebook deploying face recognition for in-store payments. The social network already works with data brokers to link Facebook users’ online activity and profiles with offline behavior.

A Facebook spokesman said the company has no plans for facial-recognition products beyond the one announced Tuesday, and that the company often patents ideas never put into practice. He didn't answer a query about why Facebook didn't allow users to opt in to facial recognition.

Facebook’s stance on that may be tested in court before long. The company is fighting a suit in federal court brought by a user who says the company’s opt-out approach to facial recognition breaches an Illinois privacy law.