Harvard students made Meta Ray-Bans do facial recognition. Meta execs once thought this was a good idea.

• Harvard students rigged their Meta Ray-Bans to do facial recognition, 404 Media reports.
• Meta’s smart glasses can’t do this by default, but in 2021, executives considered the idea.
• Andrew Bosworth, now Meta’s CTO, suggested it could help remember someone’s name at a party.

The idea of smart glasses that can dox people in real time — long a futuristic concept — is getting closer to reality.

404 Media reported on two Harvard students who hacked Meta Ray-Ban glasses to do just this. AnhPhu Nguyen and Caine Ardayfio created software that would scan a video stream from the smart glasses for faces and then identify them using the facial recognition tool Pimeyes. Upon finding a face match, the software would track down other personal information like addresses and jobs. In a demo video, the students showed examples of finding a stranger on the subway and identifying where that person worked.

It’s important to note that the facial recognition wasn’t executed on the glasses themselves. Meta Ray-bans can stream video to your Instagram account or to a device; the students fed that video into the software. All the rest — the facial recognition and retrieval of identifying details — was completed by a program on a laptop that worked separately from anything Meta had done.

Nguyen told Business Insider that the experiment wasn’t meant to show that popular Meta Ray-Bans are a problem.

“We do not want this to be a criticism of their product at all, and we just had them on hand —this could have been done on a phone camera,” he said. “This is mainly a demonstration of what’s possible today.”

Another goal of the experiment, he said, was to help spread awareness on how to protect yourself digitally against facial recognition software.

This isn’t the first time the two students have played around with a dangerous homemade technology: they also made a flamethrower that singed off Ardayfio’s leg hair.

Meta’s glasses can’t recognize faces by default

A spokesperson for Meta told Business Insider, “To be clear, Ray-Ban Meta glasses do not have facial recognition technology. From what we can see, these students are simply using publicly available facial recognition software on a computer that would work with photos taken on any camera, phone, or recording device.”

Glasses are more discrete than most recording devices, however, and hands-free. (Meta Ray-Bans feature a small light that glows while recording. The light was easy to cover with tape in the first model, but the newer version prevents people from disabling it.)

404 Media pointed out that other tech companies like Google have experimented with the idea of putting facial recognition in smart glasses, and then tossed the idea because it was too dangerous.

Business Insider’s Peter Kafka tested out Meta’s new Orion smart glasses prototype and found that it could do some really mind-blowing things that showed immense promise for the future of AR/VR wearables. But one thing that Orion does not do? Facial recognition.

It’s important to note, however, that Meta executives have considered adding the tech to its smart glasses.

Meta considered adding facial recognition

In early 2021, BuzzFeed News reported that Andrew Bosworth, now Meta’s CTO and head of their Reality Labs department, had said at a companywide meeting that Meta was weighing whether it had the legal capacity to offer the software in its smart glasses.

“Face recognition … might be the thorniest issue, where the benefits are so clear, and the risks are so clear, and we don’t know where to balance those things,” Bosworth said in response to an employee question, according to BuzzFeed. He said one benefit could help you, or anyone with face blindness, remember someone’s name at a dinner party.

At the time, Bosworth followed up with a (now deleted) tweet saying, “Face recognition is a hugely controversial topic and for good reason and I was speaking about was how we are going to have to have a very public discussion about the pros and cons.” Meta Ray-Ban smart glasses debuted a few months later in the fall of 2021.

It’s now been three years since Meta executives weighed the possibilities of facial recognition in smart glasses — and no such feature has yet been added. And it isn’t because it’s too much of a technical challenge (hey, a couple of college kids can do it!).

It would appear that Meta has decided that either consumers find it too creepy or that doing it is too legally thorny. Illinois, Texas, and the European Union have laws against certain uses of facial recognition technology.

Legal issues aside, consumer attitudes change. When the Meta Ray-Bans first launched, many people were freaked out by the idea of camera glasses. But over the last few years, the glasses have become a hit product. Meta has sold more than 700,000 pairs since last October, according to the market intelligence firm IDC. It’s possible that at some point in the future, facial recognition in glasses will seem like a welcome convenience rather than a privacy nightmare.