Wait, why is there a camera hole in that Coke machine?

By | March 30, 2024

If you’ve recently visited a fast-food restaurant like Wendy’s or Burger King or a movie theater like AMC, chances are you’ve grabbed a drink from a modern-looking touchscreen soda fountain.

The machine in question, called the Coca-Cola Freestyle, debuted in 2009 and lets customers choose from more than 100 drinks and flavors — from the traditional Coke or Sprite to fringe favorites like flavored ginger ales. Although carbonation snobs have been known to complain that these soda fountains allow remnants of one drink to seep into the next glass, the Freestyle machines are so popular that there are more than 50,000 of them in public use.

And yet, over time, the occasional mystery has surfaced about the machines… If you look closely, why do many of them have a seemingly small camera lens embedded above the touch screen?

This question started with a tip passed along to this reporter: a salesperson from a Big Tech company had bragged that Coca-Cola was capturing images of soda fountain customers via cameras, then working with said cloud computing giant to gain insight into facial reactions when they blend certain flavors together to create a custom drink. With that data, the thinking went, Coca-Cola could gain new insights into new flavors it should consider producing.

This is according to a source who worked for Coca-Cola for years Fortune they had heard a similar rumor but could not confirm it with certainty. A few Reddit posts also pondered the camera hole issue, with mostly unsatisfactory answers. And a Coca-Cola spec sheet found online for one of the Freestyle models lists a built-in camera as one of the soda fountain components, along with “future motion detection and facial recognition capabilities.” Hmm.

While a camera embedded in a soft drink machine may sound absurd, companies and event venues are increasingly using new facial analysis and recognition technologies in public environments. MSG Entertainment, owner of the Madison Square Garden sports arena and other venues such as Radio City Music Hall, recently used facial recognition technology to block lawyers working for companies in legal battles with the company from attending events at its New York venues live. . More airports are also adding it at security checkpoints. And as recently as February this year, students at the University of Waterloo pressured the administration to remove vending machines that secretly incorporated facial analysis technology.

What does Coca-Cola have to say about this?

A company spokesperson, Scott Leith, said in a statement: “Camera capabilities were included in previous designs and tested in a laboratory environment in 2018 and 2019. The company has no plans to use cameras in the future.”

Leith did not elaborate on whether machines in the field still contain cameras, nor the exact nature of the tests. But a patent application filed in the summer of 2018 by two former Coca-Cola employees could provide a snapshot of the intent, saying the system “could potentially determine consumer sentiment.”

“For example, whether a consumer who appears angry appears happier after being served a drink and/or whether the consumer enjoyed the drink,” it says. “Such data can also be used to identify new mixes preferred by consumers… So entirely new drink combinations can be created and promoted.” This reasoning is very similar to the original tip this reporter received.

The application also mentioned the potential of a soda machine to identify demographic information about a customer “for marketing purposes and for an improved consumer experience.”

“The method may include the steps of observing physical characteristics of the consumer, matching those physical characteristics of the consumer with demographic characteristics, promoting a beverage choice to the consumer based on the matching demographic characteristics, and offering a drink to the consumer,” said the councilor. reading application.

But there is something more. Last month, a case study on the website of an AI company called Quantiphi referenced work on a facial recognition project related to the Coca-Cola Freestyle machine.

“Each vending machine comes with a camera installed, which captures an image of each customer interacting with the vending machine,” according to the case study on Quantiphi’s website. “Coca Cola’s marketing team wanted to use these images to understand consumer preferences and usage patterns,” including “analyzing the types of ‘mocktails’ preferred by their consumers.”

The mocktails in question refer to the moment when a Freestyle soda fountain customer creates their own drink by mixing two or more flavors available from the machine.

The case study also said Quantiphi “built a custom machine learning model, trained on 8,000 to 10,000 images, capable of detecting customer demographics… using facial recognition. As a result, Coca-Cola’s marketing team is able to capture and evaluate customer demographics, which are then used to develop targeted marketing strategies and launch potential individual products, driving growth.”

A spokesperson for Quantiphi, Hadley Mayes, said the collaboration took place in 2017, but it was just an experiment. Quantiphi later removed the case study from its website Fortune inquired about it and said it did not reflect the company’s current capabilities. Mayes declined to provide more details about the initiative, such as where it took place and whether it was in a public setting, citing a non-disclosure agreement.

Today, thousands of Coca-Cola Freestyle machines are still on public display with a clear slot for a camera and some questions about how the company has used the cameras to date. If you have any information, this reporter would love to know more.

And with that, here’s what else is happening in tech news today.

Jason DelRey

Would you like to send your thoughts or suggestions to Data Sheet? Draw a line here.

The rest of today’s data sheet was written by Alexei Oreskovic.

This story originally appeared on Fortune.com

Leave a Reply

Your email address will not be published. Required fields are marked *