Computers can now read emotions — here’s how that will help you sell products

Two years ago, luxury car brand Bentley created an app to solve what must be the epitome of first world problems: helping customers pick the right Bentayga SUV, based on facial emotion analysis.

The Inspirator App, which is no longer available, scanned your facial expressions while showing different videos containing “lifestyle-themed stimuli” to work out which customizable features — wheels, paint, and interior —  best fit your taste.

The app was a gimmicky tool at best; something fun to play with for those who can afford a $300,000 car but in no way a scientifically valid method to match emotion to preference.

Still, technology that can read your emotions is becoming increasingly advanced — and marketers are taking notice. Emotions play an important role in the customer’s decision to make a purchase, so emotional analytics are of great value to marketers and advertisers.

Emotional creatures

“We tend to think of ourselves as rational beings, but many of our decisions are driven by emotions,” says Daan Bolder. He’s managing partner at Braingineers, an emotion analytics company based in Amsterdam. The company combines EEG’s (which record electrical activity in the brain) with eye-tracking technology to analyze how people respond to online content.

“That means we invite participants to our lab to test certain websites or watch video content. An EEG headset measures their brain activity while they complete certain tasks. Simultaneously, eye-tracking software tells us which focus points attract their gaze.”

This data in itself tells little about the user’s emotional state, says Bolder. “The EEG just measures three simple things: joy, attention, and frustration. Yet people experience different types of frustration, depending on the tasks and context.”

To make sense of their participants’ brain activity, Braingineers uses a machine learning algorithm. “By deliberately creating frustrating online experiences  — adding buttons that don’t work, for instance — we can identify the neurological patterns that occur when people get frustrated, which we then use to “train” the algorithm. This helps us to not just identify emotional bottlenecks during a user’s experience, but also understand why they occur.”

So how exactly do our feelings relate to our decision to buy products? To answer that question, the researchers conducting this 2015 neuroscience study by Nielsen had people watch a hundred different commercials while measuring their emotional response. These were then divided into three categories: below average, average and above average. The ads that caused an “above average” emotional response generated a 23% increase in sales compared to the more “meh” commercials.

Angry voices, happy faces

The brain isn’t the only part of the human body that offers emotional clues: analyzing someone’s facial expressions, voice, heart rate, and even gait can provide insights into someone’s state of mind – which, just like brain waves, can be used for commercial purposes.

Take Cogito, for instance. This Boston-based startup sells voice analysis software to call centers to better read and anticipate the customer’s emotional state. By measuring variables such as intonation, pitch and speech rhythm, the software provides real-time analytics about the caller. Are they on the verge of becoming angry? Ping: a notification is sent. Does the caller’s voice indicate they’re receptive to hearing a sales pitch? Again, a notification is sent by the software.

Affectiva, which is also based in Boston, specializes in software that can read facial emotions and other nonverbal cues. Using a simple webcam or phone camera, Affectiva scans your face while watching a certain video or image (you can try it yourself). Deep learning algorithms then analyze these expressions to match them with the right emotions, typically classified as anger, contempt, disgust, fear, joy, sadness and surprise.

But it’s not just these basic emotions; the software can detect micro expressions as well, explains CEO Rana El Kaliouby in this podcast by Bloomberg: “When we first started it could read your smiles and brow furrows, which are big expressions, but now it has gotten even better so that it can tell if you’re squinting or smirking – for instance because you’re not persuaded by someone’s argument.”

Affectiva’s software is used by one-third of the global Fortune 500 companies, says El Kaliouby. “A lot of advertisers want to understand how their customers are connecting with their content and their products because emotions drive decisions; our purchase decisions, whether we choose to share content or not, as well as our loyalty and our brand perception.” Using facial emotion detection, Affectiva can tell whether an advertisement is engaging, which segments are boring or offensive – remember the toe fungus commercial featuring Digger the Dermatophyte, also known as Schimpie in Dutch? – and even if the viewer identifies with the featured characters.

Affectiva targets Fortune 500 companies while Braingineers’ website showcases large Dutch brands such as KLM and SNS Bank. Can businesses with considerably smaller pockets also benefit from neuromarketing?

The short answer is yes, says Bolder. “Our services start at 3000 euro. For businesses that depend on offering an impeccable user experience, such as e-commerce companies, I’d say it’s worth the investment.”

Still, it doesn’t always take an AI-powered headset to better understand what users are doing — or aren’t doing — on your website.

When thinking about your own customers’ journey, make sure to keep it as frictionless as possible. This means not overloading them with information (keep it clean and simple), don’t ask them for too much information (nobody wants to fill out huge forms) and design each page as if it will be the user’s first experience with your site.

“When in doubt, ask yourself what the user would do, then slowly work your way back to the business goals,” adds UX content strategist Jerry Cao, who authored Web Design Best Practices.

That said, following best practices in user experience design without taking into account how your customer feels about your brand is not advisable, says Bolder. “For instance, it’s a common misconception that the user’s journey from entering a website to paying for a product should be as quick as possible. That’s only true when the customer feels little emotional connection to the brand. When people do feel strongly about the brand, they actually love clicking through pages and lingering on the website.”

Although neuromarketing has become a more popular method of gaining consumer insights over the past decade, many companies still rely on surveys and written feedback in order to better understand their customers. And that’s a pity, says Bolder.

“We know 95% of our purchase decisions are taking place in our subconscious mind, so understanding that process is of key importance to any business.”