Scientists studying emotional expression now study faces in context so AI designers should too

Sign up for our newsletter, Artificiality, to get our latest AI insights delivered directly to your email inbox.

Last year, new research came out about how the science of emotions isn’t keeping up with the power of the emotional AI technology. Now, Nature magazine has released a handy update on the emerging science of emotions. It includes a challenge to the AI community: “Are AI companies going to continue to use flawed assumptions or are they going do what needs to be done?”

This is important for AI designers to understand. Failing to understand the full complexity, nuance and variability of emotional expression risks both bias and avoidance. People will easily be able to outsmart an AI by faking expressions. Humans do this all the time; in customer service, people smile even though they don’t necessarily feel happy.

The modern approach to emotional analysis from facial expression is to include the highly varied context under which humans express emotions through facial expression. Our facial expressions are far more nuanced and variable than was previously assumed and much of this nuance stems from the context in which we express emotion. There are three main contextual considerations:

  • situation – what’s the background? who is present? what kind of situation?
  • personality traits – what is someone’s internal context?
  • temporal – how does perception of emotion change as someone face moves? what about with acoustic changes in vocalization? how about with changes in body posture?

Scientists studying emotion have shifted their focus to study faces in context, not just faces. This means that AI companies working in emotional analysis should do the same thing.

In practice this means:

  • use moving faces and introduce dynamic analysis not static pictures. Vary the situation.
  • enhance labeling of training video/images to include context.
  • use technology (rather than actors) to discover patterns between facial movements and perception of emotion.

This video from Nature is worth a watch.

Photo by Lidya Nada on Unsplash

About Sonder Scheme: We are on a mission to help humans win in the age of AI by making AI design easier, more inclusive, more ethical and more human. Through our workshops, learning journeys and the Sonder Scheme Studio, we enable people to use design thinking-based tools to create innovative, effective and ethical human-machine systems. Both offline and online, Sonder Scheme empowers companies around the world to design human-centered AI.

Share on email
Share on facebook
Share on linkedin
Share on twitter