I dreamed last night that my phone had a magical app to distinguish between healthful foods and desserts. I simply snapped a photo of any edible item, and the app instantly classified it as “Food” or “Dessert.” It was nearly 100% accurate, because a hoard of Ph.D. scientists had trained the app using millions of photos of meals, snacks, and sweets.
My dream app didn’t work so well, however, when I photographed a random object, such as a doorknob. The app still classified it as Food or Dessert. That’s because the app didn’t really identify objects, it just computed similarity: namely, whether the doorknob’s appearance was “closer” to that of food or dessert.
How did my dream app determine closeness? Using the millions of photos it had learned previously. The app built up a statistical summary from all the photos of fruits, vegetables, and other healthful foods, and a second summary from the photos of desserts. It then measured mathematical distances from the doorknob to the two summaries, and the shorter distance determined the winner. (Yeah, I dream in statistics.)
In real life, scientists use the same technique to distinguish between emotions. They train a piece of software with hundreds of photographs of faces, using techniques from artificial intelligence, so it can distinguish between (say) scowling vs. smiling faces. Or, instead of photographs, they train the software with data about heart rates, blood pressure, respiration, and sweat, or with activity in various brain regions, in the hopes that the software can recognize biological patterns and classify the emotion someone is experiencing.
These techniques are collectively called pattern classification, and I discuss them in chapter 1 of How Emotions Are Made: The Secret Life of the Brain. They work extremely well, just like my imaginary Food/Dessert app did. The emotion “classifier” program computes statistical summaries of faces or bodily activity or brain scans or whatever. Then, given a new instance, the classifier determines which summary is the closest fit. The results are eerily accurate.
Emotion classifiers are also subject to the “doorknob problem.” If you have a facial emotion classifier for scowls vs. smiles, and you hand it a photo of (say) a cross-eyed face with puffy chipmunk cheeks, it’ll still classify the photo as anger or happiness. Remember, the classifier is just computing the minimum distance from a bunch of summaries.
If scientists had stopped there and celebrated their success, this story would have a happy ending. But unfortunately, some of them have taken a further step that makes no scientific sense. They point to the statistical summary of (say) anger and claim they have captured the elusive essence of anger. They call it a “biomarker” and report that it’s an actual picture of what happens inside every person who feels angry.
Well, no.
Think back to my dream app that summarized a million photos of desserts of all types, shapes, and sizes. The summary is not some inner essence of dessert. A luscious slice of apple pie, dripping with syrupy goodness, needn’t match the dessert summary well at all. It just has to be relatively more like the dessert summary than like the nutritious food summary.
So when it comes to emotion, suppose that you computed a summary for anger that includes heart rate and blood pressure in a particular range, or increased activity across certain brain networks. A given instance of anger may match none of these values and still be classified as anger because it’s closer to the anger summary than to the happiness summary. In fact, you can have an experience of anger that exactly matches nothing from the anger summary.
In pattern classification, the bodily patterns are not biomarkers. The brain patterns are not actual brain states. They are both statistical summaries that need not exist in nature, but they still work great for classification.
Put another way, the average family in the U.S. has 3.14 people, but I have yet to meet a real family of this size. Scientists who say that an anger summary is the biological marker of anger are mistaking a summary for the norm.
Moreover, different pattern classification studies can (and do) produce different-looking summaries for the same emotion. That is, the summaries don’t replicate, even if their classifiers are very accurate at their job.
For additional scientific details, see:
- “Pattern classification,” from the web endnotes of my book, How Emotions Are Made: The Secret Life of the Brain.
- Clark-Polner, E., Johnson, T. D., & Barrett, L.F. (2016). “Multivoxel pattern analysis does not provide evidence to support the existence of basic emotions.” Cerebral Cortex. DOI: 10.1093/cercor/bhw028.
