Who Are You Trying to Fool?

The most frequently cited result in the entire research literature on overconfidence comes from a 1981 paper published by the Swedish psychologist Ola Svenson.₍₁₎ Svenson asked people in the United States and in Sweden how they thought their driving abilities stacked up against other drivers. Most scholars cite Svenson’s results as showing that the majority of people think that they are better drivers than average. But that wouldn’t be impressive in itself, because it might be entirely possible for everyone to be better than average.

As leggy scholar Shane Frederick has pointed out, for instance, more than 99 percent of the population boasts more legs than average. His point is that in a skewed distribution, a few low numbers can pull the average down so that the majority of people are above it. As with leg counts, it is plausible to think that most drivers are adequately skillful, but there are a few drivers in the habit of texting in between bites of lunch and touching up their hair while they inspect themselves in the mirror. Most of us are better than those maniacs.

What’s interesting about the popular misrepresentation of Svenson’s result is that what he did was more impressive than showing that the majority of people think they are above average. What Svenson actually did was to ask people to place themselves in one of 10 deciles with respect to their driving safety and skill. A respondent who thought she was among the 10 percent of most skillful drivers should place herself in the top decile. Ranked on a percentile scale like this, it is, in fact, statistically impossible for the majority of people to be in the top five deciles. The median (which splits the top from the bottom five deciles) exactly divides the population into two equally sized halves. Yet Svenson found that 93 percent of Americans in his sample claimed to be more skillful than the median driver. The Swedes were not quite so recklessly overconfident — only 68 percent of them believed they were more skillful than the median.

So what exactly were Svenson’s respondents telling him? Was their claim to being better than other drivers an honest reflection of their beliefs? (Beliefs that would also lead these drivers to decline the purchase of insurance, thinking that their greater skill decreased their risk of getting in an accident?) Or was it more a strategic act, like telling the person interviewing you that you think have what it takes to succeed at the job, or telling your kid’s teacher that you think they belong in the gifted program?

There are at least three possible explanations for Svenson’s results. The first is that what they were doing was trying to impress him or make themselves look good by asserting their skill. If this were the case, then paying people for their accuracy should ameliorate their overconfidence. After all, if I expected to get a $1 million prize for accurately estimating my percentile ranking on a driver’s test, I would do my darnedest to accurately estimate my ranking. But it seems unlikely that Svenson’s respondents were just showing off. I don’t mean to insult Svenson or anything, but boasting on a survey about your enormous driving talents is like boasting to a census-taker about how many people live in your house — it is not clear that your boast gets you anything, or that anyone really ought to be impressed. Personally, I am skeptical that impressing others is a powerful motive in this instance. When other researchers have attempted to motivate respondents to answer accurately by paying them more for accuracy, it does not come close to eliminating overplacement._{(2, 3)}

A second possible explanation for Svenson’s results is that different drivers have different definitions of skill. It is possible that some drivers think skill is reflected in being able to drive while texting, eating, and resolving squabbles between children in the back seat. Whereas others may believe they are skillful, because they drive so carefully that they never exceed 30 miles per hour. If everyone agreed what it meant to be a skillful driver and had a good sense of their own and others’ skill, then exactly 50 percent should rate themselves above the median. On the other hand, if every person had his or her own idiosyncratic definition of what it meant to be a good driver, then all drivers could rate themselves as the best, and all of them could be right. There is good evidence that this ambiguity accounts for a good deal of “better-than-average” effects, like Svenson’s drivers.(4)

These results also speak against the third explanation: that drivers are simply fooling themselves about how skilled they are. Self-delusion may be the most commonly cited explanation for better-than-average beliefs. But if that’s the main reason for it, then overplacement should be greatest for things that people regard as important. After all, I don’t get as much out of believing that I am the best flagpole-climber than I do out of believing that I am the smartest or most virtuous.

 

I have been working for several years with my colleagues Jenn Logg and Uriel Haran to test this hypothesis.₍₅₎ Mostly, we have failed to find much supportive evidence of self-delusion. After searching long enough and hard enough, we have found some, but the effect is limited, and the conditions have to be just right to get it. We do find that the more important something is, the more people claim to be better than others, but only when the skill and its measurement are left vague. To pick a particular example, lots of people will claim to be more intelligent than others, but overplacement decreases substantially when those same people estimate their percentile ranks on an IQ test they all took.₍₆₎ This suggests that “better-than-average” beliefs are strongest when people can “get away” with employing an idiosyncratic definition of what it means to be good at something.

So if you want to reduce bias and the risk of people (including you) deluding themselves, be clear about what you are assessing and how you are measuring it. Do not content yourself with vague assessments of intelligence or driving ability; instead, examine scores on tests, problems solved, and accident records. Quantify your assessments using clear criteria and probabilities. And keep score. It will help keep you humble and help keep you honest.

By Don A. Moore Ph.D.