Focus Bias and Survey Bias


“Nothing in life is as important as you think it is, while you are thinking about it.”

Daniel Kahneman

Thinking, Fast and Slow

That may be my favorite quotation.  Those words pop into my head all the time, both at home and at work.  At home, that quote reminds me not to obsess over little things that aren’t important.  At work, I’m reminded that the headlines jostling the stock market around every day probably won’t have much long-run impact on our clients’ financial wellbeing.

The source is Daniel Kahneman.  After training as a psychologist, Kahneman shared the 2002 Nobel prize in a different field, Economics, for research studying how people make financial decisions.  In 2011, Kahneman published a popular work of non-fiction, titled Thinking, Fast and Slow, summarizing many of his most interesting conclusions.  His book was very popular and influential, turning up on many best-books-of-2011 lists.

Today, Kahneman’s star is on the rise once again thanks to Michael Lewis, the undisputed king of popular financial non-fiction.  Late last year, Lewis published The Undoing Project, a biographical look at the collaborative work of Kahneman and his late research partner, Amos Tversky.  Interestingly, the partnership eventually cracked, in part, because Tversky’s fame outshined Kahneman’s while they were working together.  Today, it would seem that Kahneman has received his due as well.

The quotation above relates to what Kahneman calls “the focusing illusion.” When you focus your thinking on a subject, that subject suddenly takes on an undue importance in your mind.  Kahneman devotes only about five pages of his 500-page book to this subject, although related ideas pop up consistently as well.  He gives two real-world examples.  In the first, survey respondents predict that paraplegics must be unhappy most of the time due to their disabilities.  In fact, paraplegics do not spend nearly as much time thinking about their disabilities as most people assume, and therefore, they are happier on average than most people would guess.  Disability is just the concept that pops into people’s heads when they hear “paraplegic,” so they focus on it and overestimate its impact on happiness.

Kahneman’s second example hits closer to home for me.  He asks people to estimate the happiness of the average Californian and compare it against somebody living in the Midwest.  Respondents conjure up images of sunny California beaches compared to gray, cold Michigan winters.  Focusing on the weather in this way leads people to assume that Californians are much happier than Midwesterners, on average.  In fact, the weather doesn’t play much of a role in determining people’s day-to-day happiness.  This is a valuable fact to remember as summer yields to fall, and particularly as fall gives way to winter.

We overestimate the importance of whatever we happen to be thinking about right now, and tomorrow we will be overestimating the importance of whatever we are thinking about then.  Salespeople exploit this bug by hammering a product’s differentiating features, tricking us into thinking those features are important to us because we have been made to focus on them.  This is a good reason to “sleep on” big decisions before making a commitment.

Do keep in mind, however, that bias is only somewhat predictable.  Our cognitive flaws cause certain funny behavior tendencies, no more.  Psychologists can show us that our thinking is very imperfect, and so is our decision making, but it is easy to overstate the case.  When I read Thinking, Fast and Slow a few years ago, I thought the book was generally a little overrated.  I really like behavioral finance—the study of how our emotions and prejudices affect our economic behavior—but academic studies are like everything else.  You can’t believe everything you read.

In graduate school I worked as a research assistant for behavior finance economists, and I know how hard it is to design and administer surveys that test exactly what they are supposed to test.  The other side of that same coin is that it’s very easy to generate shocking, headline-grabbing survey results.  Some researchers do this regularly, and intentionally, for selfish reasons to grab headlines (nobody I worked for).  Even the most careful, scientific-minded scholars can easily fall into the trap of accidentally designing their tests to drive odd behavior they purport to have “discovered.”

Sometimes predictable biases can be exploited to manufacture spurious evidence for unexpected, headline-grabbing conclusions.  For example, given three choices people automatically tend to go for the middle option because it feels the safest.  This tendency to gravitate toward the middle represents an interesting innate psychological bias.

For example, imagine a survey asking how many Japanese yen one U.S. dollar buys? Most people have very little idea about international exchange rates, and even educated people would not necessarily have a reason to know what basically amounts to random economic trivia. Imagine a questionnaire that offered the following three options:

A) 1
B) 10
C) 100

100 is the closest correct answer for the dollar-to-yen exchange rate.  For people who don’t know, the most popular random guess will be 10 because 10 is the middle option on the survey.  In contrast, respondents will perform much better given the following three options:

A) 1
B) 100
C) 1,000

Now the correct answer is in the middle.  Survey methods matter.  If I gave a group of academic economists the first version of the question, and I gave a group of, say, dentists the second version, then the dentists might land on the correct answer more often than the economists.  This would not necessarily mean that dentists know more about international exchange rates than economists do, but a little tweak in how the question is posed could generate “evidence” they do.

There is also a strong bias affecting what findings get reported in the first place, with more unusual conclusions getting preference over more mundane ones.  A study which shows economists know less than dentists about economic trivia has a chance of going viral on Twitter.  The opposite finding won’t get a single retweet.

Our minds are not perfect, rational calculators, but we shouldn’t underestimate ourselves either because we are probably more rational than the psychologists and behavior finance economists say we are.  Our minds aren’t perfect, but they are often as good as, or better than, the scientific methods that would claim to judge them.  If we focus too much on our brains’ imperfections—we will probably overestimate their importance, and underestimate ourselves!

Miles Putnam, CFA