Why we fool ourselves into optimism

Editor's note: Tali Sharot, a faculty member of the Department of Cognitive, Perceptual and Brain Sciences at University College London, is the author of "The Optimism Bias: A Tour of the Irrationally Positive Brain." She spoke at the TED2012 conference in March. TED is a nonprofit dedicated to "ideas worth spreading," which it makes available through talks posted on its website.

(CNN) -- Close your eyes for a moment and imagine your life five years from now.

What sort of scenarios pop into your mind? How do you see yourself standing professionally? What is the quality of your personal life and relationships?

Though each of us may define "happiness" in different ways, it remains the case that we are inclined to see ourselves motoring happily toward professional success, fulfilling relationships, financial security and stable health. Unemployment, debt, Alzheimer's, any number of other regrettably common misfortunes are rarely factored into our projections.

Watch Tali Sharot's TED Talk

Are we wired to be optimistic?

According to most estimates, 80% of the population hold unrealistic, optimistic beliefs about their own future. However, ask people whether the economy is going in the right direction or how they feel about the future of their country, and you will hear: "absolutely not" and "going down the drain."

TED.com: Dan Gilbert on our mistaken expectations

Collectively we are quite pessimistic about the direction of our nation or the ability of our leaders to improve education and reduce crime. But private optimism, about our personal future, remains incredibly resilient.

Surveys show that most people overestimate their prospects for personal achievement, expect their children to be extraordinarily gifted and hugely underestimate their likelihood of divorce and cancer.

What always puzzled me was how we manage to maintain optimism in the face of reality. We experience failure and heartache, we read the newspaper -- we know the economy is unstable, but still we remain optimistic about our own odds.

As a neuroscientist I found this especially surprising, because according to all classic theories of learning when expectations are not met, we alter them. This should eventually lead to realism, not optimism.

By scanning the brains of people while they learned from positive and negative information about the future we uncovered a possible answer to this puzzle. Surprisingly, when people learn of what the future may hold, their brains faithfully encode desirable information that can enhance optimism, but fail at incorporating unexpectedly bad information. (My fellow scientists and I were able to alter this tendency using transcranial magnetic stimulation -- watch the TED talk to see how.)

When we learn of Oprah Winfrey's success story, our brain takes note and concludes that maybe, we too may become immensely rich one day. But when told the odds of divorce are almost 1 in 2 we take no notice.

This means that warning signs such as those on cigarette packets may only have limited impact. "Yes, smoking kills -- but mostly it kills the other guy." But at the same when we hear the housing market is going up we think -- "ohhhh the value of my house is going to double."

TED.com: Helen Fisher studies the brain in love

In fact, economists have suggested that optimism was a root cause of the financial downfall of 2008. The optimism bias was not only blurring the vision of the private sector, but also of government officials, rating agencies and financial analysts who constantly expected the market to go up and up.

The belief that we are relatively immune to future harms can also put us at physical risk. Take for example an e-mail I received from a California firefighter who read my book about the optimism bias. He says fatality investigations involving firefighters often include statements to the effect of "We didn't think the fire was going to do that" even when all of the available information about risks was there to enable safe decisions.

The British government for one has decided to try to address these problems. As a first step, it has acknowledged that the optimism bias causes individuals to underestimate the cost and duration of projects.

Specific guidelines of how to correct for the optimism bias in appraisals were published in the British government's Green Book, which provides an overall methodology for economic assessment. Adjustments for the optimism bias have since been factored into the budget of many UK government projects, including most recently the 2012 London Olympics.

Despite all these potential pitfalls, the science of optimism clearly indicates that, on balance, viewing the world through rose-tinted glasses is a good thing. We now know that underestimating the pain and difficulties the future undoubtedly holds lowers stress and anxiety, consequently enhancing physical and mental health.

TED.com: Four lessons from robots about being human

Believing that a goal is within reach motivates us to act in a way that will help us attain it. This may, for instance, explain why optimists work longer hours and tend to earn more.

Yes, the 2012 London Olympics budget had to be adjusted to account for over-optimistic prediction, but if the human spirit were not optimistic, would there be anyone around to participate in the actual Games?

My guess is that the number of athletes who expect to win a medal at the Olympics significantly outnumbers the number of contestants who will mount the podium to be garlanded in due course. Most athletes subject themselves to years of intensive training because they can clearly envisage the end goal. At the end of the day, to make any kind of real progress we need to be able to imagine alternative realities -- better ones, and believe them to be possible.

Follow @CNNOpinion on Twitter.

Join us at Facebook/CNNOpinion.

The opinions expressed in this commentary are solely those of Tali Sharot.