Optimism is an inherently good thing, right?
Be honest with yourself for a moment. How many times have you compensated, consciously or otherwise, for a lousy plan with the phrases "I'll figure something out" or "It'll all work out"? Statistically, you won't and it won't. But our minds are great at convincing us that these phrases are not merely a rapid escape hatch from a conversation we don't want to have; we're hard-wired to believe it.
Optimism bias is one of the oldest and most well-established facts in psychology and cognitive science. Armor & Taylor (2002) run down a list of experimental evidence. But if this seems like a simple phenomenon, it isn't. The problem is not that we underestimate the odds of negative outcomes even when we have full information (contrarily, in fact, we tend to wildly overestimate the odds of unlikely negative consequences – plane crashes, being hit by lightning, satanic ritual abuse, etc.). An example of that kind of reasoning might be, "I know 90% of smokers develop lung cancer, but I'll be one of the 10%." There's no need to give that a fancy name; denial and stupidity work just fine. No, optimism bias is our ability to convince ourselves that we're not in denial of the odds – instead, we tell ourselves we've found a way to change them in our favor.
We know that 75% of credit card holders make a late payment at some point, allowing issuers to impose punitive fees and APRs, but you won't be one of those people because you're very organized and you always pay on time.
Students beginning law or business school (and sinking $100k in the process) wildly overestimate their odds of getting one of the high-paying jobs one gets by finishing in the top 10. It's obvious that only 10% of any class can have that outcome, but the odds are not 9 to 1 against me because I'll study harder than everyone else and I'm smarter anyway.
Smokers routinely convince themselves that the grim statistics about tobacco and mortality are attenuated by some other behavior, i.e. your odds of getting lung cancer aren't really 80% because you smoke but you also jog three times per week and eat lots of organic stuff (or even more hilariously, "because I smoke lights / brand X / etc.")
Confronted with the cold reality that 50% of marriages fail, newlyweds inevitably conclude that their special bond with one another makes their odds of avoiding divorce much better than 50-50.
Gamblers understand that the odds are always on the house, but you have some kind of "system" – counting cards or whatever – that means the odds don't apply to you.
I can't help but think about optimism bias when I look at the daunting statistics about foreclosures and delinquent mortgages. Lazy minds assume that the problem is as simple as poor people taking out loans they couldn't afford, but when we look at the geographic distribution we see the fastest-growing states are leading the way – Nevada, Arizona, California, and Florida. In other words, the places where the upper-middle class buys its vacation properties and eventually moves to retire. Behind every bad loan, "investment property," and second mortgage is a series of powerful rationalizations: I can afford it as long as it keeps going up in value indefinitely (and it will!), I can't afford it but I'll be making more money soon, my 401(k) will never lose value so I'll be OK, or the generic "It'll all work out."
In other words, it's another one of those cognitive biases upon which our entire economic system is based. We invest because we think we're smart enough to do it without real risk and we buy under the assumption that we'll come into the means to pay in the future.