SMELL TEST

Posted in Quick Hits on October 10th, 2017 by Ed

We have a problem throughout society with allowing anecdotal evidence to overrule data. (Blank) can't be that common, nobody I know has one!

That said, there are plenty of statistics that we encounter that deserve a double take. The unemployment rate is a classic example. Does anyone really think only 4.2% of Americans are unemployed? Only one adult in 25? No, and a closer look at the methodology – particularly the trick of removing people from the workforce after they've been unemployed for six months – reveals that the true unemployment rate as most people would define the term is higher. How much higher? Hard to say. But if a rate of something like ten or fifteen percent were announced, I doubt many people would feel that was unrealistically high.

I admit to having this reaction when I saw a report that traffic fatalities increased despite "distracted driving" being down. Consider your own driving experience and tell me, honestly, does it seem plausible to you that texting while driving is actually becoming less common? I must live in some sort of anomalous bubble if this is true, because if I had a nickel for every person I see whipping down the interstate or navigating a busy city street with their eyes down and glued to a phone I'd be a millionaire.

I see the data. And there's no reason to be suspicious of the motives of the Department of Transportation since they're perfectly willing to admit that fatalities increased. But there's something going on with these numbers that explains the decline in distracted driving in some way that has nothing to do with actual distracted driving. Maybe cops handed out fewer citations for it. Maybe whatever sample they analyzed is atypical. Maybe the decrease was a small amount well within the margin of error for their study. But it's hard to believe that people suddenly decided to stop looking at their phones or in-dash screens while driving. More people with more smartphones getting better data connections suggest that if anything, it should be on the increase.

Nobody wants to go wading into methodology, but often it's difficult to make any sense of data like this without it. And the more people see data that give cause for skepticism, the more they'll justify being skeptical of all data.