SMELL TEST

We have a problem throughout society with allowing anecdotal evidence to overrule data. (Blank) can't be that common, nobody I know has one!

That said, there are plenty of statistics that we encounter that deserve a double take. The unemployment rate is a classic example. Does anyone really think only 4.2% of Americans are unemployed? Only one adult in 25? No, and a closer look at the methodology – particularly the trick of removing people from the workforce after they've been unemployed for six months – reveals that the true unemployment rate as most people would define the term is higher. How much higher? Hard to say. But if a rate of something like ten or fifteen percent were announced, I doubt many people would feel that was unrealistically high.

I admit to having this reaction when I saw a report that traffic fatalities increased despite "distracted driving" being down. Consider your own driving experience and tell me, honestly, does it seem plausible to you that texting while driving is actually becoming less common? I must live in some sort of anomalous bubble if this is true, because if I had a nickel for every person I see whipping down the interstate or navigating a busy city street with their eyes down and glued to a phone I'd be a millionaire.

I see the data. And there's no reason to be suspicious of the motives of the Department of Transportation since they're perfectly willing to admit that fatalities increased. But there's something going on with these numbers that explains the decline in distracted driving in some way that has nothing to do with actual distracted driving. Maybe cops handed out fewer citations for it. Maybe whatever sample they analyzed is atypical. Maybe the decrease was a small amount well within the margin of error for their study. But it's hard to believe that people suddenly decided to stop looking at their phones or in-dash screens while driving. More people with more smartphones getting better data connections suggest that if anything, it should be on the increase.

Nobody wants to go wading into methodology, but often it's difficult to make any sense of data like this without it. And the more people see data that give cause for skepticism, the more they'll justify being skeptical of all data.

Be Sociable, Share!

38 Responses to “SMELL TEST”

  1. Dean Says:

    But we do need to go wading into methodology, early and often. For example, here's what the NHTSA Overview says about distracted drivers:

    "Number of fatalities in distraction affected crashes decreased by 2.2 percent from 3,526 in 2015 to 3,450 in 2016. Fatalities in distraction affected crashes were 9.2 percent of total fatalities in 2016."

    That doesn't mean there was less distracted driving. It means there were fewer accidents involving fatalities due to distracted driving. Big difference.

  2. scott Says:

    >> particularly the trick of removing people from the workforce
    >> after they've been unemployed for six months

    this is not accurate, and borders on deceptive. the CPS asks people if they worked for pay in the prior month. if so, they are employed. if not, it asks them if they looked for work in the prior month. if they say yes, they are "unemployed", and if they say no, they are "out of the labor force". this is irrespective of whether they are getting unemployment insurance, or anything else. The CPS just asks questions and posts the responses. BLS turns those responses into labor force concepts, and if to you don't like the consensus national unemployment rate, BLS helpfully computes several other rates you can track, just google alternative measure of labor utilization.

    >> But if a rate of something like ten or fifteen percent were
    >> announced, I doubt many people would feel that was
    >> unrealistically high.

    this is a logical fallacy, appeal to popularity, and it's readily debunked by publicly available materials describing the methods and the data collection.

    did ed really right this article or is there a ghost writer? bring back ed vs. logical fallacies. that was what brought me here.

  3. Amy Says:

    Statistical analysis apologists are among the most irritating of the pedantic trolls.

    No, I don't have the numbers to back that up.

  4. Ten Bears Says:

    Maybe people are just getting better at it.

    Proper punctuation, scott, and capitalization in academic circle tend to weigh heavily in aggregate the measure of one's credibility.

    And, you are mistaken.

  5. Ormond Otvos Says:

    I couldn't follow the sense of the article at all…there are many factors, such as better safety features, that prevent fatalities. Total crashes might be better.

  6. mago Says:

    Hey Scott, I think Ed did "right" it.

  7. JustRuss Says:

    Regarding driving while distracted, every year millions of cars with built-in infotainment systems are sold while millions of older vehicles without such systems are retired. I suspect it's somewhat safer to text/map/chat when you don't have to juggle a phone while doing it. That could account for the reduction.

  8. Dean Says:

    The "sense of the article" is based on a mistaken assumption that the agency reported a "decline in distracted driving." If the assumption were true, then the post would be a reminder to buck the fashionable empiricist trend when our intuitions flag anomalous or unexpected outcomes.

    There is no need to explain the decline, because the data don't reflect a decline. The data reflect fewer accidents involving fatalities in which driver distraction is identified as a cause of the accident. Ed's suspicions were justified, but not because the agency botched the study or because technical definitions toy with our ordinary reading of the data (as in the employed/unemployed example), but because the data reflect a different phenomenon than the one Ed imagined. He urged us to wade into the methodology. I did so. The answer appeared. No need to be skeptical of the data! The sense of the post holds even though–in fact, precisely because–the assumption was false.

  9. Dean Says:

    I'll add that even if the data were to show, as our intuitions might lead us to expect, that there is a general increase in driver distraction, nevertheless fewer people are dying in car accidents as a result of the distraction. But Ed's skepticism went not only to the reported outcome–shockingly, less rather than more–but to how the data were collected, which could only practically involve representative sampling (e.g., citations). I imagine we don't need to sample reports of car accidents involving fatalities. We have a nearly complete record.

  10. Leon Says:

    The screens built in to new vehicles are bad. We got a new Toyota recently, and our options were small screen or big screen. The radio no longer has tactile feedback, so when a song comes on that I don't like, or news comes on that I don't want my toddler asking questions about, I have to look down to get the offending noise off. There are some controls on the steering wheel, but touchscreen radios are a bad idea imho. And I try not to look at screens while driving. Looking around, I get the feeling I'm the only one.

  11. democommie Says:

    I ride a bicycle and can state that people who are looking RIGHT THE FUCK AT ME nearly run me over–most recently, about six hours ago. And that particular driver had a sort of "HOW DARE YOU ride your bike through the cross walk while I'm turning right! look on her face.

    I have no idea wtf they're doing but "attentive driving" is not in the mix.

  12. LK Says:

    Is there an increase in the number of states or local jurisdictions that give out fines for "distracted driving" such as phone use? An increase in the level of fines? More local "enforcement campaigns"? People react pretty strongly to that kind of incentive (it hurts where it counts, their pockets). They don't react nearly strongly enough as the police would have you think, but it's definitely measurable. Sometimes all it takes is for a few local papers and radio stations to report "an upcoming targeted enforcement campaign for using your phone while driving", for me to see a noticeable drop in the number of drivers watching their phones instead of the road around me.

  13. Dave Dell Says:

    Labor force participation rates get mentioned quite a bit alongside unemployment rates. Currently somewhere around 63% of the labor force is working or actively looking for work. While there are various measures of numbers in the labor force, to think the true unemployment rate is much higher than commonly expressed is believable.

    https://www.bls.gov/news.release/pdf/empsit.pdf

  14. Ten Bears Says:

    I kick their doors in, demo, throw rocks at them. I've even jerked one out of his car and bounced his face off the windscreen. I am of the firm conviction pedestrians should be armed with heavy caliber firearms, that crosswalk violators should be gunned down.

  15. Verbal Says:

    People misunderstanding what the U3 unemployment measure means and claiming it's a lie are almost as annoying as people who think gdp is a good measure of total economic health.

  16. Major Kong Says:

    I was crossing the street in Cleveland a while back. I had the "walk" signal. A Range Rover came out of the turn lane behind me and missed me by maybe an inch.

    As she went by I could see she was head down, staring at her phone.

  17. democommie Says:

    @ Major Kong & Ten Bears:

    Most of them are prolly rushing to an interview.

    I was crossing the street, on my bike, without a walk signal. I KNOW that I was breaking the rules. The reason I do the things that I do is because playing by the rules of "bike v anything else with wheels" is a good way to wind up, "OMG, he got fucking CRUSHED by the that 18-Wheeler but he had his helmet on so we can still have an "open casket"–fuck that.

    I am always ready to swerve out of the way of people I can see coming but if they're overtaking from behind and decide that they can turn, after passing me, before I run into the side of their vehicle…

    Sorry, I'm turning this into another, "democommie's bad suggestions for biking" column!{;>)

  18. democommie Says:

    On Topic:

    All gummint numbers are subject to "Askancification".

    Whether it's crime, employment, GDP, welfare rolls, prison populations or any other socioeconomic data it's put collected, compiled, analyzed, fudged, tweaked and MoE'd to get a result that is, more or less, accurate. And that's when there is a DESIRE that it be accurate.

    There has been a trend with reactionaries*, especially with "news" outlets like FuckTheNew'sCorpse and ReiKKKwing radio–for at least the last 40 years to just make shit up that suits their biases and then THAT stuff is subjected to Serial Pinocchioization. This "data" is then flushed to the webzroobz and they're like, "It's gotta be WORSE than that!".

    * They are as "conservative" as I am black, gay and female.

  19. Tim H. Says:

    On distracted driving, I put a lot of miles on bicycle before the invention of cell phones, drivers demonstrated a lot of difficulty seeing a bicycle then.

  20. Johnivan69 Says:

    I do love statistics. Results of the experiment indicate that following exposure to agent X, 33% of of the laboratory rats died, 33% lived, and the third one ran away.

  21. Redleg Says:

    The anecdotal evidence and personal experiences have been the bane of many college professors. It is sometimes quite challenging to convince students that the bulk of the research demonstrates X when the student's own experience demonstrates something other than X.

    I like to start the first day of the semester addressing "common wisdom" relevant to my discipline and then showing that the research tends to not support the common wisdom or that the phenomenon in question is more complex than assumed. I don't know if this resonates with students but I hope it does at least suggest to them that my field is not just "common sense."

    I am also bothered by the way the media tends to discuss scientific/empirical studies. While they might discuss an effect of X on Y (e.g., coffee consumption improves cardio-vascular health), they rarely discuss the effect size or the confidence interval. On the other hand, many people are probably not equipped to really understand statistical concepts and research methodology. Recent research in cognition shows that social identity plays a role in how we interpret data. For example, political party identification tends to influence the extent to which subjects mis-interpret data on various political issues.

    To make the problems worse, people now have access to all sorts of information about issues and cannot always determine the validity of the information so they rely on social identity again to determine which information is "true." All this climate data and debate is so confusing so I will just believe what the leaders of my political party have to say about it.

  22. Redleg Says:

    By the way, the Bureau of Labor Statistics (BLS) does report multiple forms of unemployment and includes information on their methodology.

    I would like to see better measures of under-employment, that is, people working in jobs that are well below their skill level because they are unable to find work commensurate with their skills and experience.

  23. Major Kong Says:

    Took a bad spill on a bike a few weeks ago. Helmet saved me from at least a concussion, maybe worse.

    https://www.dailykos.com/stories/2017/9/17/1699221/-The-Fall

  24. Kaleberg Says:

    Unemployment is measured at least two ways. There's the survey as noted by scott above, and there's the payroll data based on reports from payroll data processors like ADT. Then there are the statistics. The BLS publishes six different unemployment numbers U1 through U6, each with a slightly different gradation of unemployment. Then there is the population to employment survey which gets sliced and diced several ways as well, for example, for all ages and for ages 18-54. Then there are the various statistics about pay and working hours which get one even deeper into the weeds.

    The media tends to focus on U3, the official unemployment rate. I tend to focus on U6, the total unemployed. It's now around 8.3%, seasonally adjusted. Did I mention seasonal adjustments? If there are changing patterns in seasonal work, these adjustments will lag. Unadjusted U6 is around 9.3%. I've read a lot of popular media from the 1930s, back when the unemployment estimate was that a quarter of the working population was out of work. Measuring unemployment was a novel idea back then, and everyone admitted that there were a lot of ways of measuring it. They wound up choosing something like the current criteria and are loath to change the methods lest they introduce a discontinuity in the series.

    U6 tends to come close to my perception of actual unemployment, but that's just me. If you look at how people perceive the unemployment level, you almost certainly run into a non-linear effect. Even in good times, someone you know is likely to have been handed his or her ass at work. That's HR talk for getting laid off or fired. Then there is a gap while the news is absorbed and its impact lessened before the next report. As the frequency of such reports increase, I think they get noticed more and more. I'm guessing it seems more exponential and than linear. It's like the bandwagon effect if you consider that personal job loss stories are reported on social networks, so one hears each story many times.

  25. fledermaus Says:

    The purpose is to reassure the peons that everything is fine, not to accurately reflect the emplyment situation. Similar to how CPI measures housing costs: by calling up homeowners and asking them how much they would pay to rent their house, unfurnished. Which of couse results in a garbage number unrelated to cost of housing or market rent but it does downplay any actual inflation. And that number makes up over 20% of the total CPI.

  26. democommie Says:

    @ Major Kong:

    Me, too. I was turning and my left handle bar grip came off in my hand. I went down in a heap and way laying in a traffic lane for a few seconds. I was bunged up but not that bad. Several cars went around me without using turn signals and fortunately there wasn't someone late for whatever, behind them, going too fast to swerve or stop.

    I always wear a helmet, although it's not always exactly as it should be. Bottom line, I've worn one for the last 25+ years and broken 2 of them.

  27. sluggo Says:

    @Leon
    "Tactile feel' That's the key. I never text while driving but I sure as shit need to look at my phone to dial my wife.

  28. sluggo Says:

    Sorry to get off topic, but has anyone else notice a ton of AARP members driving with small dogs sitting on their laps? Or is this a New River Valley/ Roanoke VA thing?

  29. jcastarz Says:

    In my 40+ years as a bicyclist, I've come to respect the Laws Of Physics first, and the Laws Of The Road second. If there are two tractor trailer trucks approaching each other on the same secondary road that I'm on, and I'm caught between them, I will quickly get off my bike and let them have the full road to pass each other – my own rights be damned. This behavior is not always dignified, but I believe it's kept me alive. There are some situations even a Styrofoam helmet can't bail me out of…

    Now, as to how often – statistically – this has happened, I'd have to say: approximately 1 mile in every 500.

  30. mothra Says:

    The reason I do the things that I do is because playing by the rules of "bike v anything else with wheels" is a good way to wind up, "OMG, he got fucking CRUSHED by the that 18-Wheeler but he had his helmet on so we can still have an "open casket"–fuck that.

    While I am sure Ed did not intend for this to turn into a safe biking forum, but democommie, while I am sure you will keep on keeping on, be aware that breaking the rules is a really great way to get hit and then not have insurance cover any of your injuries and maybe put you in line to be sued for damage to the vehicle that hit you. Plus, you validate every asshole driver's assumption that all cyclists are scofflaw assholes who don't belong on the street, so fuck 'em, why bother sharing the road?

  31. mothra Says:

    Also…distracted driving would go WAY down if every car came with manual transmission and automatic transmission was an expensive option. Like the bad old days.

  32. Major Kong Says:

    @jcastarz

    I've done the same thing. Don't want to end up like the old saying:

    "Here lie the bones of Henry Clay. He died defending his right of way."

  33. jcastarz Says:

    NOTE: I DID drag statistics into the end of my bike post just to keep with the theme of Ed's Post. Now I'll expound upon my statistic by asking: is 1 in 500 miles a steady trend, or is it increasing or decreasing? To which I'll again keep in theme with Ed's Post by saying: it all depends; where and when you ride makes all the difference in the world.

    Statistics can be misleading – as has been discussed herein, and I do believe those "official" employment and unemployment statistics are often hiding some underlying problems in the workforce.

  34. HoosierPoli Says:

    U3 does not track what any normal person would call "unemployed", but that doesn't make it illegitimate. It's a very specific measure of workforce participation, and if you're not informed enough to interpret it, you don't deserve an opinion. That said, nobody with two brain cells to rub together thinks that the labor market is booming, and there are plenty of other statistical measures that prove it.

    If you want to blame someone, blame the braindead news media who assigns journalists to cover beats they don't understand, and are therefore easily distracted by shiny objects. See also: all science journalism ever.

  35. Katydid Says:

    My neighbors routinely drive with their little adorable dog on their lap; I think they're in their late 40s – early 50s (I don't know them well enough to ask). I wouldn't dare drive with my dog on my lap because he would also be on my head, in the glove box, under the seats, out the window and into someone else's car, etc. But I'm not an old fogey so what do I know?

    Anecdotally, in my little bubble, drivers are ever-more distracted. I've been taking a back road to a dog park regularly for the past couple of months, and *every. single. time.* there has been an accident somewhere on the 3-mile trip–I've also witnessed quite a few of them. I used to travel that road regularly in the late 1980s through late 1990s and never once did I see an accident, now it's every week.

    As Demo points out, even when they're not obsessed with whatever little screen they're staring at–even when they're staring right at you–it's obvious their minds are anywhere else. I suspect these are the same people who fall into a coma in the grocery store aisle–always parked crossways and blocking everyone around them.

    I'm still driving my screaming-metal-deathtrap in part because the new cars tend to have all that distracting crap going on. I don't want to trade stocks on Wall Street or upload a humorous-looking cloud to Instagram while I'm barreling down the highway at 70 mph.

    Major Kong–dang, man, that was a terrible accident. Glad you're okay.

    All you bike riders and pedestrians, please take care! Your valuable input would be missed by everyone in the comments.

  36. democommie Says:

    @jcastarz, mothra and others:

    I don't ignore physics, I am very well aware of what happens when people who are not paying attention run over people who can't get out of their way.

    As far as my insurance company paying anything to cover injuries incurred while I'm on a bike:

    A.) Not holding my breath.

    B.) The VA doesn't charge me and anyone else just won't be getting paid.

    C.) If I damage your car because I'm violating the law, while you're not, we'll talk about it.

    When people in motor vehicles start treating cyclists like they're deserving to be considered as having equal rights I'll worry about conforming to the rules they routinely ignore.

    One of my neighbors just told me this morning that the car she was in nearly ran me over the other day. She was wrong, in the first instance–I was under control and never got closer to the car then about 20 feet. She was a passenger so she couldn't have put her foot on the brake–the driver never attempted to do so and was in the middle of the goddamned street–thus making it impossible for me to maintain my line going around the corner.

  37. under armour speedform gemini uk Says:

    Driving – 15 Hours to include (This will be 10 1-1 2 hour training classes): Open Road and City Driving, Road Test Procedures, Parallel Parking, Vehicle Instruction can be done on Truck (CL-B), Bus (CL-BPS) or both. The form will figure prominently if a student would be eligible for either need-based or merit-based financial aid for students.max to create your reclamations. Programming needs to be specific and well planned to avoid overuse or burn out. For more details visit In the specifi

  38. under armour speedform gemini trainers Says:

    We runs by a strategy of being exclusive from others,Vans Old Skool Canada Sale, not only we are one of a kind from others we want our clients,Vans Old Skool Canvas White Outlet, you, to be distinctive and different from others at the same time.2. Stay positive in everything and your business will once again thrive.asia at www. On top of that,, all our offices are placed to match the business world. Some of the catering services providers at Fort Worth have been at people’s service for mor