ED vs. LOGICAL FALLACIES, PART 23: PROSECUTOR'S FALLACY

Posted in Ed vs. Logical Fallacies on March 9th, 2016 by Ed

Though I try my damndest to make all of the material interesting when I teach – that's where the 100,000,000 anecdotes and random facts come in handy – it's inarguable that some topics of academic interest (and curricular necessity) are a bit dry. Teaching statistics is a challenge in this regard. In my view, probability is the easiest thing to talk about. Probability and chance and randomness are fascinating. The problem is that very few people understand how to evaluate risk, odds, and probability correctly. It's very easy to mislead people with definitive-sounding statistics if one is so inclined.

In conversation this is an annoyance. In courtrooms it is a life-or-death issue. Juries don't understand statistics. Judges don't either. Hell, the lawyers and witnesses citing statistics in court probably don't.

Say you are on trial in Chicago and evidence is presented by the prosecutor indicating that you match DNA found at the scene. Your specific DNA profile is declared "1 in 2,000,000." The prosecutor uses this statistic – and the jury most likely hears it this way without prompting anyway – to imply that your guilt is 1,999,999/2,000,000 certain, or 99.99995% certain. You're going away for a long time.

The problem is that having a 1-in-2M DNA match does not in fact means nothing more than that in any randomly selected sample of 2 million people, 1 will have your profile. In Illinois' population of 13 million, this means six people have that profile. Adding in the bordering states' population within a short drive of the Chicago area, that's four or five more people. Add in the billion people around the world with access to fly into and out of O'Hare Airport on any given day and you have a veritable horde of DNA Twins out there. But limiting it to the Chicago area only, there are, statistically speaking, 10 individuals who match your DNA profile.

That means that it is not 1,999,999/2,000,000 percent likely that you are guilty. It would be more accurate to say that it's about a one in ten chance. And that doesn't even include the rates of false positives and human errors on DNA tests, which are both small but relevant in a large sample.

Here's another (real) example. During the OJ Simpson trial the prosecution unwisely downplayed its physical evidence and instead spent two weeks detailing the football star's lengthy history of violent abuse of his wife. Defense attorneys (Yes, they can use the trick too) then told the jury that about 1 in 3000 people who abuse a spouse or partner go on to murder that person. Therefore, they said, the tales of abuse were regrettable and true but totally irrelevant since the odds were so small (0.03%). Unfortunately, the only thing irrelevant is that statistic. The relevant question is not "What percentage of men who abuse a woman will go on to murder her?" but "What percentage of women who are murdered are murdered by the person who abused them?" According to FBI Uniform Crime Reporting data, somewhere between 70-80% of women who are murdered and had been abused are murdered by the person who abused them – not exactly a lead pipe cinch, but damning enough to establish that abuse can turn into murder to the jury.

The misleading statistic of 0.03% reflects nothing other than that murder isn't very common under any circumstances. A vanishingly small percentage of any defined category of people will ever commit murder. But at the trial that was not relevant, because the murder had happened. That the abuse happened was also established. So the question of how likely it was to happen is not relevant. It did.

Think of it this way. Say a plane crashes and Boeing is on trial arguing that the cause was not mechanical failure, citing that, "Only 1 in every 100,000 planes will ever have this particular mechanical failure leading to a crash." Probably true, but who cares? The question you want answered is, of planes that do crash, what percentage had that mechanical failure? It might not be a large number, as there are many possible causes of a crash, but it sure as hell will be larger than 1 in 100,000.

Once you're aware of this and begin to notice it, you'll be amazed at how often you encounter this fallacy in everything from advertising to news to your performance evaluations at work to casual conversation. You're welcome.