TYPE I ERROR

Posted in Rants on November 9th, 2016 by Ed

I'm not the least bit embarrassed to have gotten it wrong, since having done so puts me in an exclusive club called Everyone at the moment.

Predictions have to be made based on the data available. All of the available data pointed toward one outcome. Polling has a margin of error, and we understand that. Even accounting for the margin of error, there's a 1 in 20 chance that the result lies outside of it. I don't think anyone wants to read a lengthy treatise on confidence intervals, normal distributions, and p < 0.05 right now, but the entire process of statistical analysis of pre-election data (and most data in the scientific world, period) is built upon the reality that 5% of the time you will accept a hypothesis that in reality should have been rejected. Beyond that, statistical models depend on any number of assumptions that can and often do turn out to be incorrect. The biggest loser this week is the obsession within political science with quantitative wizardry ("Check out my new estimator, bro!") and in the political media with forecasting models updated by the second as reality overtakes their assumptions.

Here's the problem: There are other ways, but there are no better ways.

Oh, you "had a bad feeling" about the election? That's nice. You had some theory you pieced together that managed to predict correctly the outcome of a contest with only two possible winners? Amazing. You have a brilliant post-election take on what "would have" happened had Bernie been nominated, had Jill Stein voters not voted for Jill Stein, had X not done Z? That's great, I can also sit around and make up hypotheses that can't be tested, theories that can't be proven or disproven. There's nothing wrong with any of this, and it's what people do during and in the wake of elections. But make no mistake about what you are doing when you engage in this kind of "logic" – you are pulling things out of your ass. You're guessing. You're in 6th grade writing a Persuasive Essay based on the prompt, "How would the election have turned out differently if ____?" Would Sanders have done better against Trump? Intelligence is not being able to answer that question; intelligence is understanding that any answer you can offer to such a question is pure conjecture.

I don't look forward to the months of hand wringing, of people explaining ("explaining") why this happened. The construction of post-election narratives is a process that interests me only in that none of them can be proven and we go through a collective process of deciding which one is Correct based on feel, like a clueless car buyer kicking tires and deciding that this model truly is superior to the alternatives. The world of data-driven predictions is not a perfect one, and it is one in which we all accept that we will be wrong a not-insignificant percentage of the time. It is a better world to live in, though, than one in which we all sit around filling the world with our Hunches and gut feelings. The modern world, and certainly our educational system, strongly encourages people to think unscientifically – begin with your conclusion, then construct an explanation that supports it. This process has the advantage on the user end of allowing everyone to feel like they are correct, with the obvious disadvantage of being like the gemstones – pretty, alluring, and fundamentally worthless.