MORE LIKE NATE SLIVER AMIRITE

Posted in Rants on May 10th, 2016 by Ed

I don't even know what that title is supposed to mean.

Of interest this election season has been fake pundit Carl "The Dig" Diggler, the creation of two comedy writers, who not incidentally has predicted correctly the outcome of twice as many primaries as Almighty Beltway Knowledge God Nate Silver. Their prediction "method" involves little more than "gut feelings" and comedic stereotypes of the residents of different states, and so to even call it a method is unjustified. But that's precisely the point.

Nate Silver is, on the whole, a force for good. Attempts to provide analysis that relies on empirical data are, and always will be, an unqualified positive. His (at this point it is hard to separate him from his FiveThiryEight colossus, which of course involves other analysts and writers) reputation has taken a blow in 2016, though, and frankly I'm not sorry to see it happen. His analysis has always been terribly basic – on the order of something a good undergraduate statistics course would cover – and the reputation he has built as some sort of data god is a bit much. He has become, intentionally or otherwise, a liberal Bill Kristol; it doesn't matter if he's always wrong, he's still brilliant.

Two aspects of Silver's predictions deserve serious criticism, one of which Mr. Diggler emphasizes. He has a really annoying tendency to hide behind probability – "I didn't say Clinton would win, I merely said there was a 99% chance she would win!" Empirically, this makes perfect sense. Probabilistic analysis is never 100% accurate and does not claim to be. In the face of a large number of incorrect predictions, though, someone treated as an idol should have a better defense of his supposedly brilliant methods than The Simpsons' classic "Well, when you're right 52% of the time, you're wrong 48% of the time" gag. ("OK Jimmy, you're off the hook!")

The part that always has bothered me – and yes, of course I'm jealous – is that Silver became A Genius by predicting the outcomes accurately of two very, very easy to predict elections – 2008 and 2012 – in which a simple average, even an unweighted one, of barely-scientific polls by state was sufficient to see that the Electoral Vote would be lopsided. Those were not especially close elections, and it is not difficult to predict the outcomes of elections that are not especially close. As for his correct predictions of many other statewide races such as Senate and gubernatorial races, his model amounts to little more than averaging poll results obtained by other organizations and which are publicly available. In 2016, so far we see that the same magical techniques that told us McCain was going to get blown out (duh) are of minimal use in predicting an outcome that isn't totally obvious.

Yes, primaries are much more volatile and difficult to predict due to a number of factors like low voter turnout and a large, shifting field of candidates. In that sense we would expect predictions to be less accurate. But that's exactly the problem for Silver; it's becoming very easy to say "Well if you can't predict a race correctly unless all the conditions for making a correct prediction exist, what are you really doing? What good is this?" That has been the thorn in my paw with Silver all along. It's like saying that you can hit a lot of home runs, provided the pitcher throws the ball exactly where you want it, how you want it, with the wind blowing out at 50 mph. The limits of the Big Data approach and worship thereof are becoming very obvious. Like public opinion polling (on which much of 538/Silver is based), it is a useful tool when the gap between or among options is greater than the margin of error. When it isn't, the data don't tell you much at all. At that point you're effectively guessing. And Silver puts a number on his guesses, which gives them the imprimatur of scientific authority ("Clinton has a 63% chance to win!" – so precise!). But in reality he's telling you that Clinton is slightly more likely than Sanders to win a given race; the odds are about 3 in 5. That's an improvement on a coin flip, but it isn't much of an improvement.

It is far better to hear someone talk about data than to listen to some empty suit talk about his hunches or his conversations with various cab drivers. I'll take a Nate Silver column over 99% of what's out there for consumption. But people really need to stop chanting his name like it's a magical talisman that all but guarantees victory. What he's doing is not that complicated and, more importantly, not that useful unless the outcome of a given race is not in doubt. If the new working definition of genius is the ability to avoid being wrong about outcomes that are obvious, then I wish someone had sent out a memo in the mid-00s that the position was being filled. I, or anyone else with a basic understanding of political data, could just as easily have filled it.