tl;dr version: It's not exactly clear whether or not Nate Silver is the wunderkind he is made out to be (his true gift may be more in the explaining than in the predicting), but poll aggregation is an idea whose time has come -- and pundits and pols ignore it at their peril.
Let's get one thing out of the way up front: I really like Nate Silver. His writing style is pitch perfect, and people like me really get off on his blend of scientific humility and technical swagger. Independent of anything else, his mini-celebrity status is well-deserved, in my opinion.
Still, there are many who have questioned whether his "special sauce" is really contributing anything of value. There are even people whom I really respect that are quite down on Silver. I think this all raises some fair questions. Despite Silver's legendary performance in 2008 and 2010, and even despite the fact that he continued the streak by making (by some measures) the most accurate predictions in 2012, there is still room to argue that some of this is just statistical flukiness.
To be sure, while Silver was arguably the most accurate quant on this most recent election, his advantage over other poll aggregators was razor-thin. I consider it an open question whether the "special sauce" really does yield a consistently more accurate answer than other simpler approaches, or if Silver has just gotten a little lucky. But the thing is, it doesn't matter all that much.
Whether Silver is really doing something special, or if one can make do with a more streamlined approach that simply seeks to aggregate polling data in a sensible way, without all of the meticulous adjustments performed by the 538 model, one thing is perfectly clear: In the last several election cycles, poll aggregators have yielded consistently accurate results, even while blowhard pundits have floundered.
I think Silver's predictive success (as opposed to his professional success, which as I have mentioned I think is largely due to his excellent writing style and explanatory abilities) is in large part an inevitable result of the vast increase in the amount of independent state-level polling being done these days. While Darren Sherkat is correct when he says, "on average, if you heap together shit, it doesn’t equal filet mignon. The Central Limit Theorem is not something to be fucked with...", it now appears (if you'll allow me to extend the metaphor) that there is enough meat in modern state-level polling to at least make a decent hamburger, if not filet mignon.
There's a bit of an epistemological Catch-22 here, in that the low sample size will always give the doubters enough wiggle room to argue that it's all a fluke. But it really seems to me at this point that the most parsimonious explanation for Silver's continued streak, as well as the excellent performance of his quant counterparts, is that there is a core of legitimacy to poll aggregation, and that models like the one employed by Silver (if not his specific model) should now be recognized as far and away the most reliable method for predicting the results of an election.
There's still plenty of room to doubt some of Silver's more exotic "voodoo", but it is increasingly hard to deny that the basic idea is sound: Take all of the data that you have available, calibrate it to take into account each polling outfit's historical performance, and average it all together. Apply error bars of appropriate size (e.g. Silver should not get credit for calling 50 out of 50, since his 50.3% "prediction" on Florida hardly counts as "calling it"), and you now have a stable and reliable and sufficiently hedged prediction. How many more elections will it take before we can agree on this?
0 comments:
Post a Comment