Get Zogby
Let’s do a little mental exercise and compare the original post to Rothenberg’s article. The difference? Rothenberg takes specific issue with the results and demonstrates problems that fit our understanding of public opinion. The post by Ruffini tries to make an argument through innuendo and guilt by association. Oddly, Ruffini doesn?t seem aware of the Arab/Muslim-American strategy pursued by Grover Norquist and Karl Rove in 2000. For some background one should visit The New Republic stories here and here.
Is Ruffini upset at Norquist?s ties? Give Rove credit, he backed off a bit on learning the background of some of Norquist?s associates.
If one wants to address the accuracy of a poll, address the poll, not some bogeyman.
Traditionally, Zogby has been effective at estimating hang-ups and oversampling for similar demographics. Given refusals are more likely to come from Republicans, this is essential to getting an accurate poll. William Safire wrote some very complimentary columns about this before and after the 1996 election. Zogby specifically mentions this himself on his site. The art of polling is how one determines the likely voter profiles. These two methods are proprietary and as such, a big black box that people can’t evaluate except at the end of an election.
This, of course, makes Mr. Hanks comments about how Zogby was way off until the end rather strange. Nothing new there given it is Hanks, but how does one know if he was way off if one doesn’t know the characteristics of the underlying population? Details, Smetails.
Looking at the 2000 election, let’s examine the bias. For Senate races mentioned on his site we see nine Senate races from 2000. All but two of those fall within the margin of error. The two that don?t fall within the margine of error are Pennsylvania and New York. The Pennsylvania race undercounts the Democratic vote, but primarily because the Democratic candidate was horrible and undecideds probably broke late. Zogby was hardly to blame for that. In fact, he probably had an accurate snapshot going into election day.
In New York his numbers were significantly off and overestimated Rick Lazio?s support by a great deal. Why? This probably occurred because he overestimated the number of Republican voters and thus oversampled them in trying to get an accurate count. He messed up. But he did so attempting to correct for underrepresenation of Republicans.
In his other races we see Wisconsin off a bit in favor of Democrats, but this is similar to Pennsylvania where the Republican candidate was doing so poorly party regulars probably broke late. Washington shows favor towards Cantwell, but well within the margin or error. His other races were off nearly equally in both directions, but within the margins of error. Zogby wasn?t perfect, but he was pretty good.
So is there any evidence of Zogby tampering? Not really. The problem with the rant by Ruffini is polls are only as good as the user reading them. Ruffini isn?t very good at using them. The farther out from election day a poll is taken, the less reliable a poll is and thus the more likely one is to get wider swings in volatile races. This often is due to swings in the electorate itself. Believe it or not, some people change their minds due to campaigns.
Additionally, pollsters improve their sampling and likely voter methodology when they get strange results meaning later polls are better than earlier polls. Rothenberg?s criticism is accurate. Something was wrong. Zogby and other pollsters don?t admit it because it is bad for business, but common sense allows any reasonable person to figure that out. The worst thing Zogby (and about every other pollster on Earth) is guilty of is self-promotion.
Ruffini tries to draw conclusions about the swings of the polls from John Zogby?s political attitudes instead of addressing the underlying population being studied and Zogby?s techniques at capturing that dynamic. There are problems with some of the recent Zogby poll results, but those seem likely due to the factors Rothenberg cites and it still being a period where people are making up their mind. Zogby doesn?t do this for his health; he does it to make money. If your primary market is news agencies, inaccurate results will make selling your product harder.
Bottom line: Zogby is a pretty good pollster, but not perfect. Instead of hand waiving and whining about his ?political agenda,? one would be better served by looking at his polling techniques and addressing those. Rothenberg did it, why is it so hard for others?
Update: MyDD revisits the issue and makes some good points. He thinks I give Rothenberg too much credit. Fair enough, but I think Rothenberg makes a good point that we should be cautious of polls that don’t fit our intuitive sense of the race. I probably overstated his content in trying to point out how to successfully address the quality of a pollster. The New Jersey and Tennessee cases are examples of something being wrong. Missouri and Minnesota should be taken with some skepticism. Most likely the swings aren’t that great and Zogby is honing in on his correct samples. In most of his polls he’ll be within the margin of error and maybe one or two he’ll biff. His numbers before the final round are subject to his improvements, and to changing race dynamics.
My sense is that both are elections that will be determined by turnout. On the ground in Missouri there is no upswell of Talent support compared to before and from I’m hearing in Minnesota the race still feels like a damn close race. ally, both Carnahan’s and Talent’s campaigns are putting too much stock in the poll this time. Ultimately, we’ll find out on election day.