plezWorld & National Polls

I perked up in my seat this morning when reading a blog posting that suggested that the mainstream media (MSM) have been "playing with" the poll numbers to show a bigger bounce for John McCain after the Republican National Convention than for Barack Obama after the Democratic National Convention by altering the party identification (party ID) percentages of the samples. Being one to question these types of things, plezWorld decided to do some research into the matter. Polls, especially national polls, are not an exact science - that's why the numbers can vary over time (everyone doesn't answer the phone, everyone who answers does not agree to participate, some people only have a cell phone, etc.).

I found a terrific NPR audio file that has a very good explanation of polls and polling (I have the embedded file below and the show lasts about 55 minutes). In addition, I found some excellent information at FiveThiryEight that basically lays out the methodology for national polling.

Keep in mind, our president will be elected on a state-by-state basis, so national polls are only a small indicator as to who will win in November. Remember, Al Gore had a substantial popular vote advantage over George Bush, but they ran neck-and-neck in the state-by-state electoral college vote.

What follows are excerpts from blogs that raised the issue and answered the questions about polls.

From Seth Colter Walls at Huffington Post blog:

This week's mainstream coverage of the presidential horse-race has been dominated by a series of polls showing the McCain-Palin ticket with its first stable lead over Obama and Biden. Gallup's tracking poll, USA Today and CBS News all show the Republicans with some kind of lead over the Democratic ticket. But, interestingly, all three polls were also conducted using a higher sampling of Republican voters than in July, raising a question of methodology.

In a year in which Democrats have a lead of 11 million registered voters over Republicans, and have been adding to that advantage through a robust field operation, are pollsters over-sampling Republicans?

Despite a raft of advantages in the electorate for Democrats, in September's first Gallup tracking poll, an equal number of Republicans and Democrats were surveyed (including "leaners") from Sept. 3-5, compared to a 10-point Democratic identification advantage two weeks ago. That partisan makeup of the polling pool resulted in a 5-point lead for McCain in Sept. 5 tracking poll. Meanwhile, the new CBS poll features a 6-point swing in partisan composition toward Republicans, which plays some role in the poll's two-point lead for McCain. Finally, the latest USA Today poll, which claims a four-point edge for McCain, was arrived at after a 10-point swing in partisan makeup toward GOP respondents.

Some polling experts say the changing state of party affiliation in the field is slow to be reflected in polls themselves. Democratic pollster Anna Greenberg told the Huffington Post that "when it comes to registration and turnout, the polls often do a very bad job of taking those [factors] into account," because newly registered voters aren't in the voter files used by firms that survey public opinion. "You could make the argument they are under-representing new registrants," she said, which would mean that the Democrats new edge would not be taken into account.

Monday's USA Today poll had a 48-47 split between Democrats and Republicans surveyed. That represents a nearly 10 point shift in party identification toward Republicans since USA Today's July polling. When asked for comment, USA Today polling editor Jim Norman wrote that "it's possible" that their latest sample includes too many Republicans. Though he added, "it's also possible that we have too many Democrats," because "there's always the chance of an over- or under-representation" in polls.

Still, Norman admitted that the GOP identification in the latest survey has spiked. "The party ID in our most recent poll does show a shift away from what Gallup has been getting in earlier polls, going all the way back to 2005," Norman said. "But previous conventions -- the Republican one in 1988, the Democratic one in 1992, the Democratic one in 2000 -- have also shown shifts in party ID toward the party that had the convention, and those shifts seemed to last, to greater or lesser degrees, though the election. Further, I've been told by Gallup that their tracking poll has shown a similar shift in party ID since the Republican convention. ... I guarantee you we will be watching closely in all of our polls between now and election day to see whether there are further shifts in party ID in either direction."

And it's true. Gallup's own GOP identification (including leaners) has swung six points in the last month, from 42 percent of voters to 48, according to tables provided to the Huffington Post. Meanwhile, solid and leaning Democrats have fallen from 52 to 48 percent of those polled. For political scientists who believe that partisanship is largely stable over time -- and who take note of the advantage in voter registration being experienced by Democrats during the same period -- the newly GOP-heavy poll samples can raise eyebrows.

Emory University political scientist Alan Abramowitz is highly skeptical of the new Gallup, USA Today and CBS polls. About the latter, which showed a statistically insignificant two point lead for McCain, Abramowitz said: "One reason for the dramatic difference between the two recent CBS polls is that the two samples differed fairly dramatically in terms of partisan composition. The first sample was 35.2% Democratic, 26.2 percent Republicans, and 38.6 percent independent. The second sample was 34.9% Democratic, 31.1% Republican, and 34.0% independent. That's a change from a 9 point Democratic advantage to a 3.8 point Democratic advantage. That alone would probably explain about half of the difference in candidate preferences between the two [CBS] polls."

This DailyKos blog posting ran with the information in Huffington Post.

Excerpts from the DailyKos blog:

What about McCain's convention and the Palin factor? Good question. Those events certainly have helped, but McCain was just drawing from a previously depressed group within pre-existant, not additional GOP numbers. Now we need to make it clear. Dems, GOP and Independents can all swing within a specific group (and do, mainly through Independents, with some soft dems, soft repubs though).

Now this is not to say McCain-Palin haven't shown recent movement. They have. Their base (at the moment) seems energized. But the difference is, if party ID was truly representative Obama would still be leading comfortably, or big with most polls, or barely losing or keeping it tight (as he's doing now) with outlier polls like the recent 10 pt USA-Today LV Poll. In other words, when Dem party ID is down, and McCain makes movement, Obama's numbers are not going to appear robust. When party ID is accurately representative (which it rarely is) and there's movement towards Obama, then you get your Bloomberg, Newsweek 13-15 point blowout leads that the media hated. No more of those polls! Lol.

I think the truth or Barack's true representation, lies somewhere in the middle. I think Obama has a natural 5-8% lead out of the 15% Democratic cushion (+11 million out of 73million Dems, Repubs while splitting Indies). You may think his cushion is a little bit more or less (if you feel he has one at all). That's what I think. But if you follow these polls closely, it almost seems they cycle party ID regularly as if on a whim or some periodic schedule.

But my problem is, how come it seems everytime McCain makes a dramatic shift in the polls, it usually follows a good story line (this time Palin and the GOP convention bounce) and is preceded by a perceived inbalanced party ID shift or noticeable increase.

For any potential detractors of this type of article/diary. I would certainly have more faith, if every time a "new Horse Race Poll" came out, it didn't have the chicanery or stench of noticeable, inexplicable party ID shifts attached to/preceding it. Makes sense?

I don't think that's being too disillusioned or partisan.

The moral of the story Kossacks is. Stay strong, volunteer, phone bank, campaign, donate and keep a positive perspective and read through the polls when you can and/or wait for trends, before you get yourself down.

I'm sure the state polls are being done the same way as well (minus a couple of potential outliers, like that North Carolina, SUSA poll).

That's why the Obama Campaign (if you notice) never seems to worry about these sorta things, as much as the breathless press and net-root supporters do. Keep in mind, this is the team (Obama's)that produced the "Obama Spreadsheet". Remember? Despite the myriad swings, soap operas, worry and drama during the democratic primaries, the Obama team pretty much nailed it (state election results & numbers)during the 58 contest. With that kind of success and forecasting under their belt. I'm sure they feel more than equipped to handle, the upcoming 50 state contest on November 4th, as well.

I'm not saying Obama can't lose. He can! We've all discussed the previous bounce models that may end this week or last a month. We'll see and follow this closely of course. Obama also needs to go on the offense (instead continually reacting on defense) and point out all of McCain-Palin's hypocrisies and lies.

Now Sarah Palin; regardless of what we think of her, is the latest media darling. Is there real substance there; probably not. Is she going to do a revealing, potentially game changing interview, who knows? Is the finally media going to do their job (despite McCain campaign threats to the contrary) and expose/vet this womans wafer thin record, credentials and scary positions? Who really knows. Geraldine Ferraro started out like Hot Cakes in 1984 for about a month or so, but her and Mondale's campaign imploded due to repeated scandal and gaffes(sound familiar?)

Don't forget; Democrats have all the structural advantages that laid the smart money on Obama in the first place(you know bad economy, bad incumbent President, negative job loss, etc.. That's all still there. I don't want to jinx Obama, but a little known fact is, that no incumbent party in history has every won the Presidency with all those negative factors against it/present (I know "knock on hard wood twice).

Make no mistake. It will be a fight, with the usual ups and downs and suprising trials and tribulations. But Obama should win (and it may be in the long run a convincing win). Once your in the middle of the storm you can never look ahead and see the clear blue skies. But they will come again (they always do). We will withstand this current storm. So let's take it all in perspective and continue to Press the media for fairness and Obama for toughness. It's too early to toss out the white flag for this fearless, beloved, 1st time black candidate.

I dont' want to hear no more metaphors of:

"The Sky Is Falling, because Obama and our election are calling!"

Okay!

This next blog posting goes a long way to explaining the methodology used by the national polling services. And it explains why some of the party ID percentages may change over time.

From Nate Silver at FiveThirtyEightblog writes on polling methodology:

So let's get a few things straight:

1. Polls that show changes in party ID composition are not "cooked", "rigged", or "biased". Changes in party ID composition may occur for a variety of reasons, including random chance, response bias, temporary changes in party affiliation, and longer-term changes in party affiliation. It is possible that, if the partisan ID composition of a poll shifts rapidly from week to week (more so than is likely from random chance alone, and more so than is precipitated from external events), this may be an indictment of a pollster's methodology -- that the pollster is having difficulty getting a good, random sample. But the notion that credible pollsters like Gallup or SurveyUSA are deliberately rigging their samples is patently ridiculous. Polling is a very competitive industry, with relatively low barriers to entry; they would go out of business in a hurry if they did this.

2. The decision a pollster faces is whether or not to weight its sample by party ID. In fact, the whole point is that pollsters like Gallup and SurveyUSA do not weight their samples by party ID -- they just tally the results, and let the chips fall where they may. So in some sense accusing them of "cooking" their samples has it backward; what you're really arguing is that they should weight their samples, presumably in a way that is more favorable to your preferred candidate.

However, there is no one right answer as to whether to weight by party ID is the superior choice. A couple of pollsters, notably Rasmussen, do weight their samples by party ID. These pollsters tend to get more stable results from week to week. However, more stable does not necessarily mean more accurate over the long run, as these pollsters may miss real changes in party ID that occur over time.

Rasmussen, which targets its party ID figures based on a rolling, three-month sample of interviews with thousands and thousands of voters, has what I consider to be a eminently reasonable approach. Even so, the true party ID composition of the electorate is difficult and perhaps impossible to ascertain. You can take a survey of party ID, but it is still a survey, and therefore itself may be subject to issues like poor methodology and response bias.

There might be some promise in targeting party ID figures based on actual changes in voter registration -- although many states have open (non-partisan) voter registration, and some of those that do require registration by party do not report these figures. Some state-level pollsters seem to flirt with this approach, although I do not believe that any of the more prolific, national pollsters do.

3. Changes in party affiliation are tied up in the notion of the "convention bounce". Successful party conventions usually produce some shift in the self-reported partisan composition of surveys taken in the days and weeks that follow them. This may be a form of response bias, if voters of a particular partisan persuasion are fired up by their convention and more likely to take a pollster's phone call. (It may particularly be a form of response bias if the poll is taken during the convention itself, as voters of a particular party may be more likely to be at home watching their party's convention, and therefore more available to take a phone call).

However, remember what the conventions are. They are the Republican National Convention and the Democratic National Convention -- not the McCain National Convention and the Obama National Convention. They are big advertisements for the parties themselves. And so, it is not unlikely that, if voters respond favorably to a convention, they will respond favorably both to the candidate and to the candidate's party.

In other words, while some of these polls may have implausibly high numbers of self-reported Republicans in their samples, this is more a symptom of the convention bounce rather than a separate and distinct phenomenon. Moreover, conventions sometimes do trigger longer-term shifts in party identification. In 2004, for instance, the Republican convention appeared to cause a semi-permanent shift in party ID toward the Republicans, amounting to a couple of points. As I also argued recently, it is also plausible that unenthusiastic Republicans or Republican-leaning independents (which I call "shy Tories") were being undersampled prior to the Republican convention, until the convention renewed their enthusiasm.

The fact that the Bush administration is in its last days, that much of the dissatisfaction with the Republican Party had to do with George W. Bush, and that this is an election featuring two non-incumbents, is a salient factor here. It may be that, rather than trying to tie John McCain to George W. Bush, the Democrats should be making more of an effort to indict the Republican Party as a whole, of which John McCain is indisputably a member.

4. Responsible pollsters should always disclose the partisan ID composition of their samples, and should notate in their press releases when material shifts occur. Regardless of the above, there is little doubt that the partisan ID composition of a poll is an extremely important element in the proper interpretation of a poll. Pollsters should always report the partisan ID composition of their sample, and should qualify in their write-ups when substantial changes in party ID have occurred since their last survey.

-- Nate Silver at 9:48 AM

Charles H. Franklin does an excellent job of explaining how polling works and answers questions about the efficacy of the polling science. He even addresses a question on the Bradley Effect.

Political Arithmetik's Charles H. Franklin, a professor of political science at the University of Wisconsin, where he teaches statistical analysis of polls, public opinion and election results was on NPR on Wednesday. Listen here.

Read the full blog posting at Huffington Post here.

Read the full blog posting at DailyKos here.

Read the full blog posting at FiveThirtyEight here.

Read Time.com article on CNN/Time poll here.

plez sez: polls schmolls!

i'm not swayed by the polls, but there is no doubt that there is small population that may be swayed by them. what is disturbing is that the media probably uses skewed data to make a particular race seem closer in order to keep viewers engaged in the race. this year, the race in and of itself is compelling enough (first Black presidential candidate, war hero candidate, first republican woman candidate, etc.) without the questionable numbers that are bandied about on a nightly basis.

i don't feel better about polling methods, BUT i know now that i will tend to pay LESS attention to poll results (since they are so fluid, the margin of error is probably considerable higher than advertised, and the sample may not be representative of the voting population).