Exit Poll Comments, Part I
As my regular readers know, my background is in marketing and I have some expertise in both creating and reading polls. Given the events of November 2, where the Democrats thought they were going to win based on exit polls and then were shocked when the actual results came in, I thought it useful to clarify a few things. I was motivated to write this post when I see people claiming the polls were right and Kerry was robbed. I have seen some bloggers, whom I will not embarrass with a link, dismiss the importance of the moral values response because they came from the "flawed" exit polls. These people are missing both the forest and the trees.

This is a subject with plenty of nuances. So I will break this into two parts. Today, I will discuss why the exit polls did not represent the actual voting population – that is, why the exit polls were wrong.

Exit polls are taken from actual voters and unless responders are deliberately lying, which I doubt, the information in the polls is useful information. However, what most people are missing is that the population polled in the exit surveys is not representative of the population that voted. This is true for many reasons.

First, the polling locations themselves are not randomly distributed amongst the voting locations. They are either positioned in urban areas (which tend to vote Democratic, and thus will bias their results) or in battleground districts which the forecasters hope will represent the state as a whole (and unless the forecasters guess perfectly, this also introduces ensures the population sampled will not directly represent the entire voting population).

Second, in many (perhaps all) states, women were overrepresented in the exit polls. As a whole, women tend to vote for Democratic candidates (to be very accurate, single women tend to vote overwhelmingly Democratic, while married women usually vote Republican) and this had a significant impact on the exit poll results. In other words, by having a greater percentage of women in the exit polls, compared to the percentage of actual female voters, the exit polls were biased toward the Democratic party. This is an obvious error that no experienced pollster should have made, thus leading to Republican claims of deliberate bias by the media pollsters.

Third, there a more subtle type of bias built into opinion polls. In general, socially conservative folks are less likely to complete opinion polls than are socially liberal folks. Since most social conservatives now vote Republican, this fact also biases opinion polls toward the Democrats. However, in political polls, this effect is probably much smaller than the effect of the other two (this is professional judgment on my part, I do not have hard data on the relative size of the three effects, but I am confident that effect two is quite large – it is a major problem with all types of polling).

Does this mean exit polls are useless? No, it just means they have to be used with some judgment. In general, exit polls are usually fairly accurate in picking the winner despite the common problems (the first and third effects previously discussed). Most years, the exit polls can predict the winner fairly well. People will only perceive problems when either other mistakes are added to the mix (such as overrepresenting women) or when the races are relatively close.

Thus, I can confidently say that the problems with this years exit polls can be easily explained by the known facts. Conspiracy types do not need to look for mysterious and complicated explanations. My biggest question is why did the media polls release numbers knowing that women were overrepresented – was it incompetence or bias? If any media types want someone to run their own exit polls next time and receive accurate results, contact me. I can guarantee my methods would be much more accurate than how things are currently done.

Next week I will discuss why the exit polls, despite their problems, are a very useful gauge in the minds of voters.

 
 
Comments

As I understand exit polls, they are _not_ designed to pick the winner a few hours earlier in the day. Everyone is very anxious, and uses them for that purpose, but that is dangerous because the polls are not designed with that in mind.

What I understand to be the purpose of US exit polling is to determine who voted which way and why.

For example you may want to know (as Q mentioned) if single women in urban areas vote with a different pattern from married women in urban areas. Or from single women in suburban areas.

Exit polls are designed to let you see the demographics of who voted for what, and to see what reasons they give (ie "I voted for Bush and the reason was concern over Terrorism." "I voted for Kerry and the reason was I am worried I may lose my job." "I voted for Nader because I he favors a Constitutional Amendment giving Redwood trees the right to vote." Things like that.

Posted by: Drew | 11/12/2004 - 12:34 PM

Good summary of original intent. Of course, people being people, they will continue to use exit polls to predict races.

Zogby was not too happy this year.

"There was a time you could go to the bank with the early exit polls," Zogby said. "Now we have a problem."

Zogby did very poorly in his predictions this year. If the early exit polls had been correct, he would have done much better. As it was, he had to eat much crow.

Posted by: Don Quixote | 11/16/2004 - 04:32 PM

"Zogby did very poorly in his predictions this year. If the early exit polls had been correct, he would have done much better. As it was, he had to eat much crow."

THAT explains it. I had heard that PETA was furious with Zogby, but I didn't understand why until now.


Posted by: Drew | 11/22/2004 - 04:26 PM

*groan* - OK, here's some crow for you.

After Quasimodo's death, the bishop of the Cathedral of Notre Dame sent word through the streets of Paris that a new bellringer was needed.

The bishop decided that he would conduct the interviews personally. He ascended into the belfry to begin the screening process. After observing several applicants demonstrate their skills, the bishop decided to call it a day when a lone, armless man approached him and announced that he was there to apply for the bellringer's job.

The bishop was incredulous. "You have no arms!" "No matter," said the man, "observe!" He then began striking the bells with his face, producing a beautiful melody on the carillon.

The bishop listened in astonishment, convinced that he had finally found a suitable replacement for Quasimodo. Suddenly, rushing forward to strike a bell, the armless man tripped, and plunged headlong out of the belfry window to his death in the street below.

The stunned bishop rushed to the dead man's side. When the bishop reached the street, a crowd had gathered around the fallen figure, drawn by the beautiful music they had heard only moments before. As they silently parted to let the bishop through, someone asked, "Bishop, who was this man?"

"I never learned his name," the bishop sadly replied, "but his face rings a bell."

Posted by: Don Quixote | 11/22/2004 - 05:02 PM

The following day, despite the sadness that weighed heavily on his heart due to the unfortunate death of the armless campanologist, the bishop continued his interviews for the bell-ringer of Notre Dame.

The first man to approach him said, "Your excellency, I am the brother of the poor, armless wretch that fell to his death from this very belfry yesterday. I pray that you allow me to honor his life by choosing me to replace him in this duty."

The bishop agreed to give the man an audition, and as the armless man's brother stooped to pick up a mallet to strike the first bell, he gasped, clutched at his chest and died on the spot.

Two monks, hearing the bishop's cries of grief at this second tragedy, rushed up the stairs to his side.

"What happened?" the first asked breathlessly. "Who is this man?"

"I don't know his name," wailed the distraught bishop, "but he's a dead ringer for his brother."

Posted by: Don Quixote | 11/22/2004 - 05:03 PM

Where I live they just opened a new Opera House. Actually it is a refurbished 1920's movie theater and very ornate.

I wanted to see the new venu (and an opera) but my wife said we had too many things on our schedule over the coming weekends through the holidays. We will miss Marriage of Figaro, and the first thing we can go to will be Carmen in early January.

That is the first weekend we are not already too Bizet.

Posted by: Drew | 11/22/2004 - 05:11 PM

You know that all potatoes have eyes, right? Well, Mr. and Mrs. Potato had eyes for each other. They finally got married and had a little one -- a real sweet potatoe whom they called "Yam." They wanted the best for little Yam, telling her all about the facts of life. They warned her about going out and getting Half Baked because she could get Mashed, get a bad name like Hot Potato, and then end up with a bunch of Tater Tots.

She said not to worry -- no Mr. McSpud would get her in the sack and make a Rotten Potato out of her! But she wouldn't stay home and become a Couch Potato either. She would get plenty of food and exercise so as not to be skinny like her Shoestring cousins. Mr. and Mrs. Potato even warned Yam, before sending her to Europe, to watch out for the Hard Boiled guys from Ireland and the greasy guys from France called the French Fries. They also said that she should watch out for the Indians, when going out west, because she could get Scalloped.

Mr. and Mrs. Potato wanted the best for Yam, so they sent her to "Idaho P.U." (Potato University) from which the Big Potatoes come; then, when she graduated, she'd really be in the Chips. But one day she came home and told them she was going to marry Dan Rather. Mr. and Mrs. Potato were very upset. They told her that she couldn't marry him, because he's just a . . .

Common Tater!

Posted by: Don Quixote | 11/23/2004 - 12:33 PM

Two shellfish, Sam Clam and Irving Oyster live together for years
in a seabed until a furious storm washed them ashore one day, where they die.
Being a very virtuous oyster, Irving ascends to Heaven and is given a seat
in the Celestial Orchestra, playing a harp.
Sam, being of a more disreputable nature is sent to Hell, but some is
fitting in with the other damned souls (soles?) as he opens a disco and
boogies the night away.
After several decades in the Celestial Orchestra, Irving is missing his
friend, so he goes to God and asks for a weekend pass to go visit Sam.
God takes down a BIG book from his BIG shelf and looks up Irving.
"Hmmm," he says, "you played very well and been an example to us all since
you arrived. I believe a pass can be arranged. I think you should take
your harp with you, too. Maybe hearing true Celestial music can show
some of those poor, damned souls the error of their ways. Just remember,
you must be back in Heave by the stroke of midnight on Sunday!"
Irving is overjoyed. Gathering up his harp he is zips down to Hell.
He and Sam have a brief, heartfelt reunion and then Sam gets Irving out
on the dance floor where he starts strutting his stuff. The weekend passes
in a whirl and it isn't until late Sunday night that the two manage to sit
down and reminisce about old times.
Sam happens to glance at he clock and sees that it is only 2 seconds to
midnight!
Irving rushes his goodbyes and zips back to Heaven, where he is met at the
Pearly Gates by God, who is looking at his BIG watch while tapping his BIG
sandal.
"You cut that awfully close, there, Irving," God says. "And what happened
to your harp?"
Irving pats all his pockets as he looks around wildly, finally offering;

"I left my harp in Sam Clam's Disco!"

(sung to the tune of "I left my heart in San Francisco")

Posted by: khobrah | 11/24/2004 - 08:22 AM
 
 
Send this Post
Email this entry to:


Your email address:


Message (optional):