Post from S Nelson's Blog:
Correcting for Inaccuracy in Polls May be Inaccurate
Bad? Brilliant?
You can rate this post.
Register or login now and
tell us what you think.
As has been pointed out many times throught this presidential campaign season, polls are inaccurate. We get that. The reason that many people have blogged about going back to before the Iowa Caucus is that pollsters such as Rasmutten only dial land lines. So at best, the polls ONLY represent the opinions of randomly selected Americans who have land line telephones with listed numbers.

It's all well and good to point out that fact as a potential source of poll inaccuracy, but some go further to provide the demographics of people who are NOT being reached because of the land line issue, and to suggest that it is valid to correct the poll in a certain direction and by a certain amount - based on information NOT gathered.

I'm willing, for the sake of argument, because I have nothing better to do for the next 10 minutes, to assume that you can legitimately correct gathered information based on NON gathered information... I would STILL argue against a valid correction based on simply the land line error and nothing else. I'd say it's REALLY difficult to accurately poll people in Florida and Ohio who will become confused at the voting booth and vote for a can of beans for President, later claiming they couldn't understand the ballot. That's gotta be like, close to a 5% correction. I'd say it's pretty darned difficult to accurately correct for the the voting dead - Democrats who will vote in November despite being incorporeal. I expect they're planning to come out in numbers this year. I'd give that a 3% correction. And here's perhaps my favorite, I doubt if everyone answering pollsters are being 100% honest. Look, it's the age of telemarketers and other unsolicited phonecallers troubling us at the LEAST convenient times with things that "will only take a minute." There are people who will participate JUST for the opportunity of screwing with poll results as payback for the intrusion. I'd call that at least a 2% correction. Then there's a random, let's say... 5% correction needed for all the other possible errors that I didn't think of.

Let's work the math. I've heard people say 8% for the landlines, plus another 15% for the other things I mentioned.... multiply by 2 just cause it sounds good, and take the square root because I know how to do that too, and you come up with a number THAT MEANS NOTHING!!!

You can't adjust gathered information based on non gathered information.

Reader Comments
  
ANOTHER FACTOR
By AnneK Aug 21st 2008 at 10:18 am EDT (Updated Aug 21st 2008 at 10:18 am EDT)
Oh my God... I almost forgot to include my own inaccuracy demographic.

There are a lot of us who still expect to get to our voting booth in November, look at the ballot, and feel a feeling of warmth, light, and peace wash over us as we hear the calming refrains of the heavenly host singing... for when we look at our ballot, we will see the name Al Gore.

You can't poll us, because you don't KNOW what we'll do.... WE don't know what we'll do. And our name is Legion, for we are many.
Re: ANOTHER FACTOR
By AnneK Aug 21st 2008 at 11:09 am EDT (Updated Aug 21st 2008 at 11:09 am EDT)
  
I agree 100%
By Liz Aug 21st 2008 at 10:20 am EDT (Updated Aug 21st 2008 at 10:20 am EDT)
and in case you were referring to my suggestion. adjustment--that was purely a tongue in cheek assessment. In no way would I consider that accurate. LOL

My entire and main point is: THESE POLL ARE ABOUT AS ACCURATE AND RELIABLE IN THEIR PREDICTIONS AS ARE HOROSCOPES.

My 8% guesstimate is no more or less accurate than their polls. Both are just silly. But the unfortunate reality is that many people understand these poll results to be the true and accurate.

I think it is good to continue to point out from time to time how and why they are not accurate.

Once is not enough. This is the first time in over a month that I posted this reminder, but you can count on me to do it again in a few weeks.
Re: I agree 100%
By AnneK Aug 21st 2008 at 10:23 am EDT (Updated Aug 21st 2008 at 10:23 am EDT)
Well Heavens to Betsy, Liz, I'm glad you said that number was tongue in cheek!

I did use your 8%, but seriously, I've seen other people give demographics, and really try to put a numerical value on it.

That the polls are not totally accurate, and that there are many reasons why, I completely agree with.
Re: I agree 100%
By Barbara"Obama/Biden"Hussein Aug 21st 2008 at 10:49 am EDT (Updated Aug 21st 2008 at 10:49 am EDT)
lololol

I got lost reading and following along with you. Just kidding. lolol. You are right. We cant know how many. just that we and others we know have not been polled. Because we cant be reached by the old fashion ways of communication techniques. So oh well.

I will know for sure soon enough. I'm just waiting for time.

love
Re: I agree 100%
By Gretchen Aug 21st 2008 at 11:56 pm EDT (Updated Aug 21st 2008 at 11:56 pm EDT)
How many people on this site have been polled???? I sure haven't!
Re: I agree 100%
By Liz Aug 21st 2008 at 11:26 am EDT (Updated Aug 21st 2008 at 11:26 am EDT)
cool. I just wanted to correct any misunderstanding as I was totally mocking these polls which are owned and managed by conservatives. My numbers off the top of my head, their numbers, horoscope predictions--all the same.

For example, Scott Rasmussen who owns Rasmussen polls is a conservative evangelical who belongs to a religious group who have been called on the carpet more than once for discriminating against gays and blacks.

Gallup is owned by NBC which in turn is owned by GE, the largest war contractor in the world.
Re: I agree 100%
By Arius Aug 21st 2008 at 12:09 pm EDT (Updated Aug 21st 2008 at 12:09 pm EDT)
They're not as inaccurate as horoscopes. For one thing, the polls show Bob Barr and Ralph Nader have zero chance of winning. I think they're accurate on those two points.

Scientific polling can be extraordinarily accurate, but still not predictive. All it tells you is how people feel NOW, not how they will feel next week or next month. If a poll showed Obama down 10 points the night before the election, that would be alarming and I wouldn't be laying bets on Obama winning. However, if the polls in, say, August 2007 showed Obama down 30 points (just to pick a random date and margin), I wouldn't bet he would lose the primaries.
Re: I agree 100%
By Terry Aug 22nd 2008 at 1:47 am EDT (Updated Aug 22nd 2008 at 1:47 am EDT)
My old stats class, how I miss it.


nlycbyg
  
Good assumption
By Arius Aug 21st 2008 at 11:58 am EDT (Updated Aug 21st 2008 at 11:58 am EDT)
you can legitimately correct gathered information based on NON gathered information


This is actually true and a very important tool. (Quick, simple example: you're playing hide-and-seek, and you've searched the whole house except one room with two closets. You search one, and the person is not there. Based on all the information you have gathered, you know the person is in the other closet, even though you've not gathered that information yet.)
  
In your example, it's mostly true
By Arius Aug 21st 2008 at 12:05 pm EDT (Updated Aug 21st 2008 at 12:05 pm EDT)
You can't adjust gathered information based on non gathered information.

Based on your explanation, the answer here is that we don't know enough information to adjust for the non-gathered information. The uncollected information might make things even worse (or better, depending on your point of view)how an un-polled segment of the population feels.

For example: let's say that 10% of African Americans use cell-phones and not land-lines. It's possible that it so happens that if you're Black and Republican, you are almost certainly going to get rid of your land-line. So assuming you have 80% of that last 10% might well be false.

In order to be able to adjust for un-gathered data, one has to have sufficient gathered data to reduce the unknown unknowns into a manageable size.
Re: In your example, it's mostly true
By AnneK Aug 21st 2008 at 10:24 pm EDT (Updated Aug 21st 2008 at 10:24 pm EDT)
Now my brain hurts, thanks.
Have some willow water.
By Vidya Aug 22nd 2008 at 1:01 am EDT (Updated Aug 22nd 2008 at 1:01 am EDT)
It's not worth getting a headache over a poll, it's hard for me to get worked up about them, seems silly.
  
What I am
By Gretchen Aug 22nd 2008 at 3:21 am EDT (Updated Aug 22nd 2008 at 3:21 am EDT)
being told is that the polls are made to landlines only... Many of the under 30 crowd ONLY have cell phones... THIS IS OBAMA'S BASE!!! These people are not being polled!!!