Polling Error Rate and Reliability Do Not Speak To Reality

The news is filled with polling information just now.  You could take it seriously or you could ignore it.  Here are some aspects you might want to think about.

The current polling, (October 30) shows Hillary’s lead to be shrinking.  What was once a +14% in an ABC poll is now 6%.  Other polls show her well ahead, others show Trump well ahead.  How can that be?

Polling is based on the idea of a sample big enough to represent the entire population.  It says nothing about how any one person would vote but can say quite a lot about the totality.

There are several factors to consider:

  1. How confident do we want to be that the poll is accurate?  95% is usually the standard.  If your sample is drawn objectively only once in twenty will it fail to represent what the whole group would say.
  2. How big is the error rate or tolerance.  3% plus or minus is common. The precise 48.0% means any number between 45% and 51%.
  3. How big is the sample size.  Large samples tend to produce smaller error rates and higher confidence.

We know that some of the polling is biased even though it professes 95% confidence and a 3% error rate.  Often with large samples.  How do we know that?  The numbers tell us.  All the polls, if unbiased, should be within the error range.  If any lie outside the error range they are either the 1 in 20 that is wrong or they are biased somehow.

How would I bias a poll?  There are three common methods.

  1. The people who participate in the poll are not representative of the entire population is the easiest way.  Suppose I decide to do a poll on what the proper age to obtain a driver’s license should be.  It is 16 in Ontario.  If I ask the people in a representative survey, I will likely come out with something around 16.  But, suppose I ask only children age 10 to 14.  Do you think I will get 16?  Likely not.  Now matter how big the sample it will not represent the entire population.  In a voting poll, suppose I ask only those registered for one party or the other? Or, suppose I ask people without consideration of whether they will vote or not?  How do I control for lying?
  2. The question wording matters.  Double negatives confuse people.  Questions that imply an answer matter.  The means matter.  If I only talk to people who have a landline to answer the question, I will eliminate almost all young people.
  3. The method of analysis matters.  Are all the observations present in the analysis or are some discarded for whatever reason?  If you torture the data sufficiently, it will confess to whatever you want.

Polling is statistically sound but only if the surveyors ask clear questions, the participants have been selected randomly, and the data has been objectively analyzed.    In election times, polls may not be meaningful.  Some are used to make people feel part of a crowd.  Be cautious.  You cannot tell what they mean without knowing more about them.

Don Shaughnessy arranges life insurance for people who understand the value of a life insured estate. He can be reached at The Protectors Group, a large insurance, employee benefits, and investment agency in Peterborough, Ontario.  In previous careers, he has been a partner in a large international public accounting firm, CEO of a software start-up, a partner in an energy management system importer, and briefly in the restaurant business.

Please be in touch if I can help you.  don@moneyfyi.com  866-285-7772

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

%d bloggers like this: