[CQ-Contest] Re: Polling

[Lee Hiers]

On 11 Jan 2002, at 18:38, Dave_Hoaglin at abtassoc.com wrote:

> Thus, a scientific poll of QST readers would need responses from
> somewhat more than 1100 readers to give a margin of error of 3
> percentage points.

How do you determine a QST reader?  If you mean all ARRL members, you 
are assuming all are QST readers, which probably isn't valid.  

> A key point is the way in which the sample is obtained.  The margin of
> error that we've been quoting requires a random sample.  

Agreed.

> Not any old sample will be random (I'm avoiding "representative"
> because it is a loaded word). Getting a sample that is reliably
> random can take a lot of work.  That is where the professional
> polling organizations come in. 

Random is easy.

What is not easy is identifying the population which you want to poll 
and how you want to stratify it before testing.

Do you want the same weight given to opinions of people who spend 
more time operating, building stations, and buying equipment as those 
of people who may spend 5 hours a year operating?

The "science" of statistics is accepted.  For a given sample size 
from a given population, the results of a random sample will be 
correct with an X% error of margin.

The "art" of statistics is much more subjective.  That's where it is 
important to understand your population and how it relates to the 
information you're seeking.  Population, wording of questions, etc. 
could all be manipulated to produce the desired results in many (if 
not most) cases.  And still be within X% error of margin based on 
statistical theory.

> Putting a survey up on a web site and inviting people to read it and
> respond will almost surely produce a "sample" that is far from random.
> With no control over the sampling mechanism, it will not be possible
> to give any margin of error.  Worse yet, it will not be possible to
> say how the "sample" is related to the population, even though it may
> be much larger than the 1100 or mentioned above.  The "information"
> from such an activity should not be taken seriously.

No, it would not be random, but that does not make the information 
gained in that way necessarily less valid.  It depends on how you 
want to base your decision-making, and whether you're looking for 
justification of decisions already made or whether you're truly 
trying to find the best course of action.

Remember, it's easy to get in over your head in a puddle that 
averages only 1" deep...and that's based on a 100% sample!

73 de Lee

--
Lee Hiers, AA4GA
Cornelia, GA

lee at dixieliner.com


--
CQ-Contest on WWW: http://lists.contesting.com/_cq-contest/
Administrative requests: cq-contest-REQUEST at contesting.com