Statistics
Textbooks
Boundless Statistics
Sampling
Sample Surveys
Statistics Textbooks Boundless Statistics Sampling Sample Surveys
Statistics Textbooks Boundless Statistics Sampling
Statistics Textbooks Boundless Statistics
Statistics Textbooks
Statistics
Concept Version 5
Created by Boundless

How Well Do Probability Methods Work?

Even when using probability sampling methods, bias can still occur.

Learning Objective

  • Analyze the problems associated with probability sampling


Key Points

    • Undercoverage occurs when some groups in the population are left out of the process of choosing the sample.
    • Nonresponse occurs when an individual chosen for the sample can't be contacted or does not cooperate.
    • Response bias occurs when a respondent lies about his or her true beliefs.
    • The wording of questions--especially if they are leading questions-- can affect the outcome of a survey.
    • The larger the sample size, the more accurate the survey.

Terms

  • undercoverage

    Occurs when a survey fails to reach a certain portion of the population.

  • nonresponse

    the absence of a response

  • response bias

    Occurs when the answers given by respondents do not reflect their true beliefs.


Full Text

Probability vs. Non-probability Sampling

In earlier sections, we discussed how samples can be chosen. Failure to use probability sampling may result in bias or systematic errors in the way the sample represents the population. This is especially true of voluntary response samples--in which the respondents choose themselves if they want to be part of a survey-- and convenience samples--in which individuals easiest to reach are chosen.

However, even probability sampling methods that use chance to select a sample are prone to some problems. Recall some of the methods used in probability sampling: simple random samples, stratified samples, cluster samples, and systematic samples. In these methods, each member of the population has a chance of being chosen for the sample, and that chance is a known probability.

Problems With Probability Sampling

Random sampling eliminates some of the bias that presents itself in sampling, but when a sample is chosen by human beings, there are always going to be some unavoidable problems. When a sample is chosen, we first need an accurate and complete list of the population. This type of list is often not available, causing most samples to suffer from undercoverage. For example, if we chose a sample from a list of households, we will miss those who are homeless, in prison, or living in a college dorm. In another example, a telephone survey calling landline phones will potentially miss those who are unlisted, those who only use a cell phone, and those who do not have a phone at all. Both of these examples will cause a biased sample in which poor people, whose opinions may very well differ from those of the rest of the population, are underrepresented.

Another source of bias is nonresponse, which occurs when a selected individual cannot be contacted or refuses to participate in the survey. Many people do not pick up the phone when they do not know the person who is calling . Nonresponse is often higher in urban areas, so most researchers conducting surveys will substitute other people in the same area to avoid favoring rural areas. However, if the people eventually contacted differ from those who are rarely at home or refuse to answer questions for one reason or another, some bias will still be present.

Ringing Phone

This image shows a ringing phone that is not being answered.

A third example of bias is called response bias. Respondents may not answer questions truthfully, especially if the survey asks about illegal or unpopular behavior. The race and sex of the interviewer may influence people to respond in a way that is more extreme than their true beliefs. Careful training of pollsters can greatly reduce response bias.

Finally, another source of bias can come in the wording of questions. Confusing or leading questions can strongly influence the way a respondent answers questions.

Conclusion

When reading the results of a survey, it is important to know the exact questions asked, the rate of nonresponse, and the method of survey before you trust a poll. In addition, remember that a larger sample size will provide more accurate results.

[ edit ]
Edit this content
Prev Concept
Using Chance in Survey Work
The Gallup Poll
Next Concept
Subjects
  • Accounting
  • Algebra
  • Art History
  • Biology
  • Business
  • Calculus
  • Chemistry
  • Communications
  • Economics
  • Finance
  • Management
  • Marketing
  • Microbiology
  • Physics
  • Physiology
  • Political Science
  • Psychology
  • Sociology
  • Statistics
  • U.S. History
  • World History
  • Writing

Except where noted, content and user contributions on this site are licensed under CC BY-SA 4.0 with attribution required.