Home › Courses › Introduction to Statistics (STAT 101) › Collecting Data: Sampling & Experiments

Introduction to Statistics (STAT 101) · Module 3 of 7

Collecting Data: Sampling & Experiments

Sampling methods and bias, observational studies vs. experiments, and the principles of experimental design.

In this module
  1. Sampling Methods
  2. Sources of Bias
  3. Experiments vs. Observational Studies
  4. Check yourself

Sampling Methods

  • Simple random sample (SRS): every group of n individuals has an equal chance of being chosen.
  • Stratified: divide the population into similar groups (strata) and take an SRS from each, which reduces variability. Cluster: randomly select whole groups (clusters) and survey everyone in them, which is convenient. Systematic: every kth individual after a random start.
  • Convenience and voluntary-response samples are biased and can't be trusted to represent the population.
Worked example

A principal randomly picks 20 freshmen, 20 sophomores, 20 juniors and 20 seniors. Which method is this?

Solution. A stratified random sample, with grade levels as the strata.

Sources of Bias

  • Undercoverage: some groups are left out of the sampling frame. Nonresponse: selected people can't be reached or refuse.
  • Response bias: answers are inaccurate because of question wording, the interviewer, or a sensitive topic.
  • A larger sample reduces random variability but does NOT fix bias.
Worked example

A survey asks: "Don't you agree that the city's dangerous roads need repair?" What bias is present?

Solution. Response bias from leading question wording, which would likely overstate support for repairs.

Experiments vs. Observational Studies

  • An observational study records data without imposing treatments; it can show association but not causation. An experiment deliberately imposes treatments.
  • Principles of experimental design: comparison (including a control group), random assignment of treatments, replication (enough subjects), and control of other variables. Blinding and placebos reduce bias.
  • Random assignment allows cause-and-effect conclusions. Random selection (sampling) allows generalizing to the population.
Worked example

Volunteers are randomly assigned to a new drug or a placebo, and the drug group improves significantly more. What can we conclude, and about whom?

Solution. The drug caused the improvement (random assignment), but only for people like these volunteers, since they weren't randomly selected from a larger population.

Common mistake: Don't confuse random ASSIGNMENT (supports causation) with random SELECTION (supports generalization). AP questions test this distinction constantly.

Check yourself

2 of this module's 5 quiz questions. Pick your answers, then check them in the app, with an explanation for every question.

1. A researcher randomly selects 10 city blocks and surveys every household on them. This is:

2. An online poll that anyone can answer is most likely to suffer from:

Check my answers, free