
distinguish a parameter from a statistic, describe and compare sampling methods, and identify the biases that can make a sample unrepresentative.
Instruction
Statistics lets you learn about a large group from a small part of it, but only if the part is chosen well. A2.DS.A.1 frames the whole idea: a parameter is a number describing a population (usually unknown); a statistic is a number computed from a sample, used to estimate the parameter. Inference is the reasoning from statistic to parameter.
Sampling methods.
- Simple random sample (SRS): every group of n individuals has the same chance of being chosen. Number the population, then use a random number generator.
- Stratified random sample: divide the population into groups that are similar inside (strata, such as grade levels or regions of a state), take an SRS from each, and combine. This guarantees each group is represented and usually gives more precise estimates.
- Cluster sample: divide the population into groups that each look like the population (such as classrooms or city blocks), randomly choose some clusters, and include everyone in them. It is cheaper, but usually less precise.
- Systematic sample: choose a random start, then every k-th individual on a list.
- Convenience and voluntary response samples (whoever is easy to reach, or whoever chooses to answer) are not random, and they cannot support inference.
Bias is a systematic tendency to overestimate or underestimate. It comes from the design, and a larger sample does not fix it.
- Undercoverage: parts of the population are missing from the sampling frame (the list you sample from).
- Nonresponse: chosen individuals do not respond, and they differ from those who do.
- Response bias: answers are inaccurate because of wording, the interviewer, or a wish to give the socially acceptable answer.
A famous failure. In 1936 the Literary Digest magazine mailed about 10 million straw ballots and received about 2.4 million back (2,376,523, a response rate of about 24%). It predicted that Alf Landon would defeat Franklin Roosevelt, with 54% of the vote. Roosevelt won in a landslide, with about 61% of the popular vote and 523 of 531 electoral votes. The mailing lists were built largely from telephone directories and automobile registration lists, which leaned toward better-off households in the Depression (undercoverage), and only about one ballot in four came back, so the people who chose to mail a ballot back mattered a great deal (nonresponse). Two point four million responses could not rescue a biased design. The reading for this lesson prints these counts and says where each comes from.
A careful design. The Census Bureau’s American Community Survey (ACS) contacts about 3.5 million addresses each year, chosen by random sampling from the Bureau’s master address file. Response is required by law, and the Bureau follows up by mail, internet, phone and in-person visits to reduce nonresponse. The ACS publishes one-year estimates for areas of 65,000 people or more, and five-year estimates for all areas, down to small neighborhoods and rural counties. In Missouri alone it selects about 77,000 addresses a year. Every estimate comes with a margin of error, an honest statement of how far the statistic might plausibly be from the parameter because of random sampling alone. You will simulate where a margin of error comes from in Lesson 3.2.
Evaluating a claim (A2.DS.A.7). Before trusting a percentage from a survey, ask: Who was the population? What was the frame? How were people chosen? What fraction responded? What exactly was asked? A large, non-random sample is still non-random.
📖 Read for this lesson
The reading opens over the lesson; close it to come back to exactly where you were. The checks below draw on it.
Formative check
Work through these before moving on. They are not graded — they tell you whether the standard below has landed yet.
A district wants students’ views on a new schedule and wants each grade level represented in proportion. Which design fits best?
The Literary Digest’s 1936 poll failed because 2.4 million responses was too small a sample.
Parameter
Tap to revealDescribes the population; usually unknown (p, μ).
Statistic
Tap to revealComputed from a sample (p̂, x̄); estimates a parameter.
Undercoverage
Tap to revealSome of the population is missing from the frame.
Voluntary response
Tap to revealPeople choose themselves; strong opinions are over-represented.
Cluster sample
Tap to revealRandomly choose whole groups and include everyone in them.
Find a news story or website that reports a percentage from a survey or poll. Identify the population, the sampling method (if stated) and the likely sources of bias, and state what the report leaves out. Then propose a better design in four sentences. If you cannot find one, critique this hypothetical example, invented for this lesson: ‘An online poll on a sports website found that 78% of respondents support building a new stadium in their city.’ Check your answer against the five questions in the reading, or ask a parent or guardian to read your four sentences and say whether the design would convince them.
How confident are you that you can tell a parameter from a statistic, choose a sampling method and name the biases a design invites?
Practice
Work these on paper or in your notebook, then open Check your answer. Aim for all of Fluency and Application; try at least one Challenge.
Printable version: this unit’s practice workbook (PDF)
Fluency
Build speed and accuracy with the core skill.
- (Hypothetical data.) A survey of 400 randomly chosen households in a Missouri city finds that 23% have solar panels. Identify the population, the parameter, the sample and the statistic.
Check your answer
Answer: Population: all households in that city. Parameter: the proportion of all those households with solar panels (unknown). Sample: the 400 households surveyed. Statistic: p̂ = 0.23. - Parameter or statistic? The mean daily high of the 30 randomly chosen days in a sample.
Check your answer
Answer: Statistic.It describes the sample. - Parameter or statistic? The proportion of all 9,132 days from 2000 to 2024 on which St. Louis Lambert recorded rain.
Check your answer
Answer: Parameter (of that population of days). - Name the sampling method: a school lists all 1,200 students, picks a random start between 1 and 20, and surveys every 20th student.
Check your answer
Answer: Systematic sampling. - Name the sampling method: a district randomly selects 6 of its 40 homerooms and surveys every student in them.
Check your answer
Answer: Cluster sampling. - Name the sampling method: a county takes a separate simple random sample of voters in each of its five townships.
Check your answer
Answer: Stratified random sampling. - Name the sampling method: a reporter interviews shoppers at one mall on a Saturday.
Check your answer
Answer: Convenience sample (not random). - Does increasing the sample size reduce bias?
Check your answer
Answer: No. Larger samples reduce random variability, not bias from the design.
Application
Use the skill in context. Show your reasoning.
- (Hypothetical.) A radio station asks listeners to text whether the city should build a new stadium; 78% of the 2,300 texts say yes. Identify the type of sample and the likely direction of bias.
Check your answer
Answer: A voluntary-response sample. People with strong opinions (likely enthusiasts) are over-represented, so support is probably overestimated. - A phone survey of a county’s residents uses a list of landline numbers only. What is the bias, and who is likely missing?
Check your answer
Answer: Undercoverage: households without landlines (more often younger and lower-income residents) cannot be selected, so the sample may not represent them. - A survey question reads: ‘Given the dangerous condition of our roads, do you support a small tax to fix them?’ Identify the problem and rewrite the question.
Check your answer
Answer: Loaded wording (response bias). Neutral version: ‘Do you support or oppose a 0.25% sales tax for road repairs?’ - The American Community Survey requires households to respond by law and follows up by mail, internet, phone and in-person visits. Which kind of bias is this design mainly fighting?
Check your answer
Answer: Nonresponse bias.Follow-up raises the response rate among groups less likely to respond at first.
Challenge
Stretch problems. Expect to think before you write.
- A district of 3,000 high-school students (four grades of roughly equal size) wants to estimate average nightly sleep, and suspects seniors sleep less. Design a stratified sample of 200 and explain why it beats a simple random sample of 200.
Check your answer
Answer: Take a simple random sample of 50 students from each grade (random numbers on each grade’s roster). Stratifying guarantees each grade is represented and, because sleep differs by grade, reduces the variability of the combined estimate compared with an SRS, which could by chance over- or under-represent seniors.
Review
Keep earlier skills sharp.
- Use technology to find the least-squares line and the correlation coefficient for (1, 8), (2, 3), (4, 6), (11, 12), (12, 16), (13, 13).
Check your answer
Answer: ŷ = 4.037 + 0.785x; r = 0.879Interpret the slope in context: the predicted change in y for each one-unit increase in x. - (Unit 2) A regression of y on x has r = −0.6. What fraction of the variation in y is explained by the linear model?
Check your answer
Answer: r² = 0.36, so 36%. - (Unit 1) Which summary is resistant: the SD or the IQR?
Check your answer
Answer: The IQR.
Ten Million Ballots and 3.5 Million Addresses: Two Samples, Read by Their Numbers
Literary Digest 1936 returns as quoted in Lohr and Brick, Statistics, Politics and Policy 8(1), 2017 (doi 10.1515/spp-2016-0006) and arXiv 2107.14166; 1936 electoral vote from the National Archives; 1936 popular vote as tabulated on Wikipedia; ACS initial addresses and final interviews for Missouri 2005-2025 and the U.S. 2019-2025 from the Census Bureau’s Sample Size tables; retrieved November 10, 2026 · SLS original, with public-domain figures from the U.S. Census Bureau and the National Archives and published figures quoted from the cited papers
Why this reading. The lesson tells the Literary Digest story and describes the American Community Survey in words. A design is judged by its numbers: how many were asked, how many answered, and who was on the list. Here are the numbers for both, from published accounts of the Digest poll and from the Census Bureau’s own sample-size tables for Missouri and the nation, so the failure and the careful design can be compared line by line.
The Literary Digest, 1936
The figures below come from the magazine’s own final tally as quoted by later researchers, and are checked against two papers: Sharon Lohr and J. Michael Brick, “Roosevelt Predicted to Win: Revisiting the 1936 Literary Digest Poll” (Statistics, Politics and Policy 8(1), 2017), and a review of poll failures posted on arXiv (2107.14166). The magazine’s headline for its last report read “Landon, 1,293,669; Roosevelt, 972,897” (Literary Digest, October 31, 1936, as listed in Lohr and Brick’s references). The classic study of why the poll failed, Peverill Squire’s “Why the 1936 Literary Digest Poll Failed” (Public Opinion Quarterly 52, 1988), could not be opened for this reading; its conclusion is taken from how those two papers summarize it, and none of its own figures are used.
| Item | Figure |
|---|---|
| Ballots mailed | about 10,000,000 |
| Sampling frame | telephone directories and automobile registration lists, plus other lists (Lohr and Brick) |
| Ballots returned | 2,376,523 (the count quoted in the arXiv review; Lohr and Brick say “almost 2.4 million”) |
| Returned for Landon / for Roosevelt | 1,293,669 / 972,897 (the Digest’s headline); the other 109,957 returned ballots, found by subtraction, went to other candidates |
| Response rate | 24% (Lohr and Brick); 2,376,523 / 10,000,000 = 23.8% |
| Digest’s final poll shares | Landon 54.4%, Roosevelt 40.9% (Lohr and Brick: “54 percent” for Landon) |
| Election result (electoral vote) | Roosevelt 523, Landon 8 (National Archives, Electoral College results 1936) |
| Election result (popular vote) | Roosevelt 27,752,648 (60.8%), Landon 16,681,862 (36.5%) of 45,647,699 (as tabulated in the Wikipedia article on the 1936 election, a secondary source; the National Archives page lists only electoral votes) |
Check the arithmetic: 1,293,669 / 2,376,523 = 54.4% for Landon among the returned ballots, against 36.5% of the actual vote: a miss of 17.9 points, in a poll with 2.4 million answers. Lohr and Brick describe two things going wrong together: the lists leaned toward households with telephones and cars (undercoverage of everyone else), and only about one ballot in four came back, so who chose to answer mattered a great deal (nonresponse). They also show that if the answers had been weighted by how respondents said they voted in 1932, the poll would have favored Roosevelt; the Digest did not do that. The lesson’s round figures, about 2.4 million returned and about 61% for Roosevelt, are these numbers rounded.
The American Community Survey: how many asked, how many answered
The Census Bureau publishes, for every state and year, the number of housing-unit addresses initially selected for the ACS sample and the number of final interviews obtained. Missouri, from the Bureau’s Sample Size table (retrieved November 10, 2026):
| Year | Initial addresses selected | Final interviews | Interviews as % of addresses |
|---|---|---|---|
| 2025 (note 1) | 70,113 | 38,000 | 54.2% |
| 2024 | 77,590 | 43,807 | 56.5% |
| 2023 | 77,660 | 44,761 | 57.6% |
| 2022 | 77,634 | 45,018 | 58.0% |
| 2021 | 77,100 | 42,878 | 55.6% |
| 2020 (note 2) | 62,494 | 30,567 | 48.9% |
| 2019 | 77,488 | 45,010 | 58.1% |
| 2018 (note 3) | 77,500 | 46,500 | 60.0% |
| 2017 | 76,902 | 46,356 | 60.3% |
| 2016 | 76,981 | 47,975 | 62.3% |
| 2015 | 77,566 | 50,265 | 64.8% |
| 2014 | 77,505 | 50,595 | 65.3% |
| 2013 (note 4) | 77,667 | 48,353 | 62.3% |
| 2012 | 77,524 | 52,260 | 67.4% |
| 2011 | 74,478 | 48,856 | 65.6% |
| 2010 | 64,643 | 43,440 | 67.2% |
| 2009 | 64,775 | 43,390 | 67.0% |
| 2008 | 64,995 | 43,767 | 67.3% |
| 2007 | 64,541 | 43,942 | 68.1% |
| 2006 | 64,434 | 44,640 | 69.3% |
| 2005 | 64,438 | 43,493 | 67.5% |
And for the United States as a whole, the most recent seven years:
| Year | Initial addresses selected | Final interviews | % |
|---|---|---|---|
| 2025 (note 1) | 3,215,511 | 1,686,000 | 52.4% |
| 2024 | 3,543,672 | 1,944,002 | 54.9% |
| 2023 | 3,544,357 | 1,978,704 | 55.8% |
| 2022 | 3,538,392 | 1,980,550 | 56.0% |
| 2021 | 3,538,442 | 1,950,832 | 55.1% |
| 2020 (note 2) | 2,872,402 | 1,406,935 | 49.0% |
| 2019 | 3,544,301 | 2,059,945 | 58.1% |
The Bureau’s notes. (1) The lapse in appropriations in October and November 2025 suspended data collection and lowered both the sample and the interview counts for 2025, and the 2025 counts are rounded. (2) The pandemic reduced the number of addresses in sample and the number of interviews in 2020. (3) The 2018 counts were rounded to reduce disclosure risk. (4) The 2013 government shutdown removed the second mailing and the telephone and in-person follow-up for the October 2013 panel, which cut final interviews. Source: Census Bureau, ACS Sample Size page.
Read the Missouri column downward. The sample was about 77,000 to 78,000 addresses a year from 2012 to 2024, but interviews fell from 52,260 in 2012 to 43,807 in 2024, and the share of selected addresses that end in an interview slid from 67% to 56%. Some of the gap is vacant or nonexistent addresses; the rest is nonresponse, which the Bureau fights with mail, internet, telephone and in-person follow-up, and with the legal requirement (Title 13 of the U.S. Code) to respond. The Bureau then weights the interviews to match known population totals. A falling response rate does not make an estimate wrong, but it makes the weighting do more work, and it is the reason the ACS publishes a margin of error with every number. One-year estimates are published for areas of 65,000 people or more; five-year estimates cover every area, down to small neighborhoods and rural counties.
The difference in one sentence
The Digest reached ten million people it chose for convenience and heard back from only about one in four; the ACS reaches 3.5 million addresses chosen at random from a list of every address, follows up the ones that do not answer, and reports how far off it might still be.
What to do
- For the Digest poll, name the population, the frame, the sampling method, the response fraction and the two biases, each with a number from the table.
- For the ACS, compute Missouri’s 2020 interview percentage (30,567 / 62,494) and explain from the Bureau’s note why it is the lowest in the Missouri table.
- Now critique the lesson’s hypothetical sports-site poll (“78% of respondents support building a new stadium”) with the same five questions, and propose a design in four sentences that borrows at least two features from the ACS.
Sources: U.S. Census Bureau, American Community Survey: Sample Size (Missouri) (census.gov); U.S. Census Bureau, American Community Survey: Sample Size (United States) (census.gov); National Archives, Electoral College results: 1936 (archives.gov); S. Lohr and J. M. Brick, Roosevelt Predicted to Win: Revisiting the 1936 Literary Digest Poll, Statistics, Politics and Policy 8(1):65-84 (2017) (doi.org); Why Do Polls Fail? The Case of Four US Presidential Elections, Brexit, and Two India General Elections (arXiv 2107.14166) (arxiv.org); 1936 United States presidential election (Wikipedia; popular vote table) (en.wikipedia.org); P. Squire, Why the 1936 Literary Digest Poll Failed, Public Opinion Quarterly 52:125-133 (1988) (not opened; cited as summarized by the two papers above) (doi.org); U.S. Census Bureau, ACS: When to use 1-year or 5-year estimates (census.gov)