
estimate a population proportion from a sample and develop a margin of error by simulation, and use a randomization test to compare two treatments in an experiment.
Instruction
Estimating with a margin of error (A2.DS.A.4 / A2.DS.A.5). A random-sample survey of 400 Missouri high school students finds that 248, or 62%, say they have a part-time job or regular paid work (illustrative figures). The population proportion is unknown. How far from 62% could it plausibly be?
Answer by simulation. Suppose the true proportion were 0.62, and imagine drawing many random samples of 400. Each sample gives its own proportion, and those proportions vary. Using technology, one set of 1,000 simulated samples produced proportions with a standard deviation of about 0.025, and 95% of them fell between about 0.568 and 0.670. The formula for this spread, √(p(1 − p)/n) = √(0.62 × 0.38/400) ≈ 0.0243, agrees. The margin of error is about 2 standard deviations, roughly 0.049 or 4.9 percentage points. So we estimate that between about 57% and 67% of the population have paid work. Reports often say “62%, plus or minus 5 points.”
Three things control the margin. Larger samples shrink it, but only as the square root: quadrupling the sample to 1,600 halves the margin. A margin of error measures only sampling variability; it does not account for bias from bad wording or nonresponse. And it describes the method: about 95% of intervals built this way capture the true proportion.
Comparing treatments (A2.DS.A.3). In an illustrative experiment, 20 volunteers were randomly assigned to study a list of 40 words for five minutes, 10 in silence and 10 with phone notifications sounding. Words recalled:
- Silence: 31, 28, 35, 30, 27, 33, 29, 32, 26, 34 (mean 30.5)
- Notifications: 25, 29, 24, 27, 30, 22, 26, 28, 23, 26 (mean 26.0)
The difference in means is 4.5 words. Could random assignment alone have produced a gap this big even if notifications had no effect? The randomization test answers directly. If the treatment made no difference, each person would have recalled the same number of words in either group, so the 20 scores are fixed and only the labels were random. Shuffle the 20 scores, deal 10 to each group, compute the difference in means, and repeat many times. In one run of 1,000 shuffles, only 2 differences were 4.5 or larger; checking every one of the 184,756 possible splits gives an exact proportion of about 0.0017. A difference this large almost never arises from the shuffling alone, so the data give strong evidence that notifications reduced recall for these volunteers. Because the treatments were randomly assigned, the conclusion is causal. Because the volunteers were not a random sample, generalizing beyond them needs care.
Reporting (A2.DS.A.7, MP.4). A good report names the design, gives the observed difference with units, describes the simulation, states the proportion of simulated results as extreme as observed, and draws a conclusion in the language the design supports. You will use exactly this structure in the unit’s performance task.
📖 Read for this lesson
The reading opens over the lesson; close it to come back to exactly where you were. The checks below draw on it.
Formative check
Work through these before moving on. They are not graded — they tell you whether the standard below has landed yet.
A survey’s margin of error is 4.9 points with n = 400. About what would it be with n = 1,600?
Simulation & tools
Simulating 1,000 surveys
In the blank CODAP document, create a table with one attribute sample and paste a column of the numbers 1 to 1,000 from a spreadsheet (CODAP accepts pasted CSV text). Add a second attribute phat with the formula randomBinomial(400, 0.62)/400. Drag phat to a graph, then use the ruler menu to show the mean and standard deviation. Record both, and count how many values fall below 0.57 or above 0.67. Question: does your simulated standard deviation agree with √(0.62 × 0.38/400) ≈ 0.024, and what margin of error would you report? Change 400 to 1,600 and describe the change. No device: use the class’s pooled results from drawing 40-card samples (62% marked) as a scaled-down version.
Open in a new tab ↗ · CODAP, Concord Consortium · MIT open source
In the word-recall experiment, a randomization proportion of about 0.0017 means notifications caused lower recall for every person.
Write the conclusion paragraph for the illustrative word-recall experiment as it should appear in a report: the design, the observed difference with units, the randomization result, the causal conclusion, and the limit on generalizing. Then name one change to the design that would let the conclusion apply to all students at your school.
How confident are you that you can simulate a margin of error and run and interpret a randomization test?
Practice
Work these on paper or in your notebook, then open Check your answer. Aim for all of Fluency and Application; try at least one Challenge.
Printable version: this unit’s practice workbook (PDF)
Fluency
Build speed and accuracy with the core skill.
- In a random sample of 1000, 289 said yes. Construct a 95% confidence interval for the population proportion.
Check your answer
Answer: p̂ = 0.289; margin ≈ 0.028; interval ≈ (0.261, 0.317). We are 95% confident that the proportion of the whole population who would say yes is between 0.261 and 0.317.z* ≈ 1.960. Conditions: a random sample; the sample is less than 10% of the population; successes and failures (289 and 711) are both at least 10. - In a random sample of 500, 244 said yes. Construct a 90% confidence interval for the population proportion.
Check your answer
Answer: p̂ = 0.488; margin ≈ 0.037; interval ≈ (0.451, 0.525). We are 90% confident that the proportion of the whole population who would say yes is between 0.451 and 0.525.z* ≈ 1.645. Conditions: a random sample; the sample is less than 10% of the population; successes and failures (244 and 256) are both at least 10. - In a random sample of 800, 255 said yes. Construct a 95% confidence interval for the population proportion.
Check your answer
Answer: p̂ = 0.319; margin ≈ 0.032; interval ≈ (0.286, 0.351). We are 95% confident that the proportion of the whole population who would say yes is between 0.286 and 0.351.z* ≈ 1.960. Conditions: a random sample; the sample is less than 10% of the population; successes and failures (255 and 545) are both at least 10. - In a random sample of 200, 62 said yes. Construct a 99% confidence interval for the population proportion.
Check your answer
Answer: p̂ = 0.310; margin ≈ 0.084; interval ≈ (0.226, 0.394). We are 99% confident that the proportion of the whole population who would say yes is between 0.226 and 0.394.z* ≈ 2.576. Conditions: a random sample; the sample is less than 10% of the population; successes and failures (62 and 138) are both at least 10. - In a random sample of 1000, 486 said yes. Construct a 95% confidence interval for the population proportion.
Check your answer
Answer: p̂ = 0.486; margin ≈ 0.031; interval ≈ (0.455, 0.517). We are 95% confident that the proportion of the whole population who would say yes is between 0.455 and 0.517.z* ≈ 1.960. Conditions: a random sample; the sample is less than 10% of the population; successes and failures (486 and 514) are both at least 10. - Find the margin of error (about 2 standard deviations) for p̂ = 0.62 with n = 400, and with n = 1,600.
Check your answer
Answer: 0.0485; 0.0243Quadrupling n halves the margin. - Find the margin of error for p̂ = 0.5 and n = 1,000.
Check your answer
Answer: 0.0316, about 3.2 points
Application
Use the skill in context. Show your reasoning.
- A survey (illustrative) of 1,500 Missouri adults finds 58% support, with a reported margin of ±3.3 points. Compute 2√(p(1 − p)/n) and explain any difference. Does the survey establish majority support?
Check your answer
Answer: About 2.5 points; weighting and design widen published margins. Yes: 54.7% to 61.3% is above 50%. - In a randomized experiment, silence scores averaged 30.5 words and notification scores 26.0. In 1,000 shuffles, 2 differences were 4.5 or more. Write the conclusion.
Check your answer
Answer: Strong evidence that notifications lowered recall on average for these volunteers; not proof, and not generalizable beyond them without random selection. - A poll finds 52% ± 3 points support. A headline says ‘Majority supports.’ Evaluate it.
Check your answer
Answer: Not established: the interval 49% to 55% includes values below 50%.
Challenge
Stretch problems. Expect to think before you write.
- How large a sample gives a margin of error of 3 percentage points when p̂ is near 0.5?
Check your answer
Answer: About 1,112 people2√(0.25/n) = 0.03 gives n = 1,111.1; round up.
Review
Keep earlier skills sharp.
- Solve for x: 10x = 250. Give an exact answer using logarithms and a decimal to the nearest thousandth.
Check your answer
Answer: x = log 250 / log 10 ≈ 2.398Take the logarithm of both sides and use the power property. - Find the exact value of cos(4π/3).
Check your answer
Answer: −1/2Use the reference angle and the sign of the function in that quadrant.
A Clear Majority, a Lead Without One, and a Tie: What a Reported Margin of Error Does and Does Not Cover
SLU/YouGov Poll, August 2026, Top Line Results: 900 likely Missouri voters surveyed August 13-24, 2026 by YouGov (1,276 registered voters interviewed, matched to 1,074); margin of error 4.09% for the full sample; credible interval plus or minus 4.1 points; estimated design effect 1.25, effective sample size 720; Saint Louis University. Retrieved November 5, 2026 · SLS original, with figures as published by Saint Louis University and YouGov
Why this reading. The lesson’s 62%-with-a-part-time-job survey is illustrative and says so. Real surveys publish a margin of error computed a different way from the formula you are about to learn, and the gap between the two is where the interesting argument lives. Here is one published Missouri survey, its method, its numbers, and three claims that could be made from it.
The survey
| Item | As published |
|---|---|
| Organization | Saint Louis University and YouGov, the SLU/YouGov Poll |
| Fieldwork | August 13–24, 2026 |
| Mode | Online, by YouGov |
| Who was interviewed | 1,276 Missouri registered voters, matched down to a sample of 1,074, yielding 900 likely voters |
| How the sample was made to look like Missouri | Respondents matched to a frame drawn from the Missouri part of the 2024 Current Population Survey on gender, age, race and education; then weighted with propensity scores and post-stratified on 2024 presidential vote, gender, age, race and education |
| Reported margin of error, full sample | ±4.09 percentage points (“a credible interval of ±4.1 percentage points”) |
| Reported design effect | 1.25, “yielding an effective sample size of 720” |
| Questions asked of half the sample | about 450 respondents each, with margins of ±5.67 or ±5.92 points |
Three results, and a claim from each
| Question (likely voters) | Asked of | Support / first answer | Oppose / second answer | Not sure |
|---|---|---|---|---|
| A–F grades (like a report card) for Missouri public schools and districts: support / oppose | 900 | 60% | 19% | 21% |
| Amendment 6, the process for initiative petitions: support / oppose | 900 | 48% | 21% | 32% |
| Rating of the public schools in your community: fair / good | 900 | 33% fair | 32% good | 4% (7% excellent, 24% poor) |
Three claims a headline could make: “A majority of Missouri voters want A–F grades for public schools”; “A majority of voters back Amendment 6”; “More Missourians rate their local schools fair than good.” Those are the claims to test.
Check the margin against the formula
The lesson’s formula for a 95% margin on a proportion is 2√(p(1 − p)/n), largest when p = 0.5. With n = 900:
2√(0.25/900) = 2(0.016667) = 0.03333, or 3.33 percentage points.
SLU reports 4.09. The formula is not wrong and neither is SLU; they are answering slightly different questions. The formula assumes a simple random sample in which every respondent counts once. This poll is weighted to match Missouri’s voters, and weighting makes some respondents count for more than others. That inflates the variance by a factor called the design effect. The poll states its own design effect, 1.25, and an effective sample of 900/1.25 = 720; the formula with n = 720 gives 2√(0.25/720) = 3.73 points, closer but still short of 4.09. Working backward from the published margin instead, (4.09/3.333)2 = 1.51, as if the 900 weighted respondents carried the information of 0.25/(0.0409/2)2 = 598 people in a simple random sample. The rest of the gap comes from the way YouGov computes its interval (it calls it a credible interval, a term from a different branch of statistics), which the simple formula does not model. The lesson for you: the formula gets the size right, and a published margin is usually somewhat larger, because it carries adjustments the formula cannot see.
The half-sample questions show the same thing. With n = 449, 2√(0.25/449) = 4.72 points against a published 5.92; with n = 450, 4.71 against 5.67.
Margins are smaller for percentages far from 50%. The formula depends on p:
| p | 0.50 | 0.60 | 0.48 | 0.33 | 0.32 | 0.21 | 0.19 |
|---|---|---|---|---|---|---|---|
| 2√(p(1−p)/900), points | 3.33 | 3.27 | 3.33 | 3.13 | 3.11 | 2.72 | 2.62 |
A reported margin of error is almost always the maximum, at p = 0.5. Near 50% that hardly matters; for the 19% opposing A–F grades it overstates the uncertainty noticeably.
Testing the three claims
For a gap between two answers to the same question, the question is not whether each answer’s interval is wide. It is whether the difference is larger than the uncertainty in that difference — a different quantity, because the two percentages come from the same respondents and move against each other. For two shares p1 and p2 from one sample, the standard error of the difference is √((p1 + p2 − (p1 − p2)2)/n).
- A–F grades, 60% support against 19% oppose. √((0.60 + 0.19 − 0.412)/900) = √(0.6219/900) = 0.02629, so 2.63 points; twice that is 5.26 points, and with the design effect (×√1.25) 5.88. The observed gap is 41 points, far outside. And the majority holds on its own: 60 ± 4.09 runs from 55.9% to 64.1%, entirely above 50%. This claim is well supported.
- Amendment 6, 48% support against 21% oppose. Support leads opposition by 27 points, and twice the standard error of that difference is only 5.24 points (5.86 with the design effect), so “more support it than oppose it” is clearly established. A majority is not: 48 ± 4.09 runs from 43.9% to 52.1%, which includes values below 50%, and 32% said they were not sure. Same survey, two claims about the same question, different verdicts.
- Local public schools, 33% fair against 32% good. √((0.33 + 0.32 − 0.012)/900) = √(0.6499/900) = 0.02687, so twice it is 5.37 points (6.01 with the design effect) against a 1-point gap. “More rate their schools fair than good” is not supported; the honest sentence is that the survey cannot tell the two answers apart.
What the margin of error does not cover at all
A margin of error measures sampling variability and nothing else. Question wording, question order and the timing of the fieldwork can all move results; so can the people who never join an online panel, the people who join and do not answer, and — for a pre-election poll — the question of who actually turns out to vote, which no margin of error addresses. This poll shows the wording effect directly: one half of the sample was asked about a data center in their own community before one in the state, the other half in the opposite order, and the answers differ (17% against 20% support for one in their own community, 29% against 23% for one in Missouri). A margin of error is a statement about one source of variation, computed on the assumption that everything else went right.