
design and carry out a sample survey or randomized experiment, analyze the data with simulation, and report conclusions with their proper scope.
Instruction
The task. Design, run and report your own study. Choose one of two routes.
Route A: randomized comparative experiment. Pose a question about whether a treatment changes a response. Examples that are safe and quick: Does a notification sound change reaction time on a ruler-drop test? Does font style change how many words are recalled? Do paper helicopters with longer blades stay aloft longer? (Here the “units” are helicopters, so no people are needed.) You need at least 20 units, a control, random assignment by a random number generator (not by choice), a response measured the same way for everyone, and blinding if you can arrange it. A matched-pairs design, in which each person does both treatments in a randomly chosen order, is allowed. Then you analyze the differences.
Route B: sample survey. Pose a question about a proportion or mean in a defined population (for example, the proportion of the students in your homeschool group or online class who get at least 8 hours of sleep on school nights). Define the population and frame, choose a random sampling method (SRS or stratified), plan for nonresponse, and write neutral questions. Estimate the parameter and a margin of error by simulation, as in Lesson 3.2.
Ethics and privacy, required. Participation is voluntary, and anyone may decline or withdraw. Collect no names with responses. Ask no questions about health, family, immigration status, grades or other sensitive topics. A parent or guardian reads and signs your plan before you collect anything. A student who prefers not to collect data from people may use the paper-helicopter experiment or analyze samples from the St. Louis NOAA daily dataset of Lesson 3.2 (Route B with days as the population).
Analysis. Summarize each group with a display and statistics suited to the shape. For Route A, carry out a randomization test with at least 500 shuffles and report the P-value (as in Lesson 3.4). For Route B, report p̂ or x̄ and a simulated margin of error.
Scope of inference, the heart of the report. State both halves: to whom can you generalize (was there random selection from a population?), and can you conclude cause and effect (was there random assignment?). Most student experiments have random assignment but not random selection: causal for these participants, with generalization a judgment. Most surveys have random selection but no assignment: they describe a population and show no causation.
What the design defense tests. The defense answers one question about a possible bias or confounder, which you pick from the list in the checklist below (or a parent or guardian asks). Answer it by pointing to the specific design feature that addresses it, or by admitting the limitation.
📖 Read for this lesson
The reading opens over the lesson; close it to come back to exactly where you were. The checks below draw on it.
Formative check
Work through these before moving on. They are not graded — they tell you whether the standard below has landed yet.
A student randomly assigns 24 volunteer friends to two study methods and finds a significant difference. What can be concluded?
A survey asking ‘Don’t you agree that the unfair new schedule should be changed?’ is a neutral question.
Stage 1 plan: state your route, question, units or population, treatments or sampling method, exactly how you will randomize (with the tool), your response variable and how it is measured, two bias safeguards, and your ethics plan.
How confident are you that you can design, run and report a study, stating exactly what its design lets me conclude?
Unit 3 checkpoint
Do this before you start the task. It covers the whole unit and tells you what to review first.
A statistic is…
A school randomly picks 5 classrooms and surveys every student in them. This is a…
A voluntary-response sample of 10,000 is more trustworthy than a random sample of 500.
If samples of 100 give a margin of error of 0.092, samples of 400 give about .
A margin of error includes errors from nonresponse and question wording.
Which study can show cause and effect?
Random selection allows you to…
An inactive treatment given to a control group is a .
A randomization test gives P = 0.013. Which is correct?
A P-value of 0.40 proves the treatment has no effect.
Students choose whether to join a program, and joiners score higher. The main problem is…
To halve a margin of error, multiply the sample size by .
Submit: Study report (1,200-1,500 words) + four-minute design defense
Task: Design and carry out a randomized experiment (Route A) or random sample survey (Route B), analyze it with simulation, and report the conclusions with their scope. Tools: CODAP (Sampler and Scrambler plugins), a free spreadsheet, or cards and dice. A parent or guardian must sign your plan before data collection.
Submission workflow
- Stage 1 — Design plan. Question, route, units or population, treatments or sampling method, randomization procedure, bias controls and the ethics plan; a parent or guardian reviews and signs the plan before you collect anything.
- Stage 2 — Self-checked draft. After collecting data, submit a draft. Hunt for one bias or confounder in your own design, check your simulation against the model studies in the reading, and write down what you found and your revision plan. A parent or guardian may read the draft too.
- Stage 3 — Final report. 1,200-1,500 words with design, data table, displays, simulation result and a scope-of-inference paragraph.
- Stage 4 — Design defense. Four minutes, recorded or written, answering a question about bias or confounding.
How it is scored
| Criterion | What proficient looks like |
|---|---|
| Study design (A2.DS.A.3) | Appropriate route; randomization clearly described; control, replication and bias safeguards in place. |
| Population and sampling (A2.DS.A.1) | Population, frame or units defined; sampling or assignment method justified. |
| Analysis (A2.DS.A.2, A2.DS.A.3, A2.DS.A.4, A2.DS.A.5) | Correct summaries; randomization test or simulated margin of error carried out and interpreted. |
| Model reasoning (A2.DS.A.2) | Explains what the null model or simulation represents and whether the data are consistent with it. |
| Scope and critique (A2.DS.A.7) | Accurate statement of generalization and causation; limitations named honestly. |
| Ethics and communication (MP.4) | Consent and privacy plan followed; clear report and defense. |
Two Model Studies: Twenty Paper Helicopters (Illustrative) and One Real Sample of a Hundred St. Louis Days
Constructed helicopter flight times (illustrative) with exact randomization test; a seed-2026 simple random sample of 100 days from the NOAA GHCN-Daily record for USW00013994 (St. Louis Lambert International Airport), 2000-2024; retrieved November 10, 2026 · SLS original (constructed teaching data), with public-domain NOAA data
Why this reading. The task is to design and run your own study. Two complete model studies are here, one for each route, carried all the way from design to scope-of-inference paragraph, so that you can see the shape of a finished report before you plan yours. The helicopter data are constructed and labeled as such; the sample of days is a real random sample from the real St. Louis population in the Lesson 3.2 reading.
Route A, modeled: paper helicopters (illustrative data)
Question. Do paper helicopters with 10 cm blades stay aloft longer than ones with 7 cm blades? Units. Twenty helicopters cut from the same paper to the same template, numbered 1 to 20, identical except for blade length. Treatments. Long blades (10 cm) and short blades (7 cm, the control). Random assignment. Python, random.Random(2026), shuffle the numbers 1-20, first ten get long blades: long = 1, 2, 3, 5, 6, 7, 8, 12, 14, 19; short = 4, 9, 10, 11, 13, 15, 16, 17, 18, 20. Response. Seconds from release at 2.0 m to touching the floor, timed with the same stopwatch by the same person, mean of three drops. Blinding. Impossible for the timer, who can see the blades; the fixed release height and the three-drop average are the safeguards instead.
The flight times below were written for this reading to show the method. No helicopters were flown.
| Group | Unit 1 | Unit 2 | Unit 3 | Unit 4 | Unit 5 | Unit 6 | Unit 7 | Unit 8 | Unit 9 | Unit 10 | Mean | Median | SD |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Long blades (s) | 2.31 | 2.18 | 2.45 | 2.02 | 2.27 | 2.39 | 2.11 | 2.50 | 2.22 | 2.35 | 2.280 | 2.290 | 0.151 |
| Short blades (s) | 2.10 | 1.98 | 2.24 | 2.05 | 2.31 | 1.92 | 2.15 | 2.28 | 2.01 | 2.19 | 2.123 | 2.125 | 0.133 |
“Unit 1” to “Unit 10” in each row are the ten helicopters of that group in the order listed above (helicopter 1, 2, 3, … for long; 4, 9, 10, … for short).
Analysis. Observed difference in means: 2.280 − 2.123 = 0.157 s. Randomization test, exact: of the 184,756 ways to split the twenty times into two groups of ten, 2,441 give a difference of 0.157 or more, so the one-sided P-value is 0.0132 (two-sided 0.0264). A simulation of 1,000 shuffles with seed 2026 found 15 at or above the observed difference, P ≈ 0.015. Both groups are roughly symmetric with no outliers by the 1.5 × IQR rule, so the mean is a fair summary.
Scope of inference. Because blade length was randomly assigned, the extra 0.16 seconds can be attributed to the blades and not to some other difference between the helicopters; a difference this large would arise from random assignment alone about 1.3% of the time if blades made no difference. Because the twenty helicopters were not sampled from any larger population, the result generalizes to other paper, sizes and rooms only as a judgment, which the report should say.
Route B, modeled: a real sample of 100 St. Louis days
Question. What proportion of days at St. Louis Lambert International Airport, 2000–2024, had measurable rain? Population and frame. The 9,132 days in the NOAA record, listed in date order (the frame is complete; see the Lesson 3.2 reading). Sampling. Simple random sample of 100 days: random.Random(2026).sample(range(9132), 100). Nonresponse. None; every sampled day has a recorded value. Response variable. 1 if PRCP ≥ 0.01 in, else 0.
Result. 34 of the 100 sampled days were rainy: p̂ = 0.34. Margin of error by the Lesson 3.2 method, two standard deviations of p̂: 2√(0.34 × 0.66 / 100) = 0.095, so the estimate is 0.34 ± 0.09, or 0.245 to 0.435. The true proportion, 0.3117, is inside that interval. It will not be inside about one interval in twenty, and a report cannot know in advance which. The sample drew 16 January days and 4 September days; random sampling does not promise a perfect spread across months, only a fair chance for each day.
Scope of inference. Random selection from the whole frame supports generalizing to all 9,132 days. Nothing was assigned, so nothing causal is claimed. The days are the population, so this is exactly the “days as the population” option the task offers to students who would rather not survey people. It says nothing about 2025, or about a station in another part of Missouri.
What your Stage 1 plan needs
Every bold heading above, filled in for your study, plus the ethics plan the task requires. A parent or guardian reads your plan and signs it before you collect anything from anyone. If you run Route A on people, the twenty numbers above show the minimum: twenty units, two groups, a fixed measurement rule and a randomization you can show.
Sources: NOAA National Centers for Environmental Information, GHCN-Daily (Daily Summaries), station USW00013994, St. Louis Lambert International Airport (ncei.noaa.gov)