THE VR SCHOOL
  • Schools & Districts
    • License the academy
    • Free program for your students
    • Request a demo
  • Catalog
    • AI & Data
    • Cybersecurity & Web
    • Programming
    • Health Science
    • All Guilds
    • All courses
  • Families
  • Accreditation
  • Log In
  • Request a demo
  • Home
  • Courses

RLP Statistics A — Exploring Data, Study Design and Probability

Curriculum

  • 4 Sections
  • 20 Lessons
  • 10 Weeks
Expand all sectionsCollapse all sections
  • Unit 1: Exploring One-Variable Data
    5
    • 1.1
      Distributions and the Statistical Process
      50 mins
    • 1.2
      Measuring Center and Spread
      50 mins
    • 1.3
      Comparing Distributions Across Missouri
      50 mins
    • 1.4
      Standardizing, Transforming and Reporting
      50 mins
    • 1.5
      Performance Task — Climate Station Profile
      150 mins
  • Unit 2: Two-Variable Data and Regression
    5
    • 2.1
      Scatterplots and Correlation
      50 mins
    • 2.2
      Least-Squares Regression
      50 mins
    • 2.3
      Residuals, Influence and Causation
      50 mins
    • 2.4
      Exponential Models and the Limits of Extrapolation
      50 mins
    • 2.5
      Performance Task — Bivariate Investigation Report
      150 mins
  • Unit 3: Designing Studies: Samples and Experiments
    5
    • 3.1
      Populations, Samples and Bias
      50 mins
    • 3.2
      Simulating Sampling Variability and the Margin of Error
      50 mins
    • 3.3
      Experiments, Observational Studies and Confounding
      50 mins
    • 3.4
      Comparing Treatments with a Randomization Test
      50 mins
    • 3.5
      Performance Task — Design, Run and Report a Study
      150 mins
  • Unit 4: Probability, Random Variables and the Binomial Model
    5
    • 4.1
      Probability Rules and Two-Way Tables
      50 mins
    • 4.2
      The Multiplication Rule, Trees and Counting
      50 mins
    • 4.3
      Random Variables, Expected Value and Variance
      50 mins
    • 4.4
      The Binomial Model
      50 mins
    • 4.5
      Performance Task — Expected-Value Decision Analysis
      150 mins

Simulating Sampling Variability and the Margin of Error

Unit 3  ·  Data Investigation Lab  ·  Lesson 12 of 20

Simulating Sampling Variability and the Margin of Error

A2.DS.A.4 / A2.DS.A.5RLP-STATSA
Hands draw rainy and dry-day tokens into sample trays, producing wide and narrow clusters around one proportion.
By the end of this lesson I can…

use simulation to see how a sample proportion varies from sample to sample, estimate a margin of error, and explain how sample size affects it.

Instruction

If you took a different random sample, you would get a different statistic. This lab measures how different, using a population whose parameter we actually know.

The population. NOAA’s daily records for St. Louis Lambert International Airport (GHCN-Daily station USW00013994) cover every day from January 1, 2000 to December 31, 2024: 9,132 days. On 2,846 of them the station recorded measurable rain (at least 0.01 inch). The population proportion of rainy days is

p = 2,846 / 9,132 ≈ 0.3117

In real studies p is unknown. Here we know it, so we can see how well samples estimate it.

One sample. Choose 100 days at random and compute p̂, the proportion of rainy days in the sample. You might get 0.27, or 0.36. Neither is “wrong”. Each is a legitimate estimate, and the difference from p is sampling variability.

Many samples. A computer can repeat the process thousands of times. One seeded run of 10,000 simple random samples of each size (seed 2026; the reading prints the recipe) gave:

Sample size nMean of the p̂’sSD of the p̂’sMiddle 95% of p̂’s
250.3110.0920.16 to 0.48
1000.3110.0460.22 to 0.40
4000.3110.0220.268 to 0.355

Your own run will differ slightly in the third decimal place. That is itself sampling variability.

Three patterns to notice.

  1. Centered on the truth. For every n, the p̂’s average out to about 0.311, the parameter. Random sampling is unbiased: it does not systematically miss high or low.
  2. Spread shrinks with n, but slowly. Going from 100 to 400 days (four times the data) cut the SD from 0.046 to 0.022, only in half. Spread is proportional to 1/√n, which you will prove in Statistics B.
  3. A predictable shape. For n = 100 and 400 the dot plot of p̂’s is mound-shaped and roughly symmetric. For n = 25 it is lumpy, because only 26 values are possible, and its right tail is slightly longer than its left.

The margin of error (A2.DS.A.4 / A2.DS.A.5). For n = 100, about 95% of the simulated p̂’s fell within 2 SDs (2 × 0.046 ≈ 0.092) of p. Turn that around: if you take one sample of 100 days and get p̂, then p is very likely within about 0.092 (9.2 percentage points) of p̂. That distance is the margin of error. With a sample of 400 it drops to about 2 × 0.022 ≈ 0.045.

What the margin of error does not cover. It measures only random sampling variability. It says nothing about bias from a bad frame, nonresponse or wording. A biased survey with a tiny margin of error is precisely wrong.

Checking a model (A2.DS.A.2). Simulation also lets you test a claim. If someone says “it rains on one day in five in St. Louis”, simulate samples of 100 from a population with p = 0.20. If your real sample gave p̂ = 0.34, and no simulated sample comes out that high, the one-in-five model is not consistent with the data. A claim of 30% is different: samples from p = 0.30 reach 0.34 or higher about a quarter of the time, so that model survives the same sample.

📖 Read for this lesson

The reading opens over the lesson; close it to come back to exactly where you were. The checks below draw on it.

  • The Population Itself: 9,132 St. Louis Days as 300 Monthly Counts, and a Seeded SimulationNOAA NCEI GHCN-Daily, St. Louis Lambert International Airport (USW00013994), daily precipitation 2000-2024, counted by month; simulations with Python’s random module, seed 2026; retrieved November 10, 2026 · about 6 min

    The whole 9,132-day St. Louis population as 300 monthly counts, a seeded simulation run to compare yours with, and the exact answers to the one-in-five and 30% questions.

Vocabulary in context

  • sampling variability — The natural variation of a statistic from one random sample to another.
  • sampling distribution (simulated) — The distribution of a statistic’s values over many random samples of the same size.
  • unbiased estimator — A statistic whose values, over many samples, center on the parameter it estimates.
  • margin of error — About two standard deviations of the statistic’s sampling distribution; the likely maximum distance between the statistic and the parameter due to random sampling.

Formative check

Work through these before moving on. They are not graded — they tell you whether the standard below has landed yet.

Make a prediction

Samples of 100 days give p̂’s with SD about 0.046. What SD do you expect for samples of 400 days?

About 0.023 (the seeded run gave 0.022). Spread shrinks like 1/√n: four times the sample size halves the SD.

Simulation & tools

Sample the St. Louis weather population

Data: NOAA NCEI GHCN-Daily, St. Louis Lambert International Airport (USW00013994), 2000-2024, retrieved 2026-11-10. The reading for this lesson prints the whole population as 300 monthly counts of rainy days (they sum to 2,846 rainy days out of 9,132), so nothing needs to be downloaded. Build the population from that table: a month with 15 rainy days and 31 days is fifteen 1s and sixteen 0s. That is a column of 9,132 values, 2,846 of them 1s.

  1. Name the column rainy (1 = a day with PRCP ≥ 0.01 inch, 0 = a dry day). Check that its mean is 2,846/9,132 = 0.3117.
  2. Open the Sampler plugin (Plugins menu), or use Sample in the table’s option menu, to draw 100 random cases. Record the mean of rainy (your p̂).
  3. Collect at least 200 samples of size 100 (the Sampler can repeat and collect a measure), make a dot plot of the p̂’s, and record their mean and SD. Repeat for n = 25 and n = 400.
  4. Compare with the table in the lesson. What margin of error would you report for n = 100?

Spreadsheet route: either put the 9,132 rainy values (rebuilt from the monthly counts) in column A and use =INDEX(A:A, RANDBETWEEN(1,9132)) in 100 cells, or, without building the column, use =IF(RANDBETWEEN(1,9132)<=2846,1,0) in 100 cells, which draws each day with the same 2,846-in-9,132 chance of being rainy; average the 100 cells and copy the block to get many samples. Python route (optional): in JupyterLite, random.sample(days, 100) in a loop.

Open in a new tab ↗  ·  CODAP, Concord Consortium · MIT open source

Myth or Fact?

A survey with a margin of error of ±2 percentage points cannot be off by more than 2 points.

The margin of error covers only random sampling variability, and only about 95% of the time. Bias from the frame, nonresponse or wording is not included.

A friend claims it rains on one day in five (20%) in St. Louis. Use your simulation tool to draw 200 samples of 100 days from a population with p = 0.20 (a column that is 1 with probability 0.20). How often is p̂ as high as 0.34 or higher? Is the 20% claim consistent with a real sample that gave p̂ = 0.34? Explain in three sentences. Then repeat the simulation for a friend who claims 30% and say what changes.

0 words
Quick self-check

How confident are you that you can use simulation to estimate a margin of error and explain how it depends on sample size?

Not yetVery confident

Practice

Work these on paper or in your notebook, then open Check your answer. Aim for all of Fluency and Application; try at least one Challenge.

Printable version: this unit’s practice workbook (PDF)

Fluency

Build speed and accuracy with the core skill.

  1. Simulated samples of 100 days give p̂’s with SD 0.046. What SD would you expect for samples of 400? Of 1,600?
    Check your answer
    Answer: About 0.023 for n = 400; about 0.0115 for n = 1,600.
    SD scales like 1/√n: ×1/2 and ×1/4.
  2. Samples of 25 give p̂’s with SD about 0.092. Estimate the margin of error (about 2 SDs).
    Check your answer
    Answer: About 0.18 (18 percentage points).
  3. For samples of 100 days, the middle 95% of simulated p̂’s ran from 0.22 to 0.40, around a true p of 0.3117. Is random sampling biased here?
    Check your answer
    Answer: No: the p̂’s are centered on the true value (their mean was about 0.311).
  4. To halve a margin of error, by what factor must the sample size grow?
    Check your answer
    Answer: By a factor of 4.
  5. True or false: a margin of error of ±3 points means the survey cannot be off by more than 3 points.
    Check your answer
    Answer: False.
    It covers only random sampling error, about 95% of the time; bias is not included.
  6. A simulated sampling distribution of p̂ for n = 25 is lumpy and slightly skewed right, while n = 400 gives a smooth, symmetric one. Why?
    Check your answer
    Answer: With n = 25 only 26 values of p̂ are possible (0, 0.04, 0.08, …), so the dot plot has gaps, and the spread (SD about 0.09) is large enough to feel the lower bound at 0, which leaves the right tail slightly longer. With n = 400 the SD is about 0.022, the gaps are tiny, and the bound at 0 is more than 10 SDs away, so the plot is smooth and symmetric.

Application

Use the skill in context. Show your reasoning.

  1. A friend claims it rains on one day in five (20%) of St. Louis days. A simulation of 1,000 random samples of 100 days from a population with p = 0.20 produced p̂ ≥ 0.30 in 15 samples (seed 2026). A real random sample of 100 St. Louis days gave p̂ = 0.30. What do you conclude?
    Check your answer
    Answer: Under the 20% model, a p̂ this high happened in only 15 of 1,000 samples (1.5%); the exact binomial probability is 0.0112. The real sample is unlikely under the model, so the data are not consistent with the 20% claim (the evidence is moderately strong, not overwhelming).
    A2.DS.A.2: compare the observed result with what the model produces.
  2. Using the scaling rule, about how many randomly sampled days would give a margin of error of 0.02 for the proportion of rainy days, if n = 100 gives about 0.092?
    Check your answer
    Answer: About 2,116 days (roughly 2,100).
    Margin ∝ 1/√n, so n = 100 × (0.092/0.02)².
  3. Describe how you would use simulation to estimate a margin of error for the mean daily high temperature at St. Louis from a random sample of 30 days, using the 2000-2024 record as the population.
    Check your answer
    Answer: Draw many (say 1,000) random samples of 30 days from the 9,132 recorded days, compute each sample’s mean high, find the SD of those means, and report about 2 SDs as the margin of error. (With seed 2026 the SD of 1,000 sample means was 3.70 °F, so the margin is about 7.4 °F; the population SD of daily highs is 20.06 °F and 20.06/√30 = 3.66.)

Challenge

Stretch problems. Expect to think before you write.

  1. A newspaper runs an online poll with 50,000 responses and reports a margin of error of ±0.4 points. Explain why this margin of error is misleading.
    Check your answer
    Answer: A margin of error describes random sampling variability for a random sample. An online opt-in poll is not random, so the formula does not apply; the result may be badly biased, however many responses there are.

Review

Keep earlier skills sharp.

  1. (Unit 2) Missouri’s cancer death-rate line (CDC, age-adjusted rate per 100,000) is predicted rate = 210.66 − 2.353x (x = years since 1999). Predict 2015.
    Check your answer
    Answer: About 173.0 per 100,000.
    x = 16: 210.66 − 2.353 × 16 = 173.01.
  2. (Unit 1) For 3, 6, 6, 7, 13 find the median and IQR.
    Check your answer
    Answer: Median 6; Q1 = 4.5, Q3 = 10, IQR = 5.5.

Missouri Learning Standards — Mathematics (data and probability) addressed: A2.DS.A.4, A2.DS.A.5, A2.DS.A.1, A2.DS.A.2, MP.5, MP.8

Instructional focus: Simulation-based reasoning about sampling variability

Reading

The Population Itself: 9,132 St. Louis Days as 300 Monthly Counts, and a Seeded Simulation

NOAA NCEI GHCN-Daily, St. Louis Lambert International Airport (USW00013994), daily precipitation 2000-2024, counted by month; simulations with Python’s random module, seed 2026; retrieved November 10, 2026 · Public domain (works of the United States government)

Why this reading. The lab treats the 9,132 days of the St. Louis record as a known population and quotes the results of “one run” of 10,000 samples. Nine thousand rows will not fit in a reading, but the whole population will: every rainy day of 2000–2024 is counted below by month, 300 counts that sum to the 2,846 the lesson states, and from which the 9,132-row column of 0s and 1s can be rebuilt exactly (a month with 15 rainy days and 31 days contributes fifteen 1s and sixteen 0s). Then a simulation run is printed with its seed, so that your own run has something to be compared with.

The population, as monthly counts of rainy days

NOAA GHCN-Daily, St. Louis Lambert International Airport (USW00013994), January 1, 2000 to December 31, 2024: 9,132 days, none missing. A day is rainy if PRCP ≥ 0.01 inch. Retrieved November 10, 2026.

YearJanFebMarAprMayJunJulAugSepOctNovDecTotal
20001157118128996712105
200110989121010105798107
20028714111588841378111
200371211111410671110912120
2004871661591210212146117
20051311513769128569104
20067310910678879791
200711910812107878613109
20084141211171114674813121
200966814161388715714122
2010991112151317613588126
20111091117171258961111126
20127131111635111110512105
2013612141014138471467115
201410871011951391199111
20159111391417107551210122
20167815912716109488113
20179513161079841275105
20186121391288115101211117
201912121113161591268147135
20201111131116911761096120
20219129131110118711511117
20227912151251169779109
202391016677131379412113
20241626101789752914105
All years2222262762743162382362171802112082422846

p = 2,846 / 9,132 = 0.3117. Read the bottom row before you sample: it rains in every month, from 180 rainy September days (out of 750, 24.0%) to 316 rainy May days (out of 775, 40.8%). The season matters less here than in a place with a dry summer, but it still matters: a sample of 100 days drawn only from May would not be a sample of this population. The lesson’s Sampler draws from all 9,132 days for that reason.

One run, reproducibly

Build the 9,132 values in date order as a Python list rainy. Then, for each n, rng = random.Random(2026) and [sum(rng.sample(rainy, n)) / n for _ in range(10000)]. Anyone who runs those lines gets this table.

nMean of p̂SD of p̂√(p(1 − p)/n)Middle 95% of p̂Within 2 SDs of p
250.31140.09190.09260.160 to 0.48095.3%
1000.31130.04590.04630.220 to 0.40096.2%
4000.31140.02230.02320.268 to 0.35596.2%

The fourth column is the theoretical standard deviation of p̂ for a simple random sample, √(p(1 − p)/n); the simulated SDs sit within 0.002 of it, which is the fact Statistics B proves. “Within 2 SDs of p” counts the p̂’s that fall within two theoretical SDs of the true p. Your own run will differ in the third decimal place; that difference is sampling variability of the simulation itself.

The distribution of p̂ for n = 100 in this run

p̂0.160.170.180.190.200.210.220.230.240.250.260.270.280.290.300.310.320.330.340.350.360.370.380.390.400.410.420.430.440.450.460.47
samples191122396813618327233747161869675687286686779469757947738428519713692633319686

Mound-shaped and close to symmetric around 0.31. For n = 25 the same run gives:

p̂0.040.080.120.160.200.240.280.320.360.400.440.480.520.560.600.64
samples7571824569171342163517281406105264734515352192

Only 26 possible values (0, 0.04, 0.08, …, 1), so the dot plot has gaps, and the right tail is a little longer than the left. The skew is real but mild here, because p is not close to 0.

Testing two claims about the rain

A friend says it rains on one day in five (p = 0.20). You draw one real random sample of 100 days (random.Random(2026).sample(range(9132), 100)) and find 34 rainy days, p̂ = 0.34. How often would a sample of 100 days give p̂ ≥ 0.34 if p really were 0.20? Exactly, from the binomial distribution: P(X ≥ 34 | n = 100, p = 0.20) = 0.00074, about 1 in 1,357. In a simulated run of 200 samples of 100 from a population with p = 0.20 (seed 2026, rng.random() < 0.2 for each day), 0 samples reached 0.34 or more (the largest p̂ was 0.30); the mean was 0.203 and the SD 0.042. The data are not consistent with the one-in-five model.

Now a different friend says 30% (p = 0.30). Exactly, P(X ≥ 34 | n = 100, p = 0.30) = 0.2207, about 1 in 4.5. In a simulated run of 200 samples from p = 0.30 (seed 2026), 50 samples (25.0%) had p̂ ≥ 0.34. A result that common is no evidence against the 30% model, and the true proportion, 0.3117, is indeed close to 0.30. Consistent with a model is not the same as proof of the model; it only means this sample cannot rule it out.

What to do

  1. Rebuild the population from the monthly counts (or import the NOAA file) and confirm the mean of the rainy column is 0.3117.
  2. Run your own 200 or more samples at n = 25, 100 and 400. Report mean, SD and the middle 95%, next to the table above, and say in one sentence why yours differ.
  3. State the margin of error you would report for n = 100 and for n = 400, and what it does not cover.
  4. Answer the one-in-five question in three sentences, using either the exact probability above or your own simulation. Then say what the same sample says about the 30% claim.

Sources: NOAA National Centers for Environmental Information, GHCN-Daily (Daily Summaries), station USW00013994, St. Louis Lambert International Airport (ncei.noaa.gov); NCEI Access Data Service request used for the daily file (ncei.noaa.gov)

Up NextComplete & Continue →

Leave a Reply Cancel reply

Populations, Samples and Bias
Prev
Experiments, Observational Studies and Confounding
Next

© 2026 The VR School | Career content by Wiley | Powered by Sejal Learning Systems.

Join free · Accreditation · Privacy Policy · Terms of Service

Modal title

Main Content