THE VR SCHOOL
  • Schools & Districts
    • License the academy
    • Free program for your students
    • Request a demo
  • Catalog
    • AI & Data
    • Cybersecurity & Web
    • Programming
    • Health Science
    • All Guilds
    • All courses
  • Families
  • Accreditation
  • Log In
  • Request a demo
  • Home
  • Courses

RLP Statistics B — Probability Models and Statistical Inference

Curriculum

  • 4 Sections
  • 20 Lessons
  • 10 Weeks
Expand all sectionsCollapse all sections
  • Unit 1: Continuous Distributions and Sampling Distributions
    5
    • 1.1
      Continuous Random Variables and Density Curves
      50 mins
    • 1.2
      The Normal Distribution
      50 mins
    • 1.3
      When Is a Normal Model Appropriate?
      50 mins
    • 1.4
      Sampling Distributions and the Central Limit Theorem
      50 mins
    • 1.5
      Performance Task — Sampling Distribution Simulation Study
      150 mins
  • Unit 2: Estimating with Confidence
    5
    • 2.1
      Estimating a Proportion with Confidence
      50 mins
    • 2.2
      Confidence Intervals for a Mean
      50 mins
    • 2.3
      What Confidence Means, and What It Does Not
      50 mins
    • 2.4
      Comparing Two Groups with Confidence Intervals
      50 mins
    • 2.5
      Performance Task — Estimation Study
      150 mins
  • Unit 3: Testing Claims: Significance Tests
    5
    • 3.1
      The Logic of a Significance Test
      50 mins
    • 3.2
      t-Tests for Means and Paired Data
      50 mins
    • 3.3
      Errors, Power and Significance in the Real World
      50 mins
    • 3.4
      Two-Sample Tests from Experiments and Observational Data
      50 mins
    • 3.5
      Performance Task — Significance Test Investigation
      150 mins
  • Unit 4: Chi-Square, Regression Inference and the Capstone Study
    5
    • 4.1
      Chi-Square Goodness of Fit
      50 mins
    • 4.2
      Chi-Square Tests for Two-Way Tables
      50 mins
    • 4.3
      Inference for the Slope of a Regression Line
      50 mins
    • 4.4
      Choosing the Right Procedure and Writing It Up
      50 mins
    • 4.5
      Capstone — Full Statistical Inference Study
      200 mins

The Logic of a Significance Test

Unit 3  ·  Launch & Concept Development  ·  Lesson 11 of 20

The Logic of a Significance Test

PS.18RLP-STATSB
Hands sort anonymous survey cards beside a reference mix and a dot distribution with extreme tails highlighted.
By the end of this lesson I can…

state hypotheses about a parameter, compute a test statistic and P-value for a one-proportion z-test, and write a conclusion in context.

Instruction

A significance test asks whether data give convincing evidence against a claim. You already did one in RLP Statistics A: the randomization test asked whether a difference between treatments was bigger than chance assignment would produce. The formal test has the same logic.

The four steps.

  1. State. The null hypothesis H0 is the claim of no effect or no difference, and it names a specific parameter value. The alternative Ha is what you suspect instead: one-sided (< or >) or two-sided (≠), decided before looking at the data. Choose a significance level α (often 0.05).
  2. Plan. Choose the test and check its conditions.
  3. Do. Compute the test statistic, (statistic − null value) / (standard deviation of statistic), and the P-value.
  4. Conclude. If P ≤ α, reject H0: the data give convincing evidence for Ha. If P > α, fail to reject H0: the data do not give convincing evidence. That is not proof that H0 is true.

The P-value (PS.18) is the probability, computed assuming H0 is true, of getting a statistic at least as extreme as the one observed. A small P-value means the data would be surprising if H0 were true.

Worked example (illustrative numbers). A city report states that 60% of residents support a transit plan. An advocacy group suspects the true figure is different and surveys a simple random sample of 150 residents: 78 support the plan. The survey numbers are illustrative.

  1. State: p = the proportion of all residents who support the plan. H0: p = 0.60; Ha: p ≠ 0.60; α = 0.05.
  2. Plan: one-proportion z-test. Random: SRS. 10%: 150 is far less than 10% of a city. Large counts, using p0: 150(0.60) = 90 and 150(0.40) = 60, both at least 10.
  3. Do: p̂ = 78/150 = 0.52. Under H0, the SD of p̂ is √(0.60 × 0.40/150) = 0.040. z = (0.52 − 0.60)/0.040 = −2.00. Two-sided P-value = 2 × P(Z ≤ −2.00) = 2(0.0228) ≈ 0.046.
  4. Conclude: Because 0.046 < 0.05, reject H0. The data give convincing evidence that the proportion of residents who support the plan differs from 60%. The sample suggests it is lower.

Note the SD. A test uses p0 in the standard deviation, because the calculation assumes H0 is true. A confidence interval uses p̂, because there is no hypothesized value. They answer different questions.

An honest check. The normal model approximates the binomial. The exact binomial P-value for 78 or fewer (or equally extreme on the other side) out of 150 when p = 0.60 is about 0.055, just above 0.05. At α = 0.05, the two methods reach different decisions. This is a warning, not a contradiction: the evidence sits right at the conventional line, and a careful report says “moderate evidence, P ≈ 0.05” rather than treating 0.05 as a cliff. Lesson 3.3 returns to this.

Connecting to models (A2.DS.A.2). A test asks whether the model named by H0 is consistent with the data. You could answer it by simulation: generate thousands of samples of 150 from a population with p = 0.60 and see how often p̂ ≤ 0.52. The z-test uses the normal model to do the same thing by formula.

Tools. Spreadsheet: =NORM.S.DIST(−2, TRUE) for the lower tail, and =BINOM.DIST(78, 150, 0.6, TRUE) for the exact lower tail. GeoGebra Probability: Normal with μ = 0, σ = 1, left-sided at −2.

📖 Read for this lesson

The reading opens over the lesson; close it to come back to exactly where you were. The checks below draw on it.

  • Behind a Borderline P-Value: The Exact Binomial and Five Hundred Simulated SurveysExact Binomial(150, 0.60) probabilities and a 500-sample simulation record (Python random module, seed 2026), computed for this course for the lesson’s illustrative transit survey · about 4 min

    The exact Binomial(150, 0.6) probabilities that give the lesson’s 0.055, both tails shown, the continuity correction that reconciles it with the normal, and a seeded 500-sample simulation under the null.

Vocabulary in context

  • null hypothesis (H₀) — The claim tested, usually of no effect or no difference, stated as a specific parameter value.
  • alternative hypothesis (Hₐ) — The claim the study seeks evidence for: one-sided or two-sided, set before seeing data.
  • significance level (α) — The P-value threshold, chosen in advance, below which H₀ is rejected.
  • test statistic — A standardized measure of how far the sample statistic is from the null value.
  • P-value — The probability, assuming H₀ is true, of a statistic at least as extreme as the one observed.

Formative check

Work through these before moving on. They are not graded — they tell you whether the standard below has landed yet.

+50 XP

In the transit example, P ≈ 0.046. Which statement is correct?

A P-value is computed assuming H₀ is true; it is not the probability that H₀ is true.
Fill in the blank

Under H₀: p = 0.60 with n = 150, the SD of p̂ is , so p̂ = 0.52 gives z = .

Myth or Fact?

If P > α, the test has shown that H₀ is true.

Failing to reject means the data are not convincing evidence against H₀. The sample may simply be too small to detect a real difference.

Carry out the transit test yourself with a spreadsheet or GeoGebra, then compute the exact binomial P-value. Explain in four sentences why the two P-values differ and what a careful analyst should report. Then run a simulation of 500 samples of 150 with p = 0.60 (CODAP or =RAND()<0.6) and report how often p̂ ≤ 0.52.

0 words
Quick self-check

How confident are you that you can carry out a one-proportion test and interpret its P-value correctly?

Not yetVery confident

Practice

Work these on paper or in your notebook, then open Check your answer. Aim for all of Fluency and Application; try at least one Challenge.

Printable version: this unit’s practice workbook (PDF)

Fluency

Build speed and accuracy with the core skill.

  1. Test H0: p = 0.4 against Ha: p > 0.4, given 52 successes in a random sample of 150. Find the test statistic and the P-value.
    Check your answer
    Answer: p̂ = 0.347; z ≈ −1.33; P-value ≈ 0.9088. At α = 0.05, fail to reject H0: there is not convincing evidence that the population proportion is greater than 0.4.
    Use p0, not p̂, in the standard error of a test. State the conclusion in context: reject or fail to reject H0, never “accept”.
  2. Test H0: p = 0.25 against Ha: p > 0.25, given 112 successes in a random sample of 500. Find the test statistic and the P-value.
    Check your answer
    Answer: p̂ = 0.224; z ≈ −1.34; P-value ≈ 0.9103. At α = 0.05, fail to reject H0: there is not convincing evidence that the population proportion is greater than 0.25.
    Use p0, not p̂, in the standard error of a test. State the conclusion in context: reject or fail to reject H0, never “accept”.
  3. Test H0: p = 0.2 against Ha: p < 0.2, given 69 successes in a random sample of 500. Find the test statistic and the P-value.
    Check your answer
    Answer: p̂ = 0.138; z ≈ −3.47; P-value ≈ 0.0003. At α = 0.05, reject H0: there is convincing evidence that the population proportion is less than 0.2.
    Use p0, not p̂, in the standard error of a test. State the conclusion in context: reject or fail to reject H0, never “accept”.
  4. Test H0: p = 0.3 against Ha: p ≠ 0.3, given 66 successes in a random sample of 300. Find the test statistic and the P-value.
    Check your answer
    Answer: p̂ = 0.220; z ≈ −3.02; P-value ≈ 0.0025. At α = 0.05, reject H0: there is convincing evidence that the population proportion is different from 0.3.
    Use p0, not p̂, in the standard error of a test. State the conclusion in context: reject or fail to reject H0, never “accept”.
  5. Test H0: p = 0.6 against Ha: p > 0.6, given 203 successes in a random sample of 300. Find the test statistic and the P-value.
    Check your answer
    Answer: p̂ = 0.677; z ≈ 2.71; P-value ≈ 0.0034. At α = 0.05, reject H0: there is convincing evidence that the population proportion is greater than 0.6.
    Use p0, not p̂, in the standard error of a test. State the conclusion in context: reject or fail to reject H0, never “accept”.
  6. Test H0: p = 0.3 against Ha: p ≠ 0.3, given 55 successes in a random sample of 150. Find the test statistic and the P-value.
    Check your answer
    Answer: p̂ = 0.367; z ≈ 1.78; P-value ≈ 0.0748. At α = 0.05, fail to reject H0: there is not convincing evidence that the population proportion is different from 0.3.
    Use p0, not p̂, in the standard error of a test. State the conclusion in context: reject or fail to reject H0, never “accept”.
  7. Which is a correctly stated null hypothesis? (a) H₀: p̂ = 0.6 (b) H₀: p = 0.6 (c) H₀: p > 0.6
    Check your answer
    Answer: (b). Hypotheses are about parameters, and H₀ states a single value.
  8. A test gives P = 0.03 at α = 0.05. State the decision in words.
    Check your answer
    Answer: Reject H₀: the data give convincing evidence for Hₐ.
  9. A test gives P = 0.21 at α = 0.05. State the decision in words.
    Check your answer
    Answer: Fail to reject H₀: the data do not give convincing evidence for Hₐ (this does not show H₀ is true).
  10. In a one-proportion z-test, why does the standard deviation use p₀ rather than p̂?
    Check your answer
    Answer: The P-value is computed assuming H₀ is true, so the sampling distribution is centered at p₀ with SD √(p₀(1 − p₀)/n).

Application

Use the skill in context. Show your reasoning.

  1. A random sample of 200 St. Louis days (2000-2024) contains 66 rainy days. A friend claims one day in four is rainy. Test H₀: p = 0.25 against Hₐ: p ≠ 0.25 at α = 0.05, with conditions, and state the conclusion in context.
    Check your answer
    Answer: Random; 200 < 10% of 9,132; np₀ = 50 and n(1 − p₀) = 150. p̂ = 0.330, z ≈ 2.61, P ≈ 0.009. Since P < 0.05, reject H₀: the sample gives convincing evidence that the proportion of rainy days differs from 25%; the sample suggests it is higher.
    The exact binomial P-value is about 0.014.
  2. A random sample of 150 Springfield days (2000-2024) contains 55 rainy days. Test whether fewer than 40% of Springfield days are rainy (Hₐ: p < 0.40), at α = 0.05.
    Check your answer
    Answer: np₀ = 60 and n(1 − p₀) = 90, both ≥ 10. p̂ = 0.367, z ≈ -0.83, one-sided P ≈ 0.202. Fail to reject H₀: not convincing evidence that fewer than 40% of Springfield days are rainy.
  3. A student writes: ‘P = 0.046, so there is a 4.6% chance that H₀ is true.’ Rewrite the sentence correctly.
    Check your answer
    Answer: If H₀ were true, a sample result at least as far from p₀ as ours would occur about 4.6% of the time.

Challenge

Stretch problems. Expect to think before you write.

  1. For the St. Louis test above, the normal approximation gives P ≈ 0.009 and the exact binomial gives P ≈ 0.014. When can two methods lead to different decisions at α = 0.05, and how should a report handle that?
    Check your answer
    Answer: When the evidence is near the threshold, small approximation differences can put one P-value on each side of 0.05. A report should give the P-value and describe the evidence as moderate, rather than treating 0.05 as a cliff; here both P-values are well below 0.05 and point the same way.

Review

Keep earlier skills sharp.

  1. In a random sample of 800, 162 said yes. Construct a 95% confidence interval for the population proportion.
    Check your answer
    Answer: p̂ = 0.203; margin ≈ 0.028; interval ≈ (0.175, 0.230). We are 95% confident that the proportion of the whole population who would say yes is between 0.175 and 0.230.
    z* ≈ 1.960. Conditions: a random sample; the sample is less than 10% of the population; successes and failures (162 and 638) are both at least 10.
  2. (Unit 2) A 95% interval for p is (0.14, 0.25). Would a two-sided test of H₀: p = 0.30 at α = 0.05 reject?
    Check your answer
    Answer: Yes: 0.30 is outside the 95% interval.

Missouri Learning Standards — Mathematics (data and probability) addressed: A2.DS.A.2, A2.DS.A.3, MP.3, MP.2

Extends beyond Missouri Statistics (enrichment): content Missouri places after Statistics

Instructional focus: Hypothesis testing logic and P-values for a proportion

Reading

Behind a Borderline P-Value: The Exact Binomial and Five Hundred Simulated Surveys

Exact Binomial(150, 0.60) probabilities and a 500-sample simulation record (Python random module, seed 2026), computed for this course for the lesson’s illustrative transit survey · SLS original

Why this reading. The lesson’s transit survey is illustrative, and says so; it is the method that is being taught. But the lesson then states an exact binomial P-value of “about 0.055” and asks you to reproduce it and to run a simulation of 500 samples. Both need something the lesson does not give: the binomial probabilities themselves, and a simulation you can point to. Here they are. With this page a student with no device can do every step of the open question and see exactly where the normal approximation and the exact answer part ways.

Table 1. Exact Binomial(150, 0.60) probabilities near the observed count

Computed for this course; P(X = x) = C(150, x) 0.6x 0.4150 − x. The mean is 90 and the SD is √(150 × 0.6 × 0.4) = 6.0. Values below 66 are each under 0.0001 and add to 0.0000.

x66676869707172737475767778
P(X = x)0.00000.00010.00010.00020.00030.00050.00080.00130.00200.00300.00450.00650.0091
x798081828384858687888990
P(X = x)0.01250.01660.02150.02720.03340.03990.04650.05270.05820.06250.06530.0664

The lower tail. P(X ≤ 78) = 0.0000 (for x ≤ 65) + the entries for 66 through 78 = 0.0284. The spreadsheet’s BINOM.DIST(78, 150, 0.6, TRUE) gives the same.

The upper tail, “equally extreme on the other side”. The observed 78 is 12 below the mean of 90, so the matching upper tail starts at 90 + 12 = 102.

x9899100101102103104105106107108109110
P(X = x)0.02770.02180.01670.01240.00890.00620.00420.00280.00180.00110.00070.00040.0002

P(X ≥ 102) = 0.0265 (entries 102 through 110 add to 0.0263; the rest of the tail is 0.0002). So the two-sided exact P-value is 0.0284 + 0.0265 = 0.0549, the lesson’s “about 0.055”. Notice that the two tails are not equal: the binomial with p = 0.6 is slightly skewed, so the upper tail is a little lighter. The common shortcut of doubling the lower tail gives 0.0568, close but not the same thing.

Against the normal. The lesson’s z = −2.00 gives 2 × 0.0228 = 0.0455. With a continuity correction, z = (78.5 − 90)/6.0 = -1.917, and 2 × Φ(-1.92) = 0.0553, which is very close to the exact 0.0549. The normal approximation was not wrong; the half-unit correction that Lesson 1.3 taught is what was missing. The four-sentence explanation the open question asks for should say so: the discrete count was replaced by a continuous curve without the correction; the result is a P-value on the wrong side of 0.05; and a careful analyst reports “P ≈ 0.05, moderate evidence” and the exact value, rather than a decision.

Record: 500 simulated samples of 150 under H0

Generated with rng = random.Random(2026), then for each of 500 samples sum(rng.random() < 0.6 for _ in range(150)). This is the CODAP or =RAND()<0.6 simulation the open question describes, with its seed stated.

Supporters x in 15060–6970–7475–7879–8283–8687–9091–9495–9899–101102–110
Samples of 50012114287128102852219

Mean of the 500 counts 90.04, SD 6.39, against 90 and 6.0. Samples with x ≤ 78, that is p̂ ≤ 0.52: 14 of 500, or 0.028. Samples with x ≥ 102: 20 of 500, or 0.040. Two-sided: 34 of 500 = 0.068.

The exact two-sided probability is 0.0549. The simulated 0.068 is above it, and a simulated proportion of about 0.055 from 500 runs has standard error √(0.055 × 0.945/500) = 0.010, so this run is about 1.3 standard errors high: unremarkable. Your own run of 500 will land somewhere else in that range. Report the count you got, the number of repetitions, and the exact value beside it. A simulation estimates a probability that Table 1 gives exactly; its use here is to make the meaning of the P-value visible — of 500 surveys taken in a city where support really is 60%, 14 came out looking as low as the advocacy group’s did.

The standard deviation in a test, once more

Every number above used p0 = 0.60, because the question is “how would samples behave if H0 were true?” A confidence interval for the same data would use p̂ = 0.52: SE = √(0.52 × 0.48/150) = 0.0408, interval 0.52 ± 1.96(0.0408) = 0.440 to 0.600. That interval excludes 0.60, barely, which is the same borderline verdict in the other language.

Sources: Python documentation, the random module (docs.python.org)

Up NextComplete & Continue →

Leave a Reply Cancel reply

Performance Task — Estimation Study
Prev
t-Tests for Means and Paired Data
Next

© 2026 The VR School | Career content by Wiley | Powered by Sejal Learning Systems.

Join free · Accreditation · Privacy Policy · Terms of Service

Modal title

Main Content