Topic 6.7 Potential Errors When Performing Tests: distinguish Type I and Type II errors and their consequences, define the power of a test, and explain how significance level, sample size, and effect size affect error probabilities and power.

What this topic is asking

The College Board (Topic 6.7) wants you to distinguish Type I and Type II errors, describe their consequences in context, define the power of a test, and explain how the significance level $\alpha$, the sample size $n$, and the effect size affect error probabilities and power.

The four outcomes

A clear way to keep them straight: a Type I error is a false alarm (you cry effect when there is none); a Type II error is a missed detection (you miss a real effect). Always describe each in the direction of the specific test, naming the context, because exam credit depends on saying which error means what here, not reciting the textbook definition.

Consequences drive the trade-off

Which error is worse depends on context, and naming the real-world consequence of each is routinely examined. In a medical screen, a Type I error (false positive) might cause needless treatment, while a Type II error (false negative) might leave a disease untreated; the relative harms decide whether you set $\alpha$ low or high. There is no universally "safer" choice; you balance the costs. This is why $\alpha$ is chosen before the data, as a policy about acceptable false-alarm risk.

Power and what raises it

Three factors raise power:

1. Larger sample size $n$. The biggest controllable lever. More data shrink the standard error, so a true effect produces a more extreme statistic and is detected more often, all without changing $\alpha$. This is the standard answer to "how can power be increased without raising the Type I error rate?"
2. Larger true effect (effect size). The farther the true parameter is from the null value, the easier it is to detect, so power rises. This is not under the experimenter's control, but it explains why small effects need large samples.
3. Larger $\alpha$. Loosening the rejection threshold rejects $H_0$ more readily, raising power, but at the cost of a higher Type I error rate. So this lever trades one error for the other rather than improving the test for free.

Reduced variability (for example, a less variable population or a better design) also raises power by shrinking the standard error, the same mechanism as a larger $n$.

Try this

Q1. Define a Type II error and name its consequence in a test of whether a treatment works. [2 points]

• Cue. Failing to reject $H_0$ when the treatment truly works (a missed effect); consequence: an effective treatment is not adopted, so its benefits are lost.

Q2. Name two ways to increase power, and which one also raises the Type I error rate. [2 points]

• Cue. Increase the sample size (does not change $\alpha$) and increase $\alpha$ (which does raise the Type I error rate). A larger true effect also raises power.