What are the principles of a well-designed experiment, and what does each one protect against?
Topic 3.5 Introduction to Experimental Design: identify the components of an experiment (units, treatments, response) and apply the principles of comparison, random assignment, replication, and control, including blinding and the placebo effect.
A focused answer to AP Statistics Topic 3.5, on experimental units, treatments and factors, and the principles of comparison, random assignment, replication, and control, plus blinding and the placebo effect.
Reviewed by: AI editorial process; not yet individually human-reviewed
Have a quick question? Jump to the Q&A page
Jump to a section
What this topic is asking
The College Board (Topic 3.5) wants you to identify the parts of an experiment, experimental units, treatments and factors, and a response, and to apply the four principles of good design: comparison, random assignment, replication, and control (including blinding and the placebo effect).
The parts of an experiment
Naming these precisely is the first thing exam questions check. In a study of a fertilizer at two doses on tomato plants, the units are the plots, the factor is fertilizer dose, the levels/treatments are the doses used, and the response is the yield. Getting the vocabulary right frames everything that follows.
The four principles
Each principle defends against a specific threat. Comparison defends against having no baseline (you cannot tell if "many improved" is impressive without a control). Random assignment defends against confounding. Replication defends against mistaking random noise for a real effect. Control, including blinding, defends against the placebo effect (people respond to being treated, not just to the treatment) and against assessors whose expectations color their measurements.
Random assignment is the heart of it
The single feature that turns an experiment into causal evidence is random assignment. By using chance to decide which units get which treatment, you make the groups similar in every respect except the treatment, not just in variables you thought to measure, but in all of them, known and unknown. Age, motivation, soil quality, and a hundred unmeasured factors are, on average, spread evenly across the groups. So if the groups' responses then differ by more than chance would explain, the treatment is the only systematic difference left to credit, and you may conclude cause and effect. This is the precise reason an experiment can do what an observational study (Topic 3.2) cannot: random assignment manufactures comparable groups, eliminating confounding by design rather than hoping it is absent.
Blinding, placebos, and control
Two refinements deserve emphasis because the exam tests them. A placebo is a dummy treatment indistinguishable from the real one (a sugar pill); a control group receiving it isolates the placebo effect, the real tendency of people to improve simply because they believe they are being treated. Blinding keeps the treatment assignment hidden: in a single-blind experiment the subjects do not know which treatment they got, removing the placebo effect's confounding; in a double-blind experiment neither the subjects nor those who interact with or assess them know, which also removes assessor bias (a doctor who knows who got the drug might rate them more favorably). Control more broadly means keeping all other conditions identical across groups, same timing, same environment, same instructions, so the only thing that differs is the treatment. Together, comparison, random assignment, replication, and control make the difference in response a clean read on the treatment's effect, which is exactly what Topic 3.7 then formalises as the basis for a causal conclusion.
Try this
Q1. State what random assignment achieves that distinguishes an experiment from an observational study. [2 points]
- Cue. It balances confounding variables (known and unknown) across treatment groups, so a difference in response can be attributed to the treatment, allowing a causal conclusion.
Q2. Why include a control group that receives a placebo? [1 point]
- Cue. It provides a baseline and isolates the placebo effect, so the treatment's real effect is measured against people who believe they are being treated but are not.
Exam-style practice questions
Practice questions written in the style of College Board exam questions on this dot point, with worked answer explainers. The year tag is the paper they imitate, not the source.
AP 2019 (style)1 marksSection I (multiple choice). In a drug trial, neither the patients nor the doctors assessing them know who received the real drug and who received a placebo. This design feature is called (A) replication (B) blocking (C) double-blinding (D) random assignmentShow worked answer →
The correct answer is (C).
When both the subjects and those who interact with or assess them are unaware of the treatment assignment, the experiment is double-blind. This guards against the placebo effect and assessor bias.
(A) Replication is applying treatments to many units. (B) Blocking groups similar units before assigning. (D) Random assignment distributes subjects to treatments by chance. Hiding the assignment from both parties is double-blinding.
AP 2022 (style)4 marksSection II (free response). A researcher tests whether a new fertilizer increases tomato yield. She has plots of land. (a) Describe how to use random assignment to form two treatment groups. (b) Explain the purpose of including a control group. (c) Explain how replication (using plots rather than ) strengthens the experiment, justifying in context.Show worked answer →
A 4-point question on experimental principles.
(a) (1 point) Number the plots to ; use a random number generator to choose plots for the new fertilizer, with the remaining as the other group. This random assignment balances soil and other variables across groups.
(b) (1 point) The control group (no new fertilizer, or the standard treatment) provides a baseline for comparison, so any yield difference can be attributed to the new fertilizer rather than to growing tomatoes generally.
(c) (2 points) Replication means applying each treatment to many plots (1 point); with plots per group, plot-to-plot variation averages out, so a real treatment effect can be distinguished from chance differences, whereas plot each could differ purely by luck (1 point, in context).
Markers reward a correct random-assignment procedure, the comparative purpose of a control group, and replication's role in averaging out unit-to-unit variation.
Related dot points
- Topic 3.2 Introduction to Planning a Study: distinguish observational studies from experiments, identify explanatory and response variables, and recognize that only an experiment with imposed treatments can support a causal conclusion.
A focused answer to AP Statistics Topic 3.2, distinguishing observational studies from experiments, identifying explanatory and response variables and confounding, and explaining why imposing treatments is what enables causal claims.
- Topic 3.6 Selecting an Experimental Design: compare completely randomised, randomised block, and matched pairs designs, and explain how blocking and pairing control a known source of variation to make treatment effects clearer.
A focused answer to AP Statistics Topic 3.6, comparing completely randomised, randomised block, and matched pairs designs, and explaining how blocking and pairing remove a known source of variation to sharpen the comparison.
- Topic 3.7 Inference and Experiments: use the presence or absence of random selection and random assignment to determine the scope of inference, that is, whether results generalize to a population and whether a causal conclusion is justified.
A focused answer to AP Statistics Topic 3.7, on the scope of inference, using random selection (generalization) and random assignment (causation) to decide what conclusions are valid, with a worked four-quadrant analysis.
- Topic 3.1 Introducing Statistics: Do the Data We Collected Tell the Truth? Recognize that the method of data collection determines the kinds of conclusions that can be drawn, and that poorly collected data cannot be fixed by analysis.
A focused answer to AP Statistics Topic 3.1, on why the data-collection method determines what conclusions are valid, the difference between random error and bias, and why analysis cannot rescue badly collected data.
- Topic 3.3 Random Sampling and Data Collection: describe and distinguish simple random, stratified, cluster, and systematic random sampling, and explain why random selection supports generalization to a population.
A focused answer to AP Statistics Topic 3.3, describing simple random, stratified, cluster, and systematic random sampling, how each uses chance, their trade-offs, and why random selection allows generalization, with a worked SRS selection.
Sources & how we know this
- AP Statistics Course and Exam Description — College Board (2020)