Collecting Data

What you'll learn

This revision guide covers the fundamental principles of data collection for AP Statistics, focusing on sampling methods, experimental design, and observational studies. You'll learn to distinguish between different study types, identify sources of bias, and apply appropriate randomization techniques. Understanding these concepts is essential for both multiple-choice questions and free-response problems on the AP exam.

Key terms and definitions

Census — a study that collects data from every member of the population of interest.

Sample — a subset of individuals selected from a population, used to draw conclusions about the entire population.

Bias — systematic favoritism in data collection that causes sample results to consistently differ from the truth about the population in a particular direction.

Random sampling — a selection method where each member of the population has a known, non-zero probability of being selected.

Confounding variable — a variable that influences both the explanatory variable and the response variable, making it difficult to establish causation.

Randomization — the random assignment of subjects to treatment groups in an experiment, used to equalize the effects of unmeasured variables.

Blocking — grouping subjects with similar characteristics together before randomly assigning treatments within each group.

Replication — applying each treatment to multiple subjects or repeating the experiment multiple times to assess variability and increase precision.

Core concepts

Populations and samples

A population includes all individuals or objects of interest in a study. Due to practical constraints (cost, time, accessibility), researchers typically collect data from a sample rather than conducting a census.

Key considerations when selecting samples:

• The sample should be representative of the population
• Larger samples generally provide more reliable estimates
• Poor sampling methods can render even large samples worthless
• The goal is to make valid inferences from the sample to the population

Sampling methods

Simple Random Sample (SRS)

Every possible sample of size n has an equal chance of being selected. Implementation:

• Assign each population member a unique number
• Use a random number generator or table to select n numbers
• Include corresponding individuals in the sample

Stratified Random Sample

The population is divided into homogeneous groups called strata based on a characteristic (e.g., age, gender, region). An SRS is then taken from each stratum.

Advantages:

• Ensures representation from each subgroup
• Can reduce variability in estimates
• Allows comparisons between strata

Cluster Sample

The population is divided into heterogeneous groups called clusters (often based on geography or convenience). Randomly select some clusters and include all members from selected clusters, or take an SRS within selected clusters.

Differences from stratified sampling:

• Clusters should be internally diverse
• Only some clusters are selected (not all)
• Often more practical but less precise

Systematic Sample

Select every kth individual from a list after a random start. For example, if k = 10, randomly choose a starting point between 1-10, then select every 10th person thereafter.

Caution: Can introduce bias if the list has a hidden pattern matching the sampling interval.

Sources of bias

Undercoverage bias — occurs when some groups in the population are inadequately represented in the sample. Example: conducting a phone survey using only landlines excludes those who rely solely on mobile phones.

Nonresponse bias — arises when individuals selected for the sample cannot be contacted or refuse to participate, and these nonrespondents differ systematically from respondents. Example: busy professionals may be less likely to complete lengthy surveys.

Response bias — occurs when the method of data collection influences responses. Sources include:

• Wording effects: Leading or confusing questions
• Interviewer effects: Respondents alter answers based on interviewer characteristics
• Social desirability bias: Respondents give answers they believe are more acceptable
• Question order: Previous questions influence subsequent responses

Voluntary response bias — results when individuals choose whether to participate. Those with strong opinions are more likely to respond, creating an unrepresentative sample. Example: online polls or radio call-in surveys.

Observational studies vs. experiments

Observational Study

Researchers observe and measure variables without imposing treatments. They record what naturally occurs.

Characteristics:

• No manipulation of variables by researchers
• Can identify associations between variables
• Cannot establish causation due to potential confounding
• Examples: surveys, retrospective studies, prospective studies

Experiment

Researchers deliberately impose treatments on subjects and observe responses.

Characteristics:

• Researchers actively manipulate the explanatory variable
• Random assignment to treatment groups
• Can establish cause-and-effect relationships
• Controls for confounding through randomization
• Examples: clinical trials, agricultural field trials

The critical distinction: randomization in experiments allows causal conclusions, while observational studies can only demonstrate associations.

Experimental design principles

Control

Compare the treatment of interest to a control group to isolate the effect of the treatment. Control methods include:

• Placebo control: Give control group an inactive treatment that appears identical to the real treatment
• Comparison group: Use a standard or existing treatment as the baseline

Randomization

Randomly assign subjects to treatment groups to:

• Balance known and unknown confounding variables across groups
• Allow probability-based inference
• Eliminate systematic bias in group assignment

Random assignment ≠ random sampling (they serve different purposes).

Replication

Apply treatments to multiple subjects and/or repeat the experiment:

• Increases reliability of results
• Allows assessment of variability
• Provides basis for statistical inference
• "Multiple subjects per treatment" not "repeating measurements on same subject"

Blocking

Group subjects by a characteristic related to the response before randomizing:

• Accounts for known sources of variability
• Increases precision by comparing like with like
• Example: block by age group, gender, or severity of condition

Experimental designs

Completely Randomized Design

All subjects are randomly assigned to treatments without restriction.

Structure:

1. Identify all experimental units
2. Randomly assign units to treatments
3. Apply treatments and measure responses
4. Compare treatment groups

Randomized Block Design

Subjects are first grouped into blocks, then treatments are randomly assigned within each block.

Structure:

1. Create blocks of similar subjects
2. Randomly assign treatments within each block
3. Each treatment appears in each block
4. Analyze within-block comparisons

Matched Pairs Design

A special case of blocking where:

• Each subject receives both treatments (in random order), OR
• Pairs of similar subjects are formed, with one receiving each treatment

Common in before-after studies or twin studies.

Scope of inference

The design determines what conclusions are valid:

Random Sampling	Random Assignment	Can Generalize to Population	Can Infer Causation
Yes	Yes	Yes	Yes
Yes	No	Yes	No
No	Yes	No	Yes
No	No	No	No

• Random sampling allows generalization to the population
• Random assignment allows causal conclusions
• Ideally, both are present, but practical constraints often limit this

Worked examples

Example 1: Identifying sampling methods

A researcher wishes to study student attitudes toward uniform policies in Caribbean secondary schools.

(a) The researcher obtains a list of all 500 students, assigns each a number 001-500, uses a random number generator to select 50 numbers, and surveys those students. Identify the sampling method. [1 mark]

Solution: Simple random sample (SRS). Each student has an equal probability of selection, and all possible samples of size 50 have equal probability of being chosen.

(b) Instead, the researcher divides students into forms (Year 7-11), then randomly selects 10 students from each form. Identify the sampling method and state one advantage of this approach. [2 marks]

Solution: Stratified random sample (1 mark). Advantage: Ensures representation from each form/year level, allowing comparisons between year groups or guaranteeing that smaller forms are not excluded by chance (1 mark).

Example 2: Experimental design

A pharmaceutical company wants to test whether a new medication reduces migraine frequency compared to a standard treatment.

(a) Identify the explanatory and response variables. [2 marks]

Solution:

• Explanatory variable: Type of medication (new vs. standard) (1 mark)
• Response variable: Migraine frequency (1 mark)

(b) Describe a completely randomized design for this experiment using 100 volunteers. [3 marks]

Solution:

1. Randomly assign 50 volunteers to receive the new medication and 50 to receive the standard treatment (1 mark)
2. Administer treatments for a specified period (e.g., 3 months) (1 mark)
3. Record and compare migraine frequency between the two groups (1 mark)

(c) Researchers know that migraine frequency differs by gender. Describe how blocking could improve the design. [2 marks]

Solution: Separate volunteers into male and female blocks (1 mark). Within each block, randomly assign half to the new medication and half to the standard treatment, ensuring gender is balanced across treatment groups (1 mark).

Example 3: Identifying bias

A radio station asks listeners to phone in their opinion on a proposed tax increase.

(a) Identify the main type of bias present and explain why it occurs. [2 marks]

Solution: Voluntary response bias (1 mark). People choose whether to participate, and those with strong opinions (especially opposition) are more likely to call in, creating an unrepresentative sample (1 mark).

(b) Suggest how the radio station could obtain more reliable data about public opinion. [2 marks]

Solution: Take a random sample of listeners or residents from the broadcast area (1 mark) and contact them directly to ensure representative participation rather than relying on self-selection (1 mark).

Common mistakes and how to avoid them

• Confusing random sampling with random assignment. Random sampling selects subjects from a population (for generalizability); random assignment distributes subjects to treatments (for causation). Both can occur in the same study but serve different purposes.

• Claiming causation from observational studies. Without random assignment, confounding variables may explain apparent relationships. Always state "association" or "correlation" rather than "cause" for observational data, unless the question explicitly asks you to consider other evidence.

• Incorrectly identifying stratified vs. cluster sampling. Stratified sampling takes some members from all groups; cluster sampling takes all (or many) members from some groups. Remember: stratified = divide and sample all strata; cluster = divide and select some clusters.

• Forgetting to describe randomization procedures completely. When asked to describe random assignment, specify exactly how randomization occurs (e.g., "use a random number generator" or "flip a coin for each subject"), not just "randomly assign."

• Misidentifying the experimental unit. The experimental unit is whatever receives the treatment. If 20 students each solve 5 problems under two conditions, the experimental unit is the student, not the problem.

• Overlooking the importance of sample size in experimental context. While large samples are good, proper randomization and control are more critical for valid experimental conclusions. A well-designed small experiment is superior to a poorly designed large one.

Exam technique for "Collecting Data"

• Command word precision: "Describe" requires a clear explanation of procedures; "identify" needs only naming the method; "explain" demands reasoning. For experimental design questions, describe randomization explicitly and mention how responses will be measured.

• Context matters: Always relate your answer to the specific scenario. Instead of "randomly assign subjects to groups," write "randomly assign the 60 volunteers to either the exercise program or the control group." Generic answers lose marks.

• Justification earns marks: When selecting a sampling method or experimental design, explain why it's appropriate. For example, "stratified sampling ensures all regions are represented" or "blocking by age controls for this known source of variability."

• Bias identification structure: Name the type of bias, explain its source, and describe how it affects results. Example: "Nonresponse bias occurs because busy people are less likely to complete the survey, potentially excluding their perspectives from the results."

Quick revision summary

Data collection methods determine what conclusions are valid. Random sampling allows generalization to populations; random assignment in experiments permits causal inference. Key sampling methods include simple random, stratified, cluster, and systematic sampling. Bias sources—undercoverage, nonresponse, response, and voluntary response—threaten validity. Well-designed experiments employ control, randomization, and replication, often incorporating blocking to reduce variability. Observational studies can only establish associations, not causation. Understanding these distinctions and applying them correctly to exam questions is essential for AP Statistics success.