Sampling methods and questionnaire design

What you'll learn

This revision guide covers the data collection methods you need for AQA GCSE Mathematics, focusing on how to select representative samples from populations and design effective questionnaires. You'll learn when to use different sampling techniques and how to identify problems in survey questions, both essential skills for the Statistics component of your exam.

Key terms and definitions

Population — the entire group of people or items being studied or investigated

Sample — a subset of the population selected to represent the whole group

Bias — a systematic error in sampling or questioning that makes certain outcomes more likely than others, producing unrepresentative results

Random sampling — a method where every member of the population has an equal chance of being selected

Systematic sampling — selecting members at regular intervals from an ordered list (e.g. every 10th person)

Stratified sampling — dividing the population into groups (strata) and sampling from each group in proportion to its size in the population

Primary data — information collected first-hand through surveys, experiments or observations

Secondary data — information that already exists, collected by someone else for a different purpose

Core concepts

Types of sampling methods

Random sampling

In random sampling, every member of the population has an equal probability of selection. This reduces bias but requires a complete list of the population.

How to conduct random sampling:

• Assign each member of the population a unique number
• Use a random number generator, calculator, or lottery method to select numbers
• Include the corresponding members in your sample

Advantages:

• Eliminates selection bias
• Results are statistically valid

Disadvantages:

• Requires a complete sampling frame (list of all members)
• Can be time-consuming and expensive
• May not represent small subgroups well

Systematic sampling

Systematic sampling involves selecting members at fixed intervals from an ordered list.

How to conduct systematic sampling:

• Calculate the sampling interval: population size ÷ sample size
• Select a random starting point between 1 and the interval
• Choose every nth member from that point

Example: To sample 50 students from 400, the interval is 400 ÷ 50 = 8. Start randomly (say, number 3), then select students 3, 11, 19, 27, etc.

Advantages:

• Simple and quick to implement
• Spreads sample evenly across population

Disadvantages:

• Can introduce bias if there's a pattern in the list
• Requires a complete list

Stratified sampling

Stratified sampling divides the population into distinct groups (strata) based on a characteristic (e.g. age, gender, year group), then samples proportionally from each stratum.

How to conduct stratified sampling:

• Identify relevant strata in the population
• Calculate the proportion of the population in each stratum
• Determine sample size for each stratum: (stratum size ÷ population size) × total sample size
• Randomly select the required number from each stratum

Example: In a school of 800 students (400 in Year 10, 400 in Year 11), to get a sample of 80:

• Year 10: (400 ÷ 800) × 80 = 40 students
• Year 11: (400 ÷ 800) × 80 = 40 students

Advantages:

• Ensures all subgroups are represented proportionally
• More representative than simple random sampling
• Allows comparison between groups

Disadvantages:

• Requires detailed information about population structure
• More complex to organize
• Strata must be clearly defined

Identifying bias in sampling

Sampling methods can produce bias when certain groups are over-represented or under-represented.

Common sources of bias:

Convenience sampling — choosing whoever is easiest to reach (e.g. asking only your friends). This is not representative as it excludes people who aren't readily available.

Voluntary response sampling — relying on volunteers who choose to participate. People with strong opinions are more likely to respond, creating bias.

Time/location bias — sampling only at certain times or places excludes those not present (e.g. surveying shoppers only on weekday mornings misses people at work).

Non-response bias — when selected people don't participate, and non-responders differ systematically from responders.

Questionnaire design principles

Well-designed questions produce reliable, unbiased data. Poor questions lead to unusable responses.

Characteristics of good questions

Clear and unambiguous:

• Use simple language everyone understands
• Avoid technical jargon unless surveying experts
• Each question should have one clear meaning

Relevant:

• Only ask questions necessary for your investigation
• Ensure respondents can reasonably answer

Unbiased and neutral:

• Avoid leading questions that suggest a preferred answer
• Don't use emotive language

Appropriate response options:

• Provide suitable choices for closed questions
• Ensure options don't overlap
• Cover all reasonable possibilities
• Include "Other" or "Prefer not to say" where appropriate

Problems with questions

Leading questions suggest or encourage a particular answer:

• ❌ "Don't you agree that homework is too difficult?" (implies difficulty)
• ✅ "How difficult do you find your homework?"

Ambiguous questions can be interpreted differently:

• ❌ "Do you regularly exercise?" (what counts as "regularly"?)
• ✅ "How many times per week do you exercise for at least 30 minutes?"

Personal or sensitive questions that people may not answer honestly:

• ❌ "How much do you weigh?"
• ✅ Provide weight ranges: 50-60 kg, 61-70 kg, etc.

Double-barreled questions ask two things at once:

• ❌ "Do you enjoy Mathematics and Science?"
• ✅ Ask separately about each subject

Overlapping response options:

• ❌ Ages: 0-10, 10-20, 20-30 (where does 10 or 20 go?)
• ✅ Ages: 0-10, 11-20, 21-30 or 0-<10, 10-<20, 20-30

Missing options:

• ❌ "How do you travel to school: walk, bus, car?" (excludes cycling, train, etc.)
• ✅ Include all reasonable options plus "Other (please specify)"

Response formats

Closed questions

Provide specific options to choose from. Easier to analyze but may miss unexpected responses.

Examples:

• Yes/No questions
• Multiple choice
• Rating scales (1-5, strongly disagree to strongly agree)
• Frequency options (never, sometimes, often, always)

Open questions

Allow respondents to answer in their own words. Provide detailed information but are harder to analyze.

When to use: For complex opinions or when you can't predict all possible answers.

Time periods and frequency options

Questions about frequency should provide clear, non-overlapping options:

✅ Good: "How often do you eat breakfast?"

• Every day
• 4-6 days per week
• 1-3 days per week
• Never

❌ Poor: "How often do you eat breakfast?"

• Often
• Sometimes
• Rarely (These are subjective and mean different things to different people)

Worked examples

Example 1: Stratified sampling calculation

Question: A leisure centre wants to survey 60 of its 480 members. The membership consists of 200 adults, 180 teenagers, and 100 children. Use stratified sampling to determine how many of each group should be surveyed. (3 marks)

Solution:

Adults: (200 ÷ 480) × 60 = 25 ✓

Teenagers: (180 ÷ 480) × 60 = 22.5 = 23 (round to nearest whole) ✓

Children: (100 ÷ 480) × 60 = 12.5 = 12 (round to nearest whole) ✓

Mark scheme:

• 1 mark for correct method shown for one group
• 1 mark for two correct answers
• 1 mark for all three correct (accept 25, 22, 13 or 25, 23, 12)

Note: When rounding stratified samples, ensure the total equals your target sample size (may need to adjust one group by 1).

Example 2: Identifying problems with questions

Question: Tom designs a question for his survey:

"How much time do you waste on social media each day? □ 0-2 hours □ 2-4 hours □ 4-6 hours"

Give two reasons why this is not a good question. (2 marks)

Solution:

Any two from:

• The word "waste" is biased/emotive language that suggests social media use is negative ✓
• The response options overlap (someone who uses social media for exactly 2 hours could tick two boxes) ✓
• There's no option for people who use social media for more than 6 hours ✓
• There's no option for people who don't use social media at all ✓

Mark scheme:

• 1 mark for each valid criticism
• Accept equivalent descriptions of the problems

Example 3: Sampling method comparison

Question: A headteacher wants to find out students' opinions about the school canteen. The school has 1200 students.

(a) She asks the first 30 students who arrive at school one morning. Why might this sample be biased? (1 mark)

(b) Describe a better method she could use to obtain a representative sample. (2 marks)

Solution:

(a) Students who arrive early may have different characteristics/circumstances from those who arrive later (e.g. they might not use the canteen because they eat breakfast at home) ✓

OR: Only samples students who arrive early / doesn't give everyone an equal chance of selection ✓

(b) Use stratified sampling by year group ✓

Work out how many to sample from each year group proportionally / randomly select students from each year group in proportion to year group size ✓

OR: Give each student a number and use a random number generator to select 30 students ✓ ensuring she samples across different times/places to avoid time bias ✓

Mark scheme:

• (a) 1 mark for explaining why the sample doesn't represent all students
• (b) 1 mark for naming an appropriate method; 1 mark for explaining how to implement it

Common mistakes and how to avoid them

• Confusing population and sample — Remember: the population is everyone you're interested in; the sample is the subset you actually survey. Always identify both clearly.

• Incorrect stratified sampling calculations — Always multiply by the total sample size, not the stratum size. Formula: (stratum size ÷ population size) × total sample size. Check your answers add up to the required sample size.

• Identifying only one problem with a question — Exam questions often ask for two criticisms. Look for: bias, overlapping options, missing options, ambiguity, sensitive content, and double-barreled questions.

• Not justifying why a sampling method is biased — It's not enough to say "it's biased" — explain which groups are excluded or over-represented and why this matters.

• Describing how to improve a question without identifying the original problem — When asked to criticize a question, state what's wrong first, then suggest improvements if asked.

• Forgetting that systematic sampling needs a random start — Simply selecting every nth person from position 1 isn't truly systematic; you must randomly choose your starting position.

Exam technique for "Sampling methods and questionnaire design"

• "Describe" and "Explain" command words — "Describe" requires you to state what method to use and outline the steps. "Explain" needs reasons why it's appropriate or how it avoids bias. For stratified sampling questions, always show your calculations.

• Two-mark question structure — For questions about problems with questions/sampling, you typically need two distinct points. Don't make the same criticism twice in different words. Common mark allocation: 1 mark per valid problem identified.

• Show your working for sampling calculations — Even if the answer seems obvious, write the calculation (stratum ÷ population × sample size). Marks are often awarded for method even if the final answer is wrong.

• Use mathematical terminology precisely — Say "stratified sampling proportional to year group size" rather than "choosing some from each year." Use "bias," "representative," "random selection" correctly to access full marks.

Quick revision summary

Sampling methods: Random sampling gives everyone equal selection chance; systematic sampling selects at regular intervals; stratified sampling proportionally represents different groups. Calculate stratified samples using (stratum size ÷ population) × sample size. Identify bias by considering which groups are excluded or over-represented. Good questions are clear, unbiased, have non-overlapping response options, and cover all reasonable answers. Watch for leading language, ambiguous terms, missing options, and overlapping categories when critiquing questions.