Flawed much - a hunt for flaws in experiments

A teacher wants to know if chewing gum helps students focus. She gives gum to only the top-performing students and observes their attention during class.

• 1. The flaw is that she didn't test enough types of gum.

• 1. She should have told the students the purpose of the study before observing them.

• She should have randomly selected students instead of choosing only top performers.

• She should have used a survey instead of observing them.

A researcher tests whether a new memory app improves recall. She asks her friends to try the app for a week and then take a memory test.

• 1. The study fails because she didn’t test memory on a weekend.

• 1. The app should have been deleted after the study.

• She should have used a random sample instead of just her friends.

• 1. She should have used a different brand of phone for each participant.

A psychologist wants to test if loud music affects problem-solving. She tests one group in the morning in a quiet room, and another group in the afternoon with loud music.

• 2. Participants should have been allowed to choose the music they liked.

• 1. The flaw is that the same music wasn’t played in both sessions.

• She should have tested both groups at the same time of day.

• 1. The psychologist should have worn headphones during the experiment.

A student wants to know if boys are better than girls at math. She gives a difficult math test to the girls and an easier one to the boys.

• 1. The girls should have been given more time.

• 1. She should have allowed calculators for both groups.

• 1. She should have used the same test for both groups to ensure fairness.

• She should have marked leniently to avoid hurt feelings.

A teacher compares reaction times of students who play video games with those who don’t.

• 
  She should have randomly assigned participants to groups to eliminate pre-existing differences.

  
  

• She should have tested the same students twice.

• She should have told them which games to play.

• The teacher should have asked students to stop gaming for a week.

A scientist lets participants choose whether to sleep 4 or 8 hours before a memory test.

• Participants should have been randomly assigned to sleep conditions.

• Only visual puzzles should have been used.

• Participants should have eaten breakfast.

• The test should have been done at night.

A counselor tests if meditation reduces stress by asking students if they feel less stressed after 5 minutes, but has no control group.

• She should have used a control group and measured stress levels before and after.

• She should have used yoga poses.

• Students should have sat in the dark.

• She should have played classical music.

A nutritionist gives fruit daily to 10 students to test its effect on memory, but doesn’t test memory before or after.

• She should have used tropical fruits.

• She should have measured memory both before and after the week.

• She should have checked if students liked fruit.

• She should have given juice instead of fruit.

A coach tests a new drill on only his best athletes without comparing to others.

• The athletes should have had protein shakes.

• He should have tested athletes from different ability levels and included a comparison group.

• He should have posted videos on social media.

• The athletes should have worn favourite shoes.

A psychologist makes one group anxious but doesn’t measure their stress before a test.

• She should have said it was a relaxation study.

• She should have measured stress levels before testing to confirm the manipulation worked.

• Participants should have smiled before starting.

• She should have provided snacks.