Saturday, May 12, 2012

Predictably Irrational (Experimental Design)

Interesting TED Talk by Dan Ariely, a professor of Psychology and Behavioral Economics at Duke University, on irrationality in decision-making. I saw this a few years ago and found it excellent for starting conversations with Stats students about experimental design, bias, and (if later in the year) practice with hypothesis testing.

Here are some questions to start discussion:
  1. Regarding the data summarized in a bar chart regarding % of drivers donating organs in the various European countries, was it collected through a study or an experiment? Explain.
  2. According to the video, what was the difference between the countries with high percentage and the countries with low percentage of organ donors?
  3. Sketch a possible experimental design diagram for the experiment involving doctors and pain medication hip replacement.
  4. Sketch a possible experimental design diagram for the experiment involving subscription to the Economist. What was the population of interest? To what extent should the conclusion be drawn?
  5. Sketch a possible experimental design diagram for the experiment regarding ugly Tom and ugly Jerry. Was there a control group in this experiment (based on just the information provided)? Explain.
  6. What bias is being investigated in the video?

Here are some screenshots from the video to set context for question 4:
Treatment 1:

Treatment 2:

This is an AP Test style sample answer to question 4:

Lastly, check the conditions and perform an appropriate hypothesis test using the data from the video. Here's the screenshot with the results:

That's all I have come up with for now. I would appreciate suggestions for improvement.

This is one of the more memorable videos for kids, especially the ugly Tom and ugly Jerry's effect on Tom and Jerry respectively. Students got to see examples of experiments and how they were designed so that conclusions can be drawn. I think it would be even better if the students also got to read the original papers, especially if the papers use the statistical methods learned at the high school level.


  1. Great post - if I get to teach AP Stats again, I'll use it. Thanks

    1. I hope your students will enjoy seeing interesting applications of stats as much mine did.

      You might also like the video on Human Computation by Luis von Ahn. It's a little longer (runs about an hour) than TED talks. You may be able to work it into one of those days when half the class is gone. It's not about stats, but there are many examples from which to ask interesting questions.