I love sports analogies. I love the way the way NYU professor William Easterly thinks about development. But I have to nitpick.

In a post at his blog, Aid Watch, he writes about what March Madness would look like if teams were coached by development economists. I get where he’s going with this but he made a bit of a mistake:

Other Professors such as Duflo, Banerjee, and Karlan set up randomized trials for which plays work. Treatments included 3-point shots, driving layups, pick and roll, and passing to the open player, compared to a control group holding the ball still. The results were of considerable interest, but players got very confused trying to remember which study to cite and apply in each pressure-packed moment of the game. They did not make the Sweet Sixteen.

That is not how randomized trials work. Comparing a team trying out different plays to a team that is holding the ball still is the equivalent of comparing someone who is receiving a new treatment to someone who is holding their breath. The ‘control’ part of randomized control trials isn’t a group of people doing absolutely nothing. It’s a group of people doing everything normally except not getting the treatment being tested.

So a better analogy would be to compare points per game, say, or winning percentage of  basketball teams all trying out the same new play to teams that play the way they always do. It might seem nitpicky but if we’re going to criticize randomized trials–and we should, particularly in development contexts–we should at least be criticizing a realistic version of them.

The part about thinking about knowing which research to cite in every moment of a pressure-packed game is bang on though. Randomized trials can give us good knowledge of development interventions but it’s much harder to know when to apply them.