Are Randomized Evaluations Out of Control

In the developing world thousands of aid programs have been created with the aim to fight poverty. Do we know if these programs actually work? The most popular way to evaluate these aid programs is through a randomized trial. Take 1,000 people and split them into two: 500 people get help (treatment group) and 500 get nothing (control group). Compare the groups before your program to make sure they are the same. If they are then you can compare the two groups after the program has been running a few years then see the results. If the group that got helped is healthier, goes to school more, or has higher income then it is good evidence the program worked.

Esther Duflo is one of the leading champions of randomized trials. I like this 15 minute TED talk she gave below.



So what about the pitfalls of randomized evaluation. First what works in one country or region may not work in another. Second the evaluations are expensive.

Finally, in a blog post yesterday Alanna Shaikh argues that by focusing too much on randomized trials researchers and development agencies focus too much on immediate results that can be measured.

I think there is validity to all these criticism. However, I still do a lot of work with experiments where treatment is randomized. In part this is because of publication bias, it's much easier to get a randomized experiment published, because you don't have to argue as much what caused the effect you found.

My favorite thing to do is to take experiments and see how different conditions influence how the programs work.