Using experiments to tackle poverty: What it gets right, what it may not
This year’s Nobel Prize in Economics went to researchers who adopted an “experimental approach to alleviating global poverty”. While it has revolutionalised the field of development economics, the approach has its own limitations.
Abhijit Banerjee, Esther Duflo and Michael Kremer have been awarded The Sveriges Riksbank Prize in Economic Sciences in Memory of Alfred Nobel 2019, widely known as the Nobel Prize in Economics. There is great excitement in the Indian media as Abhijit Banerjee is only the second Indian (currently a US citizen) to have been awarded the Prize in Economics, the first being Amartya Sen. There is also wide praise that the award going to these three economists is an acknowledgement of the importance of studying poverty and approaches to alleviate the same. It is being seen as a shot in the arm for Development Economics as a discipline, which faced an onslaught from neoliberal economics as being irrelevant and as having largely failed in making policy recommendations. Questions of equity and poverty alleviation were being put on the back burner, to the dominance of the beliefs in free-market principles and the associated ‘efficiency’ that these were supposed to bring.
In this context, this award being given to the trio for “their experimental approach to alleviating global poverty” can be seen as a recognition by an influential body such as the Nobel Committee of the importance of studying such issues. This is not the first time that research on such issues has been appreciated by the Committee, a recent example being the prize for Angus Deaton a few years back “for his analysis of consumption, poverty, and welfare”. What is especially relevant about this years’ prize is that it not only appreciates the particular research of these individuals but also is an exoneration of the ‘experimental approach’. Abhijit Banerjee himself, in this interaction with the media after the prize was announced, said that this is a “prize not only for us but also for the entire movement”. This movement is one where experiments are conducted to ascertain what works and what doesn’t to inform policy on specific development problems.
The approach is mainly based on a method known as Randomised Control Trials (RCT), which are commonly done in the field of medicine to test the efficacy of new drugs. RCTs have now been in use for almost two decades to study the impact of small interventions on the lives of the poor, largely led by the J-PAL (a global research centre on poverty, co-founded by Banerjee and Duflo) in different parts of the world. This approach it is argued is the gold standard for ‘evidence’, as it can establish causality. One group of population is ‘treated’ with the policy and the changes in this group is compared with the ‘control’ group where all other things are common except for this one intervention. Observed changes in the treatment group in comparison to the control group can then be attributed to the intervention. For example, a randomly selected group of children are given de-worming tablets and another group is not. Other things remaining the same, at the end of the intervention if the group that received the tablets showed better improvements in test scores one can safely conclude that de-worming contributes to better learning outcomes. This is a simplistic explanation of the method, but this essentially is the crux.
Criticisms of the approach
There have been several valid criticisms of this approach. The almost evangelist zeal with which such ‘evidence-based policymaking’ is promoted creates the impression that rigorous evidence can only come from RCTs. However, not all policies can be tested by an RCT and other methods of research are also important and required. For instance, where to build highways or what kind of public transport systems need to be put in place are policy questions which an RCT cannot answer. Further, there are ethical issues with RCTs, the most critical being preventing the ‘control’ group from getting the intervention until the experiment is over. Questions have also been raised on the generalisability of the findings of RCTs. Just because something worked in one place does not guarantee that it will work everywhere.
The most important, and in some ways destructive, issue with RCTs is that in their approach they tend to depoliticize policymaking as well the problem with poverty itself. Structures and institutions that create poverty are not challenged and not even discussed. Further, in the real world, policies can never be devoid of politics. Whether something gets implemented or not depends as much, if not more, on political will rather than what ‘evidence’ there is in its favour. Moreover, such an approach belittles the agency and participation of people in policymaking, where claims that are ‘evidence-based’ are supposed to be more credible than those that are backed by people’s experiences and mobilisations. The popularity of RCTs has led to a trend of using these experiments as the only credible evidence on how societies and individuals function. Such an approach has no analysis of the role of social, political, historical and structural constraints while discussing interventions for reduction of poverty.
It is not being argued that every RCT is useless or that there is no value for policy-making to be better informed by research and data. The issue arises when the definitions of what is ‘rigour’ and what is ‘evidence’ get narrow based on a single approach despite all its limitations. Intentionally or not, the work of J-PAL and the laureates had created such a situation in development economics research. However, in the last few years, there has also been a lot of valuable and critical engagement with the issue, with many in the discipline recognizing that such a direction is not entirely desirable. The approach has highlighted the need to better understand the implications of some policies before implementing them on a large scale. Further, the research by the laureates has also contributed to bringing into mainstream development economics research into micro-questions such as what affects health-seeking behaviour or learning outcomes. These are important questions that could do with more research. At the same time, these issues are also better understood when they are placed in their specific social, political and historical context. While the experimental approach raises these important questions, it ignores these very crucial determinants at the same time.
(Dipa Sinha teaches at School of Liberal Studies, Ambedkar University Delhi.)