Evaluation and Research Guidance
Aligned with the 'what works' movement, What Works Behaviour Change emphasises the importance of robust evidence in developing theories and evaluating interventions and policies, and particularly in the importance of counterfactual approaches. This section provides links to a range of useful resources and guidance for undertaking research studies.
Government Guidance
Magenta Book - UK government guidance on evaluation including supplements on complexity, quality in qualitative evaluation, quality in impact evaluation and realist evaluation
Green Book - UK government guidance on economic appraisal
Aqua Book - UK government guidance on quality assurance of analysis
BMC and what works centre evaluation guidance
Causal inference based on counterfactuals - BMC overview of impact evaluation methods
Developing and evaluating complex interventions - BMC guidance on the iterative process of developing and evaluating interventions
Interrupted time series analysis - BMC guidance on a technique commonly used when a control group is not available
The case study approach - BMC guidance on undertaking case studies to gain rich data about people and their contexts
Using the framework method for the analysis of qualitative data - BMC guidance on using the Framework approach to data management, which was developed by the National Centre for Social Research
EEF evaluation guidance - includes RCT and IPE design guidance
YEF evaluation guidance - includes guidance on outcome measures and templates for protocols
The Maryland Scientific Methods Scale - What Works for Local Economic growth guidance on a the commonly accepted hierarch of evaluation evidence
Impact evaluation
Experimenter's Inventory - an accessible overview of counterfactual methods designed for a non-technical audience
Causal inference based on counterfactuals - an authoritative guide from the British Medical Council
Recent Developments in the Econometrics of Program Evaluation - an overview of econometric methods by two of the two leading figures in the world
Field Experiments - discussion of key features of real world experiments
CONSORT guidelines - definitive guidance for reporting the findings of RCTs
Interrupted time series analysis - BMC guide to a common approach to impact evaluation in the absence of a control group
Synthetic control methods - early paper setting out how to implement this powerful counterfactual technique
How much should we trust difference-in-difference estimates - sophisticated paper discussing technical issues and challenges involved in the most common quasi-experimental design
Robust data-driven inference in the regression-discontinuity design - in-depth guidance on how to implement RDD, one of the most robust counterfactual approaches after RCTs
A full course in econometrics - a superb introduction to the approach by the econometrician Ben Lambert
Causal inference mixtape - a brilliant online textbook covering econometric theory with examples and code in Stata and R
Econometrics 421 - in-depth and mathematical but brilliantly clear lectures on econometrics by the economist Mark Thoma
IPE evaluation and qualitative research
Process evaluation of complex intervention - very useful MRC guidance that provides a robust framework for implementation and process evaluations
The case study approach - useful BMC guidance on conduction case studies
Using the framework method for the analysis of qualitative data - BMC guidance on using the Framework approach to data management and analysis
Implementation and process evaluation guidance for EEF evaluations - guidance on undertaking process evaluations from the what works centre the Education Endowment Foundation
Qualitative Research Practice - widely used textbook that provides detailed practical guidance, including on using Framework
Critical Qualitative Theory: Opening up the Black Box of Criminal Justice Interventions - discussion of the implications of taking a critical realist approach to qualitative research
What Works Behaviour Change