Improving impact evaluation production and use

Nicola Jones, Harry Jones, Liesbet Steer and Ajoy Datta
Working Paper:300 March 2009 - ISBN: 978 0 85003 899 6
Overseas Development Institute

Available online PDF [87p.] at:
http://www.odi.org.uk/resources/odi-publications/working-papers/300-impact-evaluation-production-use.pdf

“……The past five years have seen a proliferation of impact evaluations (IEs) by development agencies across the globe. This report was commissioned by the UK Department for International Development’s (DFID’s) Evaluation Department to inform discussions on impact evaluation production and use within the Network of Networks Impact Evaluation Initiative (NONIE). It builds on an initial scoping study prepared for DFID which made recommendations on improving IE production and use, focusing on clustering, coordination, knowledge management, capacity strengthening and communication and uptake. This the report goes further by expanding both the literature review and the annotated database of IEs, as well as honing in on specific dynamics of IE production across sectors.

A focus on sector-specific histories and dynamics of impact IE production, communication and use dynamics revealed a number of important similarities and differences. Similarities included a growing recognition of the need to approach IEs as part of a broader monitoring and evaluation system; the importance of involving multiple stakeholders in the evaluation process to promote uptake; and the utility of exploring alternative methods to assess impact.

Key differences appeared to be starker and were found in a number of areas. First, a longer history of IEs in health and agriculture/natural resource management (NRM) sectors has meant these sectors have a broader knowledge base from which to draw, although they diverge in the extent to which this knowledge is actually used.
The paper concludes with policy implications regarding: strategic coordination, funding, knowledge management, capacity strengthening mechanisms and improving impact evaluation communication and uptake…..”

Contents
Executive summary
1. Introduction
2. Key issues in the relevance, production and use of impact evaluations
2.1 IE: Concepts, methods, nature of the knowledge produced
2.2 Supply and demand: Commissioning, production and delivery of IEs
2.3 Use and influence of IEs
3. Sectoral case studies
Case 1: Human and social development
Case 2: Agriculture and renewable natural resources
Case 3: Humanitarian aid
Case 4: Rural/urban development and infrastructure sector
Case 5: Impact evaluations of results-based aid
4. Comparing sector-specific experiences with impact evaluations
5. Conclusions and policy implications
References

Appendix 1: Impact evaluation database overview and findings
Appendix 2: Media coverage of impact evaluation findings
Appendix 3: Stepwise evaluation model
Appendix 4: Key informants