Theories of change & impact measurement (March 2021)
GFMD's role as a technical partner in PRIMED includes convening donors, media development practitioners and researchers.
On March 15th GFMD IMPACT brought together over 80 donors and funders, journalism support and media development practitioners and researchers to address some of the biggest questions facing our sector:
- How do we design programmes that have a real impact?
- And how should we measure impact?
- How do we ensure evidence and research is gathered and shared so that it is genuinely useful for our sector?
The meeting was co-convened by GFMD the Swiss Agency for Development and Cooperation (SDC) and the Center for International Media Assistance (CIMA).
As one of the meeting’s participants noted:
“We really need to share knowledge and learning about what works and what doesn't and to catalogue that knowledge so that we can get better results and be able to articulate those results.”
The meeting heard three case studies where donors, academics, and practitioners spoke about their collaborative approaches to learning, evaluation, and the effectiveness of their work.
Based on a study examining the use and perceptions of theories of change and result chains by donors and media assistance groups, the meeting discussed how to refine and improve how they are conceived and operationalised.
Looking at how a consortium of partners and multiple country teams can effectively develop learning questions and knowledge sharing mechanisms, and how this has been incentivised by the programme’s structure and metrics for success.
Presenting and discussing the case study of the Organised Crime and Corruption Reporting Project (OCCRP), who are pioneering using accountability as the basis for measuring impact: illicitly acquired funds returned to the public sphere; civic, government, and corporate actions; resignations and sackings; official investigations; arrests, indictments, and sentences.
The PRIMED case study was presented by
“One of the principal rationales and objectives of the programme - is to really work collaboratively, work in-country, and with this very broad group of collaborators internationally, to work out what media strategies that really work and work worth continuing to invest in. And what are the ones that are not worth investing in?”
The case study presented the PRIMED consortium's approach to learning and knowledge sharing:
One of the first activities of the PRIMED consortium was to develop 14 learning questions in consultation with organisations in the target countries Bangladesh, Ethiopia and Sierra Leone (plus two “learning countries” Zambia and Iraq).
A summary of each question -- why it has been identified as a priority, what kinds of learning are designed to be generated and how that will be crystallised and communicated -- can be found here: PRIMED learning questions.
"First of all, will they be useful to policymakers or practitioners both within but also especially beyond the programme and to really inform future media development practise? And if it’s not relevant on the ground, then is it relevant at all? And that’s a key second principle."
The PRIMED consortium's criteria for the learning questions considered:
- How they would build on and contribute to the sector's evidence-base
- The feasibility of answering them clearly
The PRIMED breakout group discussion:
- Heard feedback and ideas on the PRIMED learnings questions from academic and media development practitioners from outside the consortium.
- Identified clear areas for collaboration and knowledge sharing beyond the consortium.
- Debated the relationship between media development and other development goals, which was raised in the first case study on "Theories of change in media and governance programmes"
Of those who answered the post-event survey, who attended the PRIMED breakout group:
100% wanted to attend another event on this subject
75% rated the quality and effectiveness of the structure of the meeting as Good
100% rated the speakers, chairs and rapporteur as good or excellent
75% rated the relevance of themes and topics discussed good or excellent
“Global South media development approaches; donor information and learning systems (what are they?)” - International media development organisation
“Would love to hear media outlets talking about what impact means to them.” - International media development organisation
“The notion of impact raises the question of the absence of impact and obsolete intervention methodologies. Could practitioners, scientists and donors agree on a typology of interventions that can be assumed today to have had little or no impact?” - Academic/ researcher
“PRIMED's findings from learning questions would be really interesting--testing our assumptions about what indicators do (or don't) help us get at outcomes and/or impact.” - Donor/Funder
“Case studies of support for innovative and specialized reporting.” - Academic/researcher
For more detailed feedback about the meeting visit: