Tag: M&E

Modelling and field studies: Essential M&E tools to understand our impact

Posted on by team galvmed

Healthy livestock are the cornerstone of many communities’ economies and well-being, making access to high-quality, safe, and effective animal health solutions critical to secure small-scale producers livelihoods and avoid substantial economic losses in Africa and South Asia.

At GALVmed, we partner with commercial, governmental, and non-governmental actors to increase the availability and accessibility of vaccines and essential animal health products to small-scale livestock producers (SSPs) in Africa and South Asia. However, it is equally important to ensure that these products are purchased and used by farmers to prevent or treat livestock and poultry diseases. Adoption of animal health medicines goes beyond a simple purchase; it triggers a cycle that improves animal health, boosts productivity, and enhances the wealth and well-being of these producers and their communities.

Understanding how adoption translates into economic and other benefits for SSPs is central to our mission. Monitoring and Evaluation (M&E) gives us a structured way to assess these outcomes. It is not just a check box or data collection exercise; it is about understanding the difference we are making in SSPs’ lives.

Given the vast scope and scale of our work across multiple countries and contexts, modelling is one of our key tools for understanding adoption and impact. Modelling refers to the use of mathematical and statistical methods, using epidemiological and economic data, to simulate and evaluate the potential effects and outputs of a programme or intervention. This is a powerful tool that helps organisations make informed decisions. Modelling allows us to process veterinary product sales data and estimate the resultant economic benefits for SSPs. This helps us to see and quantify how our interventions are impacting small-scale producers.

For example, in 2023 we partnered with Supporting Evidence based Interventions-Livestock (SEBI-L) to apply the model to GALVmed PLSHL 2 initiatives between 2014 and 2017. The model estimated a total NEB of $105.1M in economic benefits to the 3,664,114 estimated customers reached.

To complement our modelling work, we also conduct field studies to gain deeper insights into the changes in SSPs’ lives. These field studies allow us to gain contextual understanding to better understand the context within which our initiatives are embedded. They also allow us to dive deeper into the changes occurring in the lives and livelihoods of SSPs. By combining modelling with on-the-ground research, we get a more holistic understanding of our impact, ensuring that we can learn from  both our successes and areas where we can improve.

Ensuring the availability and the adoption of veterinary products is central to sustainable and resilient agricultural practices. From a systems perspective, GALVmed ‘s transformational market systems change framework will help us to understand the systems changes driven by our partnerships and efforts to improve access to and adoption of veterinary medicines. This framework will help us see where and how our work is contributing to long-term, systemic change, ensuring that we make a lasting impact on those who need it most.

This blog was written by Katharine Tjasink, Senior Manager, Impact, Evaluation & Learning, as part of the campaign ”M&E: Designing for Impact”.

Monitoring and Evaluation: Designing for impact

Posted on by team galvmed

At GALVmed, Monitoring and Evaluation (M&E) is not an afterthought. It is integrated from the very start of the development of new projects and programmes. This approach ensures that our projects are designed with a clear vision of impact. It also ensures that we, and our funders, have realistic expectations on what can be achieved within a project’s lifetime.

There are three key areas where we provide early input:

  1. Modelling the prospective impact of an initiative: Whenever possible, we forecast the potential economic benefits of a project for small-scale livestock producers (SSPs) before it even starts using a prospective modelling approach. This forward-looking method provides insights into the potential economic returns, in terms of animal lives saved and productivity regained, for SSPs purchasing critical livestock vaccines and other animal health products. Before a project begins, stakeholders and funders can understand the investment potential, and this helps decision making on where to allocate resources.
  2. Creating a Theory of Change: The Theory of Change is an important part of our process. It clearly outline the changes we want to see, how the project will lead to those changes, the expected timelines for impact, and the key assumptions we are working with. This clarity ensures that everyone involved in the project has a common understanding. The Theory of Change also helps us identify potential risks and challenges early on so we can course correct to keep our initiatives on track.
  3. Developing an evaluation framework: A framework that clearly defines the OECD DAC evaluation criteria we will use, along with the associated evaluation questions, indicators, data sources, sampling methods, timing, and data analysis approaches, ensures that we establish a clear plan from the beginning on how we will assess and measure the impact of our initiatives. This framework facilitates a shared understanding of our evaluation process.

By integrating M&E from the early stages of project design and implementation, our initiatives are strategically designed to maximise impact for the small-scale livestock producers we serve.

This blog was written by Katharine Tjasink, Senior Manager, Impact, Evaluation & Learning, as part of the campaign ”M&E: Designing for Impact”.

Evaluating the effects of Newcastle Disease vaccination on poultry production and livelihoods

Posted on by team galvmed

Backyard chickens are a significant source of income and nourishment in developing countries, but outbreaks of poultry diseases like Newcastle Disease (ND) severely affect productivity, flock mortality, and consequently, farmer livelihoods.

GALVmed has made important progress in combating ND with the successful development of two thermotolerant vaccines suitable for the rural environment. Vaccination is a cost-effective means to controlling ND. However, to fully grasp the benefits of vaccination interventions and establish evidence-based approaches for developing future programmes and planning, we need to measure vaccination impacts comprehensively. In order to understand and quantify the causal effects of ND vaccination on poultry production and livelihoods, in 2020, GALVmed contracted Oxford Policy Management (OPM) to design and implement, in collaboration with Tufts University, a Newcastle Disease Vaccine (NDV) intervention paired with a rigorous evaluation in rural areas of Tanzania.

The study encompasses two main activities. Firstly, a Newcastle Disease vaccination intervention in selected small-scale farming areas of Tanzania, specifically in the districts of Chemba and Mbozi. To execute this intervention plan, Community Vaccinators are tasked to visit each registered small-scale producer (SSP) household at their doorstep to:

  1. Vaccinate chickens with the I-2 Newcastle Disease vaccine, which is administered by the eye-drop method.
  2. Deliver specific training and knowledge content on the disease, vaccination, and improved poultry-rearing practices.

The second activity involves an experimental study to quantify the causal effects of the Newcastle Disease vaccination intervention. To assess these effects, the impact study was structured as a randomised controlled trial (RCT) where the study sample of intervention villages was randomly divided into two groups for comparison: a treatment group that received the ND vaccination intervention package (vaccination and training), and a control group that did not. The control group will receive one round of the intervention after the study’s endline survey.

Both the vaccination intervention and the impact study are currently underway.  OPM has completed all 6 vaccination rounds in the study’s treatment villages with one pending vaccination round to control villages. The households that were selected for the impact study have been interviewed at the project’s outset, followed by a midline assessment, and will again be interviewed at the end of the intervention. The midline survey revealed preliminary positive outcomes with reduced ND outbreaks and chicken mortality due to NDV delivery.

Underlying this intervention is a key assumption that offering NDV at an affordable price to poultry-keeping households in treatment villages, together with the promotion of improved poultry-rearing practices, will yield noteworthy benefits with respect to poultry productivity, income from poultry, household welfare, food consumption, and possibly decisions on income-generating activities.

The findings derived from this study will allow us to make informed decisions regarding ND vaccination and further enhance the effectiveness of our efforts in protecting poultry health and livelihoods.

Further details and final findings will be made available at the end of the intervention in 2024.

Quantifying our impact: A modelling framework to estimate the economic benefits of our initiatives

Posted on by team galvmed

As an organisation with a wide geographic footprint across Africa and South Asia, it is challenging to assess the impact of our programmes at scale using rigorous field studies. However, together with our wider network of funders and partners, we need to know that the investments we are making are producing productivity and income benefits for small-scale livestock producers (SSPs) that warrant the financial investment in these initiatives.

In this regard, we partnered with Supporting Evidence based Interventions-Livestock (SEBI-L) to develop a model for practical use for our market development programmes. The model is used to estimate the economic impact of the initiatives on SSPs, prioritise product development decisions, and to direct market development effort. Furthermore, these analyses can be used to advocate for further investment in the SSP animal health sector.

In a paper published in Frontiers in Veterinary Science, the model was applied to estimate the impact of products sold during GALVmed’s People and Livelihoods 2 (PL2) programme. The PL2 programme, which was implemented between 2014 and 2017 in Africa and South Asia, supported the production and distribution of poultry anthelminthics and vaccines against Newcastle disease, fowl pox, sheep and goat pox, peste des petits ruminants, and East Coast fever.

The modelling framework

The model is conceptualised in terms of three components:

  • Products: this includes sales of products and the number of animals that are expected to be treated with the product (depending on the different pack sizes).
  • Disease epidemiology: this comprises the conditions that are treated, number of infections, mortality rates and impact on growth rate.
  • Economics: this comprises losses from reduced productivity and losses from livestock mortality.

The economic impacts from mortality and growth inhibition are estimated at the individual animal level for poultry, small ruminants, and cattle. As SSPs are using veterinary products to prevent or minimise loss due to disease, we model the key ways in which those losses are experienced by SSPs and estimate the proportion of those losses that are averted by using specific animal health products. By factoring in the cost of the product and the number of doses sold, we give the net economic benefit (NEB).

The model can be adapted to incorporate new products and parameters as needed. The framework will evolve as GALVmed initiatives change over time.

The results

The model estimates a total NEB of $105.1M to the 3,664,114 customers reached by the PL2 initiatives. This translates to $139.9M in present value, and $37.97 on average per customer, many of whom were small scale poultry producers.

Within Sub-Saharan Africa, the greatest net economic benefit was realized from vaccines against East Coast fever and Newcastle disease, while in South Asia, peste des petits ruminants and Newcastle disease vaccines had the greatest net economic benefits.

The paper with the complete results and analysis is available here: A high level estimation of the net economic benefits to small-scale livestock producers arising from animal health product distribution initiatives.

By understanding how GALVmed’s interventions translate into economic benefit for SSPs, we can continuously refine and optimise our approaches, ultimately driving a greater positive change in the economic progression and well-being of SSPs across Africa and South Asia.

This blog was written by the M&E team.

Learning lessons at GALVmed

Posted on by team galvmed

The collection and analysis of data allow organisations to develop and implement impactful and evidence-based strategies. GALVmed’s Commercial Development and Impact Function is committed to a developmental approach to monitoring, evaluation, and learning (MEL), which means that we use rapid learning to improve a project programme or affirm the need for a change of course.

To be truly developmental, this learning needs to feed back in a systematic way into our organisation. How do we do that? We use a framework for lessons learning to ensure that the lessons are integrated back into GALVmed. The Collaboration, Learning and Adapting (CLA) framework, developed and used by USAID, is a useful approach that we have adapted to frame our learning process. USAID defines CLA as “a set of processes and activities that help ensure programming is coordinated, grounded in evidence, and adjusted as necessary to remain effective throughout implementation” (ADS 201, 2016).

CLA is based in the understanding that development projects operate in complex systems, which shift over time. In this changing landscape, the objectives that we set out at the beginning of a project or programme can be affected over a three or five year period of performance. Responding in an adaptive manner ensures that we keep moving towards having a positive impact. The adaptive decisions we make must be based in evidence, which is why we are integrating processes for CLA into the beginning of our projects or programmes and adjusting these throughout the project or programme life cycle. Essentially, this is an approach to learning that fits in perfectly with our developmental approach.

There are three main steps in our learning process:

  1. The first step is to accurately define and diagnose the problem (or best practice). Some problems are logistical or operational and have easy fixes. But these, while important, do not usually make or break a project. Others are deeper and more complex and require a lot more thought and reflection to unpack them and to understand how to respond to them. Sometimes there are multiple causes to a problem, with some of them being root causes.
  2. The second step is to categorise the problem (or best practice). The use of categories ensures key information is not missed and helps to focus our thinking on lessons. To ensure consistency across projects, there are standard categories for each project and additional categories specific to a project can be added as needed.
  3. The third step is to define a road map for the way forward. This involves defining what action needs to take place, and how to know when that action has succeeded. At the end of a lessons learning exercise, there should be a road map that takes us from a lesson to action and adaptation.

Throughout this process, active collaboration is key to ensuring that we view our lessons from multiple perspectives and that our stakeholders have a voice.

This blog was written by Katharine Tjasink