Modelling and field studies: Essential M&E tools to understand our impact

Healthy livestock are the cornerstone of many communities’ economies and well-being, making access to high-quality, safe, and effective animal health solutions critical to secure small-scale producers livelihoods and avoid substantial economic losses in Africa and South Asia.

At GALVmed, we partner with commercial, governmental, and non-governmental actors to increase the availability and accessibility of vaccines and essential animal health products to small-scale livestock producers (SSPs) in Africa and South Asia. However, it is equally important to ensure that these products are purchased and used by farmers to prevent or treat livestock and poultry diseases. Adoption of animal health medicines goes beyond a simple purchase; it triggers a cycle that improves animal health, boosts productivity, and enhances the wealth and well-being of these producers and their communities.

Understanding how adoption translates into economic and other benefits for SSPs is central to our mission. Monitoring and Evaluation (M&E) gives us a structured way to assess these outcomes. It is not just a check box or data collection exercise; it is about understanding the difference we are making in SSPs’ lives.

Given the vast scope and scale of our work across multiple countries and contexts, modelling is one of our key tools for understanding adoption and impact. Modelling refers to the use of mathematical and statistical methods, using epidemiological and economic data, to simulate and evaluate the potential effects and outputs of a programme or intervention. This is a powerful tool that helps organisations make informed decisions. Modelling allows us to process veterinary product sales data and estimate the resultant economic benefits for SSPs. This helps us to see and quantify how our interventions are impacting small-scale producers.

For example, in 2023 we partnered with Supporting Evidence based Interventions-Livestock (SEBI-L) to apply the model to GALVmed PLSHL 2 initiatives between 2014 and 2017. The model estimated a total NEB of $105.1M in economic benefits to the 3,664,114 estimated customers reached.

To complement our modelling work, we also conduct field studies to gain deeper insights into the changes in SSPs’ lives. These field studies allow us to gain contextual understanding to better understand the context within which our initiatives are embedded. They also allow us to dive deeper into the changes occurring in the lives and livelihoods of SSPs. By combining modelling with on-the-ground research, we get a more holistic understanding of our impact, ensuring that we can learn from  both our successes and areas where we can improve.

Ensuring the availability and the adoption of veterinary products is central to sustainable and resilient agricultural practices. From a systems perspective, GALVmed ‘s transformational market systems change framework will help us to understand the systems changes driven by our partnerships and efforts to improve access to and adoption of veterinary medicines. This framework will help us see where and how our work is contributing to long-term, systemic change, ensuring that we make a lasting impact on those who need it most.

This blog was written by Katharine Tjasink, Senior Manager, Impact, Evaluation & Learning, as part of the campaign ”M&E: Designing for Impact”.

Monitoring and Evaluation: Designing for impact

At GALVmed, Monitoring and Evaluation (M&E) is not an afterthought. It is integrated from the very start of the development of new projects and programmes. This approach ensures that our projects are designed with a clear vision of impact. It also ensures that we, and our funders, have realistic expectations on what can be achieved within a project’s lifetime.

There are three key areas where we provide early input:

  1. Modelling the prospective impact of an initiative: Whenever possible, we forecast the potential economic benefits of a project for small-scale livestock producers (SSPs) before it even starts using a prospective modelling approach. This forward-looking method provides insights into the potential economic returns, in terms of animal lives saved and productivity regained, for SSPs purchasing critical livestock vaccines and other animal health products. Before a project begins, stakeholders and funders can understand the investment potential, and this helps decision making on where to allocate resources.
  2. Creating a Theory of Change: The Theory of Change is an important part of our process. It clearly outline the changes we want to see, how the project will lead to those changes, the expected timelines for impact, and the key assumptions we are working with. This clarity ensures that everyone involved in the project has a common understanding. The Theory of Change also helps us identify potential risks and challenges early on so we can course correct to keep our initiatives on track.
  3. Developing an evaluation framework: A framework that clearly defines the OECD DAC evaluation criteria we will use, along with the associated evaluation questions, indicators, data sources, sampling methods, timing, and data analysis approaches, ensures that we establish a clear plan from the beginning on how we will assess and measure the impact of our initiatives. This framework facilitates a shared understanding of our evaluation process.

By integrating M&E from the early stages of project design and implementation, our initiatives are strategically designed to maximise impact for the small-scale livestock producers we serve.

This blog was written by Katharine Tjasink, Senior Manager, Impact, Evaluation & Learning, as part of the campaign ”M&E: Designing for Impact”.

Using a Randomised Control Trial to study the impact of Newcastle Disease vaccine on poultry farmer welfare and livelihoods

In 2020, Oxford Policy Management (OPM) was contracted by GALVmed to implement an intervention and conduct an associated impact study on the adoption of a Newcastle Disease Vaccine (NDV) by small-scale poultry farmers in rural Tanzania in the districts of Chemba and Mbozi. The objective of the study is to quantify the causal effects that the delivery of NDV has on the “production, productivity, and livelihoods of small-scale producers (SSPs)”. The study involves two main activities:

  1. The design and implementation of an NDV intervention in selected SSP farming areas of Tanzania.
  2. The design and implementation of an experimental study to quantify the causal effects of the NDV intervention.

The impact study was designed as a randomised controlled trial (RCT) where the study sample was randomly split into one treatment group and one control group. The treatment group was offered and will continue to be offered the NDV intervention package. This group will be compared with a control group, who did not and will not receive the intervention package during the study. The control group will receive one round of the intervention after the study’s endline survey.

A baseline study was conducted between September and November 2021 and the endline survey is scheduled for September to November 2023. Further details on the RCT and its findings will be made available upon publication of the results.

Blog written by Lamyaa Al-Riyami