Modelling and field studies: Essential M&E tools to understand our impact

Healthy livestock are the cornerstone of many communities’ economies and well-being, making access to high-quality, safe, and effective animal health solutions critical to secure small-scale producers livelihoods and avoid substantial economic losses in Africa and South Asia.

At GALVmed, we partner with commercial, governmental, and non-governmental actors to increase the availability and accessibility of vaccines and essential animal health products to small-scale livestock producers (SSPs) in Africa and South Asia. However, it is equally important to ensure that these products are purchased and used by farmers to prevent or treat livestock and poultry diseases. Adoption of animal health medicines goes beyond a simple purchase; it triggers a cycle that improves animal health, boosts productivity, and enhances the wealth and well-being of these producers and their communities.

Understanding how adoption translates into economic and other benefits for SSPs is central to our mission. Monitoring and Evaluation (M&E) gives us a structured way to assess these outcomes. It is not just a check box or data collection exercise; it is about understanding the difference we are making in SSPs’ lives.

Given the vast scope and scale of our work across multiple countries and contexts, modelling is one of our key tools for understanding adoption and impact. Modelling refers to the use of mathematical and statistical methods, using epidemiological and economic data, to simulate and evaluate the potential effects and outputs of a programme or intervention. This is a powerful tool that helps organisations make informed decisions. Modelling allows us to process veterinary product sales data and estimate the resultant economic benefits for SSPs. This helps us to see and quantify how our interventions are impacting small-scale producers.

For example, in 2023 we partnered with Supporting Evidence based Interventions-Livestock (SEBI-L) to apply the model to GALVmed PLSHL 2 initiatives between 2014 and 2017. The model estimated a total NEB of $105.1M in economic benefits to the 3,664,114 estimated customers reached.

To complement our modelling work, we also conduct field studies to gain deeper insights into the changes in SSPs’ lives. These field studies allow us to gain contextual understanding to better understand the context within which our initiatives are embedded. They also allow us to dive deeper into the changes occurring in the lives and livelihoods of SSPs. By combining modelling with on-the-ground research, we get a more holistic understanding of our impact, ensuring that we can learn from  both our successes and areas where we can improve.

Ensuring the availability and the adoption of veterinary products is central to sustainable and resilient agricultural practices. From a systems perspective, GALVmed ‘s transformational market systems change framework will help us to understand the systems changes driven by our partnerships and efforts to improve access to and adoption of veterinary medicines. This framework will help us see where and how our work is contributing to long-term, systemic change, ensuring that we make a lasting impact on those who need it most.

This blog was written by Katharine Tjasink, Senior Manager, Impact, Evaluation & Learning, as part of the campaign ”M&E: Designing for Impact”.

Monitoring and Evaluation: Designing for impact

At GALVmed, Monitoring and Evaluation (M&E) is not an afterthought. It is integrated from the very start of the development of new projects and programmes. This approach ensures that our projects are designed with a clear vision of impact. It also ensures that we, and our funders, have realistic expectations on what can be achieved within a project’s lifetime.

There are three key areas where we provide early input:

  1. Modelling the prospective impact of an initiative: Whenever possible, we forecast the potential economic benefits of a project for small-scale livestock producers (SSPs) before it even starts using a prospective modelling approach. This forward-looking method provides insights into the potential economic returns, in terms of animal lives saved and productivity regained, for SSPs purchasing critical livestock vaccines and other animal health products. Before a project begins, stakeholders and funders can understand the investment potential, and this helps decision making on where to allocate resources.
  2. Creating a Theory of Change: The Theory of Change is an important part of our process. It clearly outline the changes we want to see, how the project will lead to those changes, the expected timelines for impact, and the key assumptions we are working with. This clarity ensures that everyone involved in the project has a common understanding. The Theory of Change also helps us identify potential risks and challenges early on so we can course correct to keep our initiatives on track.
  3. Developing an evaluation framework: A framework that clearly defines the OECD DAC evaluation criteria we will use, along with the associated evaluation questions, indicators, data sources, sampling methods, timing, and data analysis approaches, ensures that we establish a clear plan from the beginning on how we will assess and measure the impact of our initiatives. This framework facilitates a shared understanding of our evaluation process.

By integrating M&E from the early stages of project design and implementation, our initiatives are strategically designed to maximise impact for the small-scale livestock producers we serve.

This blog was written by Katharine Tjasink, Senior Manager, Impact, Evaluation & Learning, as part of the campaign ”M&E: Designing for Impact”.

Quantifying our impact: A modelling framework to estimate the economic benefits of our initiatives

As an organisation with a wide geographic footprint across Africa and South Asia, it is challenging to assess the impact of our programmes at scale using rigorous field studies. However, together with our wider network of funders and partners, we need to know that the investments we are making are producing productivity and income benefits for small-scale livestock producers (SSPs) that warrant the financial investment in these initiatives.

In this regard, we partnered with Supporting Evidence based Interventions-Livestock (SEBI-L) to develop a model for practical use for our market development programmes. The model is used to estimate the economic impact of the initiatives on SSPs, prioritise product development decisions, and to direct market development effort. Furthermore, these analyses can be used to advocate for further investment in the SSP animal health sector.

In a paper published in Frontiers in Veterinary Science, the model was applied to estimate the impact of products sold during GALVmed’s People and Livelihoods 2 (PL2) programme. The PL2 programme, which was implemented between 2014 and 2017 in Africa and South Asia, supported the production and distribution of poultry anthelminthics and vaccines against Newcastle disease, fowl pox, sheep and goat pox, peste des petits ruminants, and East Coast fever.

The modelling framework

The model is conceptualised in terms of three components:

  • Products: this includes sales of products and the number of animals that are expected to be treated with the product (depending on the different pack sizes).
  • Disease epidemiology: this comprises the conditions that are treated, number of infections, mortality rates and impact on growth rate.
  • Economics: this comprises losses from reduced productivity and losses from livestock mortality.

The economic impacts from mortality and growth inhibition are estimated at the individual animal level for poultry, small ruminants, and cattle. As SSPs are using veterinary products to prevent or minimise loss due to disease, we model the key ways in which those losses are experienced by SSPs and estimate the proportion of those losses that are averted by using specific animal health products. By factoring in the cost of the product and the number of doses sold, we give the net economic benefit (NEB).

The model can be adapted to incorporate new products and parameters as needed. The framework will evolve as GALVmed initiatives change over time.

The results

The model estimates a total NEB of $105.1M to the 3,664,114 customers reached by the PL2 initiatives. This translates to $139.9M in present value, and $37.97 on average per customer, many of whom were small scale poultry producers.

Within Sub-Saharan Africa, the greatest net economic benefit was realized from vaccines against East Coast fever and Newcastle disease, while in South Asia, peste des petits ruminants and Newcastle disease vaccines had the greatest net economic benefits.

The paper with the complete results and analysis is available here: A high level estimation of the net economic benefits to small-scale livestock producers arising from animal health product distribution initiatives.

By understanding how GALVmed’s interventions translate into economic benefit for SSPs, we can continuously refine and optimise our approaches, ultimately driving a greater positive change in the economic progression and well-being of SSPs across Africa and South Asia.

This blog was written by the M&E team.

Using a Randomised Control Trial to study the impact of Newcastle Disease vaccine on poultry farmer welfare and livelihoods

In 2020, Oxford Policy Management (OPM) was contracted by GALVmed to implement an intervention and conduct an associated impact study on the adoption of a Newcastle Disease Vaccine (NDV) by small-scale poultry farmers in rural Tanzania in the districts of Chemba and Mbozi. The objective of the study is to quantify the causal effects that the delivery of NDV has on the “production, productivity, and livelihoods of small-scale producers (SSPs)”. The study involves two main activities:

  1. The design and implementation of an NDV intervention in selected SSP farming areas of Tanzania.
  2. The design and implementation of an experimental study to quantify the causal effects of the NDV intervention.

The impact study was designed as a randomised controlled trial (RCT) where the study sample was randomly split into one treatment group and one control group. The treatment group was offered and will continue to be offered the NDV intervention package. This group will be compared with a control group, who did not and will not receive the intervention package during the study. The control group will receive one round of the intervention after the study’s endline survey.

A baseline study was conducted between September and November 2021 and the endline survey is scheduled for September to November 2023. Further details on the RCT and its findings will be made available upon publication of the results.

Blog written by Lamyaa Al-Riyami

GALVmed discusses impact

Impact is an important topic for any philanthropic organisation and GALVmed is putting this topic front and centre of our agenda for 2022. The primary reason is that we are in the process of finalising and beginning to implement our ten-year strategy, and it is vitally important that we integrate the lessons we have learned so far and align on the topic of impact.

To kickstart this process, three workshops were held over the end of January and beginning of February 2022 with the aim to provide a common, organisational understanding of impact. We took a look at our record of impact and discussed some of the associated key lessons learned from the three main programmes GALVmed has delivered to date, namely the first and second Protecting Livestock, Saving Human Life programmes (PLSHL 1 and PLSHL 2), and the Veterinary Innovations Transforming Animal Health and Livelihoods programme (VITAL). These workshops constituted the first phase of a collective look at impact within the organisation.

A second phase is being led by the evaluation team, which operates under the Commercial Development and Impact department (CD&I) at GALVmed. Lasting eight weeks, the primary purpose is to collate further data, present findings to our donors and board, and most importantly, to implement actionable findings into the Commercial Development, Research and Development, Evaluation, and Enabling Environment programmes under the new strategy.

The key activities include taking lessons learned from previous programmes of work and considering the implications for new GALVmed projects and programmes, creating a theory of change for GALVmed at an organisational level, in which the GALVmed mission is clearly stated and pathways to impact explained, and linking impact to GALVmed’s overall assessment of organisational performance. Through this process we intend to identify our potential for impact in the new strategy as well as the key levers and drivers for change.

This blog was written by Katharine Tjasink