Malawi local consultant for project evaluation
- Talitha Hlaka, Communications Officer
Project Performance Assessment Report (PPAR) on the Malawi Nutrition and HIV/AIDS Project P125237
Terms of Reference for a Local Consultant, October, 2019
The World Bank’s Independent Evaluation Group (IEG) is seeking to recruit a health, nutrition or social sector/community development specialist with applied experience in data collection, analysis and evaluation (or an evaluation expert or economist with applied experience in social sectors, especially health and nutrition) for a short-term assignment to support an international consultant in undertaking an evaluation of a World Bank-financed Malawi Nutrition and HIV/AIDS Project.
The Project Development Objectives (PDOs) expressed in the Financing Agreement were “to increase access to and utilization of selected services known to contribute to reduction of stunting, maternal and child anemia, and the prevention of HIV and AIDS in children and sexually active adults”. This was amended “to increase coverage of selected nutrition, HIV and AIDS services and strengthen disease outbreak preparedness in project areas”. To this end it supported three components: (1) Support for Nutritional Improvement; (2) Support for the National HIV/AIDS Strategic Plan; and (3) Support for Ebola Preparedness. The Project was financed by an IDA commitment of US$102.6 million and a multi-donor trust fund of US$12.4 million. It was approved on March 27, 2012, became effective in October 17, 2012, and closed on August 31, 2018.
The PPAR serves an accountability purpose by evaluating the extent to which the operation achieved its intended outcomes. It also seeks to draw lessons to inform and guide future investments in education, health and nutrition. This assessment complements, respectively, the Implementation Completion and Results (ICR) Report prepared by the Bank’s operations teams with Borrower contributions and IEG’s desk review of the ICR (ICR Review), by providing an independent, field-based assessment more than two years after the project’s closing. This project was chosen for a field evaluation because of its potential for lessons and because of its relevance to a planned IEG evaluation of the World Bank’s support to nutrition. As such, the focus on nutrition will be emphasized. Evaluation of this operation provides an opportunity to deepen the learning, particularly with regard to: its multi-sectoral approach to nutrition, institutional/organizational framework for nutrition coordination and implementation, the role of community care groups in nutrition behavioral change and service delivery, supply- and demand-side factors and the role of M&E in strategic program management.
The local consultant will support the international consultant in collecting and analyzing data and information to assess the development effectiveness of the Project. The evaluation will assess: the relevance of the operation; its design; the extent to which stated objectives were achieved; World Bank’s (and other partners’) contribution to outcomes; the efficiency with which the operation was designed and implemented; its sustainability; the performance of the World Bank and the Borrower; the design, implementation and effective use of monitoring and evaluation systems. And it will distill lessons from the findings that may contribute to improved performance in Malawi and also be of broader relevance.
PPAR’s Objectives-Based Methodology
Assessment of the achievement of the objectives includes (but is not limited to): describing the theory of change (causal narrative); documenting the inputs, outputs, outcomes, and impacts, including activities of other donors; establishing the counterfactual and assessing contribution or attribution (e.g., what were other donors doing; what other developments might have produced the same outcomes; what would have happened without the project; what happened in the neighboring, non-project provinces, regions, or countries that are similar?)
The PPAR should use all data available in assessing the operation. Evidence for the assessment will be collected through:
- Reviewing internal project documents (Project Appraisal Document for the project, Implementation Completion Report (ICR) ICR Review, Project Supervision Reports, Aide Memoirs, correspondence, etc.) in both Washington and Malawi.
- Reviewing the published economic, policy, and education research (e.g., analytic work; published literature; where available, evidence from impact evaluations of project activities or the complete operation (or similar operations);
- Collecting (systematically) data in the field, where possible.
- Analyzing data collected by third parties, where available.
- Interviewing key Bank staff, Government officials, and relevant sector representatives and stakeholders, and donors in Washington and Malawi concerning the project’s performance.
- Conducting field visits and interviews with beneficiaries and implementers to observe first-hand the achievements and to collect additional data, as necessary.
- Mining available data to document the extent to which the operations achieved their objectives, including data noted in the ICR Review that could have been utilized in the ICR.
Duties and Accountabilities of the Local Consultant
Outputs and responsibilities of the local consultant will be further defined by the international consultant, depending on the profile of the local consultant recruited, but will likely focus on the collection, analysis and graphic presentation of the project’s implementation and outcomes. The local consultant will prepare text and present relevant data and trends (in graphs and tables) to document the efficacy of the projects against their stated targets and objectives. He or she will also undertake tasks prior to the mission to ensure its good preparation: collection of documents and data; and development of a detailed agenda and strategy for the mission with the international consultant, including coordination of interviews and field visits.
Specific support to the international consultant is likely to include many of the following tasks, once profile and comparative advantages are assessed, and some of which will be undertaken jointly with the international consultant:
- Identifying and synthesizing relevant literature, research and policy/operational documents, especially evaluations of multisectoral and community-based nutrition efforts in Malawi;
- Identifying and compiling a synthesis of key development partners’ support to nutrition sectors during the project period to understand how the project contributed to these efforts;
- Identifying key actors and stakeholders for visits and interviews, including those f government actors who were involved in the implementation of the project nationally and sub-national, as well as NGOs, experts, community groups, and key development partners who are knowledgeable about the components and activities of the project;
- During the stay of the international consultant in Malawi:
- Participating in and facilitating interviews with key stakeholders;
- Participating in the field visits within Malawi to areas targeted under the projects to observe first-hand the achievements and to collect additional data, as necessary;
- Identifying and mining available data in the country to document the extent to which the operations achieved their objectives;
- Analyzing and providing advice and input to the international consultant on the project costs and financing and contributing to the economic analysis and assessment of efficiency of the projects and helping to prepare a theory of change for the project;
- Preparing a diagram of the institutional arrangements for multisectoral nutrition in the country, including the community care groups for nutrition.
- Providing the international consultant with guidance and insights on the country background and context and on protocols for the respectful and productive conduct of the mission.
Timing of this intervention will be linked to the visit of the international consultant to Malawi, scheduled for November, 2019 (subject to Government confirmation). For a total of 30 days, the consultant will:
- read background documents; prepare for the mission agenda; prepare the diagram of the institutional actors involved in nutrition (7 days);
- work full-time with the international consultant for the duration of her visit (13 days);
- undertake follow-up/wrap-up work, including the filling of any remaining information gaps, the pursuit of unanswered questions and the revision/finalization of the local consultant’s specific inputs to the PPAR. (10 days).
- Advanced degree in public health, nutrition, health economics, or community/social development, with data collection/analysis and evaluation experience; OR a qualified, experienced evaluator with applied experience in nutrition, public health and/or social/community development
- Track record of research in/analysis of health/nutrition issues
- Knowledge and experience of community-level development
- Strong analytical, conceptual and problem-solving skills demonstrated through research or evaluation on social issues
- Strong quantitative skills, familiarity with qualitative research methodologies
- Demonstrated capacity to analyze and interpret empirical evidence, ability to think strategically and draw relevant lessons using evaluative material
- Strong written and oral communication skills
- Ability to work effectively in teams
- Ability to work under minimal supervision
- Familiarity with the nutrition program in Malawi
- Language proficiency in English and local languages.
Please send your application with CV to: Ann Flanagan, Task Team Leader, IEG World Bank, at email@example.com