Terms of Reference for Dual End-Term Evaluation & Scoping Assessment

“Combatting Food Insecurity in Northern Syria through Civil Society”

“Emergency Response to Earthquake Crisis”

Background:

The Norwegian People’s Aid (NPA) is a non-governmental organisation with roots in trade unions and labour movements. NPA supports processes towards democracy and equitable power distribution through mobilisation, popular participation, and collective organisation. The international strategy affirms civil society as a key pillar for nation-building, democracy, and development and views human rights as building blocks for development and redistribution.

NPA aims to support human worth and equal rights for all, irrespective of sex, disability, ethnicity, religion, age, sexual preference, or social status. The NPA vision is solidarity in practice. The vision demands commitment and action to protect life and health, build democratic societies, and strengthen people’s ability to master their lives. NPA is politically independent but not a politically neutral organisation.

NPA, with generous support from the Norwegian Agency for Development Cooperation (NORAD), initiated a response to mitigate the negative impact of the war in Ukraine on the global supply chains, which have driven up the price of food commodities, agricultural inputs and energy to record levels. NPA, along with its five strategic local partners (SHAR, BAHAR, DRD and KRD in Northeast Syria (NES), and MASRRAT in Northwest Syria (NWS)), initiated a set of different activities aiming to improve food security and combat hunger. Additionally, NPA was one of the front-line actors supporting the earthquake-affected population in northwest Syria. In response to the disaster in February 2023, NPA swiftly mobilised resources using its own fund for rapid response (RRF) and NMFA flexible fund, and accordingly implemented with two local partners, “MASARRAT” and “BAHAR”, an emergency response action to meet the immediate relief and survival needs of the affected population. The interventions of the emergency response (ER) included the distribution of essential supplies such as food and ready-to-eat kits (RTE), temporary shelter (tents) for those displaced by the earthquake, access to income-generating sources, and also offered mental health psychosocial support services to help communities recover and rebuild their lives.

Scope of the Dual end-line Evaluation & Scoping Assessment

The scope of this dual evaluation is to comprehensively assess two critical projects undertaken by Norwegian People’s Aid (NPA): the Food Security and Livelihoods project funded by the Norwegian Agency for Development Cooperation (NORAD), and the Emergency Response Action funded by NPA’s own resources and NMFA flexible fund. This evaluation must focus on assessing the extent to which each project met its intended goals and objectives, measure the positive and negative impacts of both projects on the respective target populations, assess how efficiently and effectively resources were utilised in implementing both projects and determine lessons, areas of improvement and the best practices that can inform future projects and strategies. The geographic coverage of the evaluation will be across northern Syria (non-regime areas), both NES and NWS, including but not limited to the governorates of Al-Hassakah, Ar-Raqqa, Aleppo Countryside, and Idlep. NPA and the downstream partners will brief the consultant consultancy firm on the accessibility and security situation of the target areas before the commencement of field visits and data collection processes, as well as NPA and the partners will collaborate with the assigned consultancy firm to determine feasible and safe methods for conducting data collection in these areas. The end-line evaluation should be performed separately for each project to ensure a thorough assessment tailored to their unique goals, challenges, and impacts, as this approach will allow for focused analysis, a better understanding of outcomes, impacts and more targeted recommendations. In parallel, NPA plans to expand the dual evaluation by supplementing it with a scoping assessment or baseline study focusing on key strategic themes for its future programming and proposals development to provide a comprehensive understanding of the current status (benchmarks), challenges and opportunities related to circular economy models, regenerative agriculture, value chains, environmental safeguarding, and disaster risk reduction (DRR). Additionally, the assessment should provide evidence-based information or recommendations on the best avenues to mobilise farmer’s associations/cooperatives and support them through advocacy efforts in northern Syria.

Objectives of the dual end-line evaluation & scoping assessment

The objective of the dual end-line evaluation is to provide a holistic understanding of the achievements, and key achieved results of the two projects, including the extent to which the projects have been adapted according to the changing context and how the projects have impacted the lives of the targeted population compared with the non-targeted population in the same locations; additionally, the objective of the end evaluation is to provide learnings and recommendations for future projects.

The specific objectives of the evaluation package include:

• To assess the project’s performance, pay particular attention to the impacts, immediate outcomes, and outputs of the project interventions against its key result indicators.
• To assess the relevance, effectiveness, efficiency, coherence, impact, and sustainability of the project implementation strategies. Emphasise community needs during projects’ design, results articulated in the log frame, and cross-cutting themes such as accountability, localisation, gender sensitivity, inclusiveness, conflict sensitivity, do-no-harm programming, disability inclusion, and capacity building of community structures. Consider the context in which the projects operated and its influence on projects’ successes and/or failures.
• To assess the complementarity synergy among the interventions of each project and the collaboration between NPA and its cooperating partners.
• To inform resource allocation for future rounds of proposals, as well as inform decisions on scaling-up, replication, and/or continuation, and test exit strategy considering institutions and projects’ synergies in the implementation area.
• To document lessons learned and success stories from each project and recommend future strategies.
• To complement the evaluation with a concise baseline study by providing a focused assessment of key themes related to circular economy models, regenerative agriculture, value chains, environmental safeguarding, disaster risk reduction (DRR), and the mobilisation of farmer’s associations/cooperatives to generate essential insights to inform evidence-based designing of future programming interventions.

Evaluation criteria

The overall evaluation should be structured around the OECD-DAC criteria and the additional criteria in the table below. The evaluation questions are indicative. In the initial stage of the evaluation exercise, the consultant is expected to review and improve the evaluation questions as appropriate and needed.

Evaluation Criteria

Key Questions

Relevance

Relevance assesses whether the project aligns with local needs and priorities and whether the intervention is effective.

• To what extent did the implemented activities address the food security, livelihood, and humanitarian needs of the population in the targeted locations?
• How do beneficiaries perceive the project’s relevance in addressing their core needs?
• Were the project’s objectives and activities aligned with the local community’s priorities and expectations?
• Were there any gaps between the project’s activities and the actual needs and priorities of the targeted population?
• To what extent was the programme able to adapt and appropriately respond to changing local needs and the priorities of affected people?

Efficiency

Efficiency concerns assessing how well resources were used to achieve intended objectives.

• To what extent were activities of individual partner organisations and for the overall consortium timely and cost-efficient?
• To what extent was the project implemented most efficiently compared to alternatives?
• What could have been done differently to complete the project more timely and efficiently?
• To what extent could funds be reallocated in time during implementation to respond to new developments?

Effectiveness

Effectiveness is about the extent to which the project has achieved its objectives.

• To what extent were the project targets and objectives achieved?
• How did the project’s results compare to the planned objectives and targets?
• What were the major factors influencing the achievement of the project’s objectives?
• Were the project’s activities and interventions successful in bringing about the desired changes

Impact

• How does the project impact the local economy and household food security?
• How have the project activities improved the lives of the targeted beneficiaries?
• What changes, positive or negative, has the project made to the beneficiaries regarding economic, social and cultural?
• What measures have been and can be taken to eliminate or reduce the negative impacts?
• Are there any success stories of positive change at the household or community level?

Sustainability

Sustainability, in the context of this intervention, concerns responding to acute and immediate needs while considering the longer term.

• Were activities carried out in a context that took longer-term and inter-connected problems into account?
• Did activities contribute to individuals being more prepared, resilient and less at risk than before?
• How likely will any positive changes be sustained in the short- and medium-term?
• Are there any factors that might strengthen sustainability? Is there any action the project should take to promote these factors?
• Are there any factors threatening the sustainability of project outcomes? How does the project seek to mitigate these risks?

Coherence

Coherence refers to the compatibility of the intervention with other interventions (and policies) in the country, sector and/or institution. The distinction is made between internal coherence (synergies with interventions carried out by the same institution/government and international norms and standards) and external coherence (consistency of the intervention with other actors’ interventions: complementarity, harmonisation, coordination and added value).

• To what extent is this intervention coherent with the other interventions with similar objectives in the area?
• To what extent was internal and external coherence of the project ensured?
• How did coordination and collaboration between the consortium members happen and contribute to coherence?

Other criteria

Gender sensitivity and inclusiveness

• Gender-sensitive programming was a mandatory requirement. Inclusiveness was required to ensure the needs of the most vulnerable were addressed.
• To what extent was a gender-sensitive programming approach applied in the project?
• How was the gender and age marker used?
• To what degree did the program address the needs of vulnerable groups?

Accountability

Accountability was mandatory to address throughout the project cycle.

• Were the project activities and outcomes transparently communicated to all stakeholders throughout the project lifecycle?
• To what extent was the affected population meaningfully informed and involved in the design and planning process and throughout implementation?
• What feedback mechanisms were in place? How has beneficiary feedback been addressed?

Conflict-sensitive and do-no-harm programming

• How were conflict-sensitivity and do-no-harm programming integrated into the project design and applied during project implementation?
• To what extent was a conflict-sensitive approach considered during all phases of the project life cycle? (Assessments, engagements, distributions, etc.)

Scoping Assessment

(Baseline Study)

• Circular Economy Models[1]: Assess the extent to which the agricultural sector has adopted circular economy principles and practices and identify opportunities for improvement. What models can be explored or developed under the agriculture, livestock, and services sectors?
• Regenerative Agriculture: Evaluate current agricultural practices and explore opportunities to promote regenerative agriculture techniques that enhance soil health and biodiversity. Share the key recommendations on feasible activities in regenerative agriculture.
• Value Chains: Analyze existing value chains and identify areas for optimisation to ensure fair and sustainable economic benefits for all stakeholders. Also provides an insight into the key value chain sectors and their sub-sectors can have the potential to be strengthened and supported in the context of Syria.
• Environmental Safeguarding: Assess the impact of agricultural activities on the environment and identify measures to mitigate negative effects while promoting sustainable resource management.
• Disaster Risk Reduction (DRR): Evaluate the resilience of agricultural systems to natural disasters and assess preparedness measures to minimise risks and enhance adaptive capacity.
• Mobilization of Farmer’s Associations/Cooperatives: Evaluate the organisational capacity and effectiveness of farmer’s associations/cooperatives in advocating for their interests and engaging in sustainable agricultural practices.
• An overview of the land reforms in Northern Syria (Northeast and Northwest Syria) and what can be the entry point to advocate for the farmer land rights in the context of Syria.

Evaluation Methodology.

The consultant is expected to determine the methodological approach for the end evaluation of this response project. The methodology should be further detailed in the inception report and will be approved by the Norwegian People’s Aid, including the assessment tools.

The consultant is expected to use and review existing project documents during the end evaluation. These documents include the project proposals and log frames, monitoring and progress reports, financial reports, policies and strategies, and any other relevant project document. Norwegian People’s Aid will provide the external consultant with all available project documentation at the beginning of the consultancy.

• A mixed research methodology is preferred, including collecting qualitative and quantitative data.
• Triangulation of data is important. Therefore, the consultant is expected to use multiple data sources to allow for the verification of results.

The consultant consultancy firm will determine the appropriate sample frame, select appropriate sampling types, and specify the sample size, considering the geographical areas, targeted groups, and the homogeneity of the target population, ensuring alignment with the evaluation’s objectives.

• If possible, the consultant (or the consultancy team) will undertake field visits to the projects implementation areas to collect primary data amongst various stakeholders, including project staff of international and local partners, local authorities, beneficiaries and other relevant persons. Participation of stakeholders in the evaluation should always be maintained, reflecting opinions, expectations, and visions about the project’s contribution towards achieving its objectives.
• The consultant must consider participants’ safety throughout the evaluation (including recruitment and training of research staff, data collection/analysis and report writing) as well as research ethics (confidentiality of those participating in the evaluation, data protection, age and ability-appropriate assent processes) and quality assurance (tools piloting, enumerators training, data cleaning).
• The consultant is expected to know and understand political sensitivities, consult the NPA Program team regularly and adapt the evaluation approach if needed.

MANAGERIAL ARRANGEMENTS / ROLES AND RESPONSIBILITIES

NPA Program Manager, Project Manager and NPA MEAL Coordinator (or designated staff) will be the main contact points for the entire evaluation process. The Project Manager/ MEAL Coordinator will be the Focal Point for day-to-day coordination and communication with partners and facilitating the consultant. Norwegian People’s Aid Syria Office will manage the contract and provide funding directly to the evaluator(s) according to the terms specified in the contract. The evaluator should make the necessary arrangements and coordinate with NPA staff before fieldwork to ensure no issues arise during field data collection. Relevant contacts will be shared with the evaluator.

Deliverables

The table below provides an overview of the expected deliverables.

Deliverables

Description

Timeframe

INCEPTION REPORT – this can be a consolidated report for the 2 end-line evaluations and the scoping study

Must contain:

• Logic of end-line evaluation proceedings based on desk review
• Findings fromthe desk review
• Revised/improved evaluation questions and an overview of how these will be answered
• Detailed work plan, research methods, sources, procedures for data collection, analysis, sampling of key indicators, etc.
• Proposed timeline of activities, schedule of tasks and submission of deliverables
• The report will be shared withNPA Programs and MEAL team for feedback and approval.

DRAFT REPORT

The draft report should be separate for each of the projects: FSL, Emergency response, and the scoping study.

• The draft report should structurally mimic the final report (see below for structure), address most of the assessment questions and work towards presenting meaningful findings, conclusions, and recommendations.
• The draft report willseparately present the tools used and findings (figures and graphs) of beneficiary data
• Draft report will be shared with NPA Programs and MEAL team for feedback and approval.

FINAL REPORT

The final report should be separate for each of the projects, which are FSL, Emergency response and the scoping study.

The final reportwill include the following:

1. Feedback on the draftreport has been addressed
2. Systematically assessthe project impacton beneficiary individuals and institutions
3. Provide factual evidence of direct and indirect results of interventions
4. Synthesize information received for purposes of conclusion and recommendation
5. Honest representation of observations fromthe desk review and primary data collection

The final report willconsist of thefollowing sections at a minimum:

1. Table of contents
2. Executive summary
3. Intervention description
4. Scope of the evaluation
5. Purpose and objectives of the evaluation
6. Methodology
7. Findings
8. Lessons learnt
9. Conclusion
10. Recommendations
11. Annexes – photos, human stories, final assessment tools, bibliography of secondary data used,and a listof persons interviewed.

The consultant will provide theraw data alongside the evaluation report.

5th August 2024 and at least five working days for the review by the NPA team

PowerPoint

Presentation

The consultant must develop and submit a high-quality PowerPoint presentation with the main findings and recommendations from the end-line evaluation.

The consultant is expected to present this to the NPA Program and MEAL team before finalizing the evaluation report.

2nd week of August 2024

The primary intended users of the evaluation include:

• NPA Syria-Iraq Country Offices
• NPA Head Office (MENA section and technical advisors)
• Norwegian Agency for Development Cooperation (NORAD)
• Local Partner Organisations

Required Qualifications

The specific requirements for this assignment are hands-on experience in evaluating an emergency response program implemented by international and national partners and experience in evaluating multi- sectoral humanitarian response programs. Additional required qualifications are detailed below.

1. Extensive experience in research work and assessments/evaluations. Knowledge of mixed research methodologies and application of various tools, including practical experience in assessments, planning, implementation, monitoring and evaluation of humanitarian interventions. Proven experience of using participatory methods is required.
2. At least a master’s degree in economics, developmental studies, business administration and social science or a related field for the lead consultant/ a minimum of bachelor’s degree in the relevant academic areas with seven years of progressive experience in research.
3. Strong experience in humanitarian response, and knowledge of humanitarian standards (CHS, Sphere, Code of Conduct).
4. Understanding of and experience in evaluating cross-cutting themes, including gender equality, accountability and conflict sensitivity / do-no-harm.
5. A good understanding of the Syrian context is required. Previous working experience in Syria is considered an advantage. The consultant/consultancy team needs to have access to the whole of Syria and the ability to collect data on short notice.
6. Excellent analytical and report-writing skills using statistical packages such as SPSS, STATA, etc.
7. Fluent in English. Understanding Arabic will be an added advantage.
8. Cultural awareness and the ability to operate in politically complex and sensitive environments are required.

[1] Circular economy models refer to economic systems that aim to minimize waste and make the most of resources by keeping materials and products in use for as long as possible. Instead of the traditional linear “take-make-dispose” approach, circular economy models promote strategies such as reusing, recycling, refurbishing, and remanufacturing to extend the lifespan of products and materials. In the context of agriculture, circular economy models may involve practices such as composting organic waste, using renewable energy sources, implementing sustainable farming techniques, and closing nutrient loops to regenerate soil health.

Timeframe

The assignment will take 45 calendar days, and the consultancy contract between NPA and the selected consultancy firm will be signed during the third or fourth week of June 2024. The preliminary draft report should be submitted to NPA by 25th July 2024. The final evaluation report should be submitted to NPA after three weeks of receiving NPA’s feedback on the draft report, at the latest, by 5th August 2024.

The consultant should present a detailed timeframe for each activity in the work plan, including working days for each activity.

Budget: The total budget should include the cost to implement the above-listed activities and deliverables per the table below. Evaluation Criteria: The application will be evaluated based on the following criteria: The key factors stated below will be taken into consideration during the evaluation process:

Criteria

Total obtained score

Demonstrated understanding, objectives, and completeness of the evaluation

15%

Methodology and implementation plan

15%

Details and quality (adequacy) of themethodology proposed for the evaluation

15%

Detailed implementation planindicating time frame

15%

Organization

• Experience in similar work in the last five years, client list, management control system.
• Exposure to working with the UN, international donors,international organizations, and other development agencies.
• Additional resources/logistics can be madeavailable to conduct the evaluation.

20%

Financial offer

20%

Evaluation Ethics:

Due to the dynamic and potentially hazardous nature of the operating environment for NPA and its partners, it is imperative that the consultancy firm/consultant strictly adhere to ethical and security guidelines. The purpose of this evaluation must not the safety or well-being of any individuals, and it should not seek information at the expense of others.

Consultancy firms or consultants will only be considered if their organizational or private consultancy profiles demonstrate a commitment to upholding ethical standards and appropriate protocols. They must show a keen awareness of the ethical and security challenges inherent in working within this context, prioritizing the protection of local communities and ensuring diligent data handling and management practices.

Gender and equal rights for women and girls.

NPA follows a gender-sensitive approach in its partnerships, organizational development, and programming, ensuring equal opportunities for women in partner organizations and beneficiary groups. The program aligns with the commitments outlined in Security Council Resolution 1325 on Women, Peace, and Security. NPA and its partners ensure that women and men have equal access to the program’s components. They also equally employ women and men within their organizational structures. This program aims to enhance the role of women in the decision-making process and promote women’s representation in local structures and involvement in local dialogues. Additionally, it aims to raise awareness among the targeted groups and locations.

Sources and Additional Readings:

• Programme Documents
• Programme Reports and Available Results Information
• NPA’s Current Strategy
• NPA Country Programme Strategies
• NPA Partnership Policy
• NPA Humanitarian Policy
• NPA Gender Policy
• NPA MEAL Policy