Open to international and national candidates

Ref No: CfA 22-07/23

1. BACKGROUND

The International Organization for Migration (IOM) is implementing the project Addressing Mixed Migration Flows in the Western Balkans. The general objective of the action is to support the migrant response in the Western Balkans, by increasing the reception, referrals and registration capacities, by providing access to assisted voluntary return and reintegration (AVRR) to migrants, and by increasing migration data and knowledge on risks and vulnerability in the region. The project is managed by the IOM mission in Bosnia and Herzegovina (BiH), with activities in BiH, Montenegro, North Macedonia, Serbia and Kosovo[1]. The expected outcomes of the project are:

1. Authorities and humanitarian responders have access to accurate, up-to-date data and enhanced knowledge on migrants arriving to, and present in the Western Balkans. 
2. Vulnerable migrants have access to increased reception and referrals capacities in migrant facilities across the region.  
3. Migrants in transit in the Western Balkans have access to return and reintegration assistance.
4. Local authorities have strengthened capacities to address the needs of communities most affected by mixed migration movements in Republika Srpska.

As part of the project cycle, IOM is seeking an experimented evaluator or evaluation team to conduct the final, external evaluation of the project. This summative evaluation will give information about the effectiveness of the project and provide the stakeholders, including donors, the opportunity to use best practices identified during the evaluation.

• SCOPE OF THE ASSIGNMENT AND TASKS

The final evaluation will cover the full scope of the project activities and the entire duration of the project, from 1 December 2021 to 30 November 2023.  Field visits will focus on the activities implemented in BiH and Montenegro. Due to the synergy and merging of project activities, the evaluation can include findings and recommendations on the Migrants Assistance in Republika Srpska (MARS) project.  More specifically, the tasks of the evaluator will be:

• In the application submitted to IOM as a response to the present call and Terms of Reference for the evaluation attached, clearly indicate additional needs and expectations regarding IOM support during the field visits.
• Independently plan and implement the data collection, analysis, and reporting activities of the final evaluation, notably the field visits and methodological planning (desk review, key informants’ interviews, focus group discussions, surveys, etc.) and quality assurance.
• Coordinate and communicate effectively with the IOM team, keep the Evaluation Manager abreast of the process and flag any challenges or emerging issues.
• Prepare and deliver the inception report, the draft and final report, the evaluation brief and any visual presentation of the evaluation of the findings and recommendations.

The internal evaluation is expected to generate key deliverables:

Inception Report: Based on the standard IOM template for inception reports. Scoping document that develops a more detailed context analysis and scoping for the assignment based on the desk review. The report focuses on the methodology and workplan. It includes an Evaluation Matrix to demonstrate the evaluator(s) understanding of the Term of reference and outline key performance indicators that will be used to assess achievements of results. It should include an indicative list of interviewees and data collection tools (e.g. questionnaires, interview guides). Moreover, the inception report should highlight clearly where primary data shared by the project is not sufficient and what tools are proposed to collect missing information. The inception report should be reviewed and endorsed by the EM prior to the start of the active phase of the evaluation.

Draft Evaluation Report: Following the desk review, field visits and data collection and analysis phase, the evaluator(s) will draft a preliminary version of the report, key findings and recommendations, which will be submitted to the EM and the PMT for review and feedback. When possible, the service provider will present its main findings and recommendations to IOM and partners during a meeting to optimize the review and feedback process/timeline.

Final Evaluation Report: Based on the standard IOM template for final evaluation reports. The service provider will finalize the information, addressing IOM's comments and suggestions. The report will include an executive summary of no more than two pages, consisting of a brief description of the project, its context and current situation, the purpose of the evaluation, its methodology, and main findings, conclusions and recommendations. The final report will be produced in English. Notwithstanding the nature of the findings, which are expected to be balanced and well informed, the quality of the writing, the structure and logical flow of ideas, as well as the general presentation, will be subject to IOM’s review and need to be deemed satisfactory by the EM for the evaluation process to be finalized. 

Evaluation Brief: Based on the standard IOM template for evaluation briefs. The service provider will develop a two-page brief summarizing project information, methodology, findings and recommendations that will be shared with the EM for review. The brief will accompany the final report and be shared with the donor, partners and internally, within IOM. The brief should be produced in English and of quality deemed ready for publication.

Draft Management Response Matrix: Based on the standard IOM template for management response, the evaluator(s) will partially fill out the matrix with the recommendations, after which the PMT will be responsible to complete and ensure fulfilment as per IOM evaluation guidance.

• TIMEFRAME OF THE CONSULTANCY

The maximum timeframe for this consultancy is from 01-09-2023 to 31-01-2024.

IOM is looking for an external evaluator with experience designing and conducting evaluations in relevant thematic and geographic context, with demonstrated  qualifications as follow:

1. Experience
2. A minimum of five years of experience conducting evaluations is required.
3. Experience in geographic and thematic areas relevant to the project a strong advantage.
4. Skills in evaluation design, qualitative data collection and analysis, drafting and editing in English and time management are required (to be demonstrated in application).

• Education
• Advanced degree in a relevant field such as social sciences or statistics is required.

• Competencies and skills
• Professionalism: demonstrates ability to work in a composed, competent and committed manner and exercises careful judgment in meeting day-to-day challenges.
• Delivering results: produces and delivers quality results in a service-oriented and timely manner; is action-oriented and committed to achieving agreed outcomes.
• Accountability: takes ownership for achieving the Organization’s priorities and assumes responsibility for own action and delegated work.
• Communication: encourages and contributes to clear and open communication; explains complex matters in an informative, inspiring and motivational way.

• Language(s):
• Fluent English. Command of languages used in the Western Balkan region an advantage.
• REPORTING

The evaluator will report to the Evaluation Manager for the purpose of the consultancy. Specific reports to be provided are provided in the list of deliverables. Timelines for the deliverables are specified in the Terms of Reference attached.

• CONFIDENTIALITY ARRANGEMENTS

Contracted consultant or individual members of a consultancy organization are not permitted to use any of the gathered data, reported findings, contributions to the draft and final reports to further their personal research.

• LOCATION AND TIME OF THE CONSULTANCY

The consultant will work remotely for the most part of the consultancy. Travel will be required during the field visits. The proposal submitted as part of this call for application should be all-inclusive of the costs of travel and field visits.

Applications should be sent via email to: bihvacancies@iom.int with the email subject ”Application: Evaluator – CfA 22-07/23”.

Documents to submit as a part of the application should include:

• Letter of expression of interest highlighting qualifications and recent experience.
• Technical proposal demonstrating excellent understanding of the assignment and outlining the proposed methodology, data collection plan, quality assurance plan, assessment of timeline, feedback for consideration by IOM when needed.
• CV and list of evaluation reports with titles, year of publication, information on commissioner.
• Separate financial proposal: detailed budget for consultancy, including consultancy fees and all expenses related to travel, accommodation, subsistence, field visits, interpretation, etc.).
• Copies (or excerpt) of at least 1 recent evaluation report. Max. 3 years.

The consultant is responsible for obtaining entry visa (if necessary) for field visits, and will be required to use their own computers, printers, photocopier etc.

Application deadline: 01 August 2023.

Posting period: 14 July 2023 – 01 August 2023.

ANNEX TO THE CFA – Terms of Reference for the External Evaluation of the project

“Addressing Mixed Migration Flows in the Western Balkans”

funded by the Royal Kingdom of Norway

Commissioned by the IOM mission in Bosnia and Herzegovina

The International Organization for Migration (IOM) is implementing the project Addressing Mixed Migration Flows in the Western Balkans. The general objective of the action is to support the migrant response in the Western Balkans, by increasing the reception, referrals and registration capacities, by providing access to assisted voluntary return and reintegration (AVRR) to migrants, and by increasing migration data and knowledge on risks and vulnerability in the region. The project is managed by the IOM mission in Bosnia and Herzegovina (BiH), with activities in BiH, Montenegro, North Macedonia, Serbia and Kosovo[2]. The expected outcomes of the project are:

1. Authorities and humanitarian responders have access to accurate, up-to-date data and enhanced knowledge on migrants arriving to, and present in the Western Balkans. 
2. Vulnerable migrants have access to increased reception and referrals capacities in migrant facilities across the region.  
3. Migrants in transit in the Western Balkans have access to return and reintegration assistance.
4. Local authorities have strengthened capacities to address the needs of communities most affected by mixed migration movements in Republika Srpska.

Through this project, IOM aimed to support the systematic collection and analysis of migration flow data, and provide critical information on migrants’ locations, profile, vulnerability and needs. Building on more reliable and up-to-date information, the project sought to increase the knowledge and skills of relevant national institutions and humanitarian actors to improve the migration response. 

In addition, IOM supported protection-sensitive assistance to migrants in transit, by increasing the accommodation and reception capacities of existing reception facilities in the WB, by supporting the functioning of these facilities, and by providing migrants, refugees and asylum seekers with access to adequate WASH, food and non-food items.

Furthermore, IOM worked to strengthen access to AVRR by providing information and return counselling, so migrants can make informed decisions and avail of the return and reintegration pathways offered to them. The activities focused on outreach to migrants in and out of Temporary Reception Centres (TRCs), information and counselling in coordination with IOM staff in the countries of return, including virtual counselling, when possible, as well as referrals and protection assistance to vulnerable migrants, in coordination with local partners and authorities.

Finally, to support the migration response in BiH, the project helped improve health and social services structures in communities most affected by migration in the BiH entity of Republika Srpska (RS).

The general objective of the external evaluation is to assess the achievement of results and sustainability of action and draw recommendations for the way forward based on the projects’ progress and likely impact. This summative evaluation will give information about the effectiveness of the project and provide the stakeholders, including donors, the opportunity to use best practices identified during the evaluation.

On the one hand, the evaluation shall inform an internal audience – IOM country office team, specialized units of regional and global IOM offices – so that the lessons learned and recognized good practices are internalized and benefits implementation of future responses. On the other hand, the evaluation will offer the donor an overall analysis of the impact of the project at the end of its implementation. The finding of the evaluation could be used by various stakeholders, for instance:

• Donor can rapidly assess the progresses of the project to achieve the objectives, results and outcomes, as indicated in the logical framework.
• Project team can document lessons learned and best practices from a completed set of activities and highlight the added value of the activities delivered during the project.
• Senior management can assess organizational effectiveness in implementing strategy, including the effectiveness of project management systems and tools (M&E protocols and frameworks),
• Stakeholders, the donor, humanitarian, and development actors can assess the relevance and accountability of a project to intended beneficiaries.

The final evaluation will cover the full scope of the project activities conducted in Albania, BiH, Montenegro and Kosovo*. The evaluation will cover the entire duration of the project, from 1 December 2021 to 30 November 2023.  The field visits will focus on the activities implemented in BiH and Montenegro. Due to the synergy and merging of project activities in the second year of the project, the evaluation can include findings and recommendations on another project funded by Norway during the same period (ref. to the Migrants Assistance in Republika Srpska (MARS) project). 

The evaluation will be based on the standard established by the UNEG and use the OECD/DAC[3] criteria of relevance, coherence, efficiency, effectiveness, impact, and sustainability. The evaluator(s) will have to develop a detailed evaluation matrix and inception report which will refine the below evaluation questions:

Relevance:

1. Are the project activities and outputs consistent with the intended outcomes and objectives?
2. Does the project meet the needs of the beneficiaries?

Coherence:

• To what extent does the project create synergies and linkages with other interventions implemented in the same thematic area, or previous projects implemented by IOM?
• Do the project activities and outputs consider relevant national policies and IOM guidelines?

Efficiency 

• How well are the resources (funds, expertise, and time) being converted into results?
• Were the project activities undertaken, and were the outputs delivered on time?

Effectiveness: 

• Have the project outputs and outcomes been achieved in accordance with the stated plans?
• What are the major factors influencing the achievement of the project's expected outcomes?
• Are the target beneficiaries satisfied with the services provided?

Impact

1. Does the impact come from the project activities, from external factors, or both?
2. Which negative or unintended effects are being produced by the project, and did the project team take timely measures to mitigate any unplanned negative impacts?

Sustainability

1. To what extent have target groups and possibly other relevant interest groups/stakeholders been involved in the planning/implementation process?
2. Are there plans and strategies in place for IOM, partners and beneficiaries to be able to continue to build on and benefit from the results of the project? ?

The evaluation will be based on available qualitative and quantitative data collected from key stakeholders at different points in time. Project documents and reports will be made available to the evaluator. Four data/information collection methods are suggested for this evaluation exercise:

• Document review: Approved and revised project proposal; project reports including financial information, field reports, and training reports and/or other relevant documents.
• Key informant interviews: Project staff at the national and provincial level, key government stakeholders, key partner organizations or beneficiaries (remote or in-person).
• On-site direct observation: Field visits to project sites in Bosnia and Herzegovina and Montenegro, including temporary reception centres, outreach, AVRR and DTM activities, as well as hosts communities supported through rehabilitation and small-scale initiative. 
• Focus Group Discussion: group discussions can be organized to meet with beneficiaries.

Following receipt of the project documents, the evaluator(s) will develop a detailed methodology, which will be presented in inception report. Field observations, interviews and focus groups will be conducted remotely and in person, upon selection of the most relevant visit locations, and in consultation with the Project Management Team (PMT). The logistical arrangements will be facilitated by the PMT, upon request by the evaluator(s), and with the oversight of the evaluation manager (EM). 

The evaluation process will be carried out in accordance with the IOM's personal data protection principles. IOM abides by the norms and standards of UNEG and expects all evaluation stakeholders to be familiar with the ethical conduct guidelines of UNEG and the consultant(s) with the UNEG code of conduct as well. The evaluator(s) should also read and seek to integrate IOM’s core values and competencies in the evaluation process.

The internal evaluation is expected to generate key deliverables:

1. Inception Report: Based on the standard IOM template for inception reports. Scoping document that develops a more detailed context analysis and scoping for the assignment based on the desk review. The report focuses on the methodology and workplan. It includes an Evaluation Matrix to demonstrate the evaluator’s understanding of the TOR and outline key performance indicators that will be used to assess achievements of results. It should include an indicative list of interviewees and data collection tools (e.g. questionnaires, interview guides). Moreover, the inception report should highlight clearly where primary data shared by the project is not sufficient and what tools are proposed to collect missing information. The inception report should be reviewed and endorsed by the EM prior to the start of the active phase of the evaluation.

• Draft Evaluation Report: Following the desk review, field visits and data collection and analysis phase, the evaluator(s) will draft a preliminary version of the report, key findings and recommendations, which will be submitted to the EM and the PMT for review and feedback. When possible, the evaluator will present its main findings and recommendations to IOM and partners during a meeting to optimize the review and feedback process/timeline.

• Final Evaluation Report: Based on the standard IOM template for final evaluation reports. The evaluator(s) will finalize the information, addressing IOM's comments and suggestions. The report will include an executive summary of no more than two pages, consisting of a brief description of the project, its context and current situation, the purpose of the evaluation, its methodology, and main findings, conclusions and recommendations. The final report will be produced in English. Notwithstanding the nature of the findings, which are expected to be balanced and well informed, the quality of the writing, the structure and logical flow of ideas, as well as the general presentation, will be subject to IOM’s review and need to be deemed satisfactory by the EM for the evaluation process to be finalized. 

• Evaluation Brief: Based on the standard IOM template for evaluation briefs. The evaluator will develop a two-page brief summarizing project information, methodology, findings and recommendations that will be shared with the EM for review. The brief will accompany the final report and be shared with the donor, partners and internally, within IOM. The brief should be be produced in English and of quality deemed ready for publication.

• Draft Management Response Matrix: Based on the standard IOM template for management response, the evaluator(s) will partially fill out the matrix with the recommendations, after which the PMT will be responsible to complete and ensure fulfilment as per IOM evaluation guidance.

The reports should be kept clear, concise and consistent. They must be of publishable quality and must be delivered in Word format, with all the tables and pictures shared separately, in their original formats.

The evaluator(s) are responsible for editing, proofreading, translating, designing and overall quality control of the content and language. As part of the reporting process, the evaluator(s) will submit a methodological note explaining how quality control has been addressed during the evaluation. All produced deliverables shall follow donor, IOM and partners visibility guidelines.

The evaluation report will be in English. All data collection tools will be developed in English and translated when needed.

The IOM Programme Support Unit (PSU) will act as the Evaluation Manager (EM), in coordination with relevant IOM project staff. The EM will coordinate and oversee the evaluation process and serve as primary point of contact for the evaluator(s). IOM will provide all logistical support for field research.

The evaluator(s) will:

The EM will:

• Oversee the general process of the evaluation.
• Act as the primary point of contact with the evaluator(s) in order to facilitate coordination and communication with relevant IOM staff, partners and beneficiaries.
• Review and provide rapid feedback and guidance to the evaluator(s) on process and deliverables, ensuring coordination with PMT when needed.
• The EM is an IOM staff with a strong Monitoring and Evaluation background, and without involvement in the implementation or management of project activities. The EM upholds the integrity of the evaluation process and work with the evaluator(s) and the Project Management team (PMT) to ensure the quality of the final evaluation report.

The PMT:

• Support the general process of the evaluation.
• Upon the request of the evaluator(s), facilitate the scheduling of group discussions, meetings with partners or provide interpretation. 
• Review and provide rapid feedback to the evaluator(s) on process and deliverables.

A quality assurance process shall be integrated into the evaluation, as follows:

• Final terms of reference (ToR) shall be agreed between the EM and evaluator(s).
• The inception report shall be reviewed by the EM and relevant IOM staff, revised by the  evaluator(s) and finalized only upon EM’s acceptance.
• The evaluator(s) will raise any emergent challenges or barriers with the EM, who will help find appropriate solutions to facilitate the evaluation process. Solutions to known or anticipated limitations should be openly discussed with the EM.
• It is the responsibility of the evaluator(s) to communicate their needs and expectations regarding IOM’s support and facilitation during the process (e.g., interpretation). 
• If IOM staff presence or proximity is deemed problematic during the visits or meetings, the evaluator(s) are responsible for communicating it clearly and rapidly to the PMT and the appropriate IOM focal points.
• The final report and brief shall be reviewed by the EM and relevant IOM staff, revised by the evaluator(s), and finalized only upon the EM’s acceptance.

This schedule is indicative of the distribution of working time and will be finalized after consulting the stakeholders regarding their availability. The evaluator(s) will regularly update the project manager on the progress of the evaluation report.

The estimated number of working days planned for the completion of the evaluation tasks by the evaluator(s) is estimated equivalent to a total of 30 working days, to take place between 1 September and 24 November 2023. The timetable proposed in the inception report will specify the distribution of the stages in the period indicated. The timetable should meet the following deadlines:

IOM is looking for an individual or team with experience designing and conducting evaluations in relevant thematic and geographic context. The evaluator(s) should demonstrate the following qualifications:

Application deadline: 01 August 2023.

Posting period: 14 July 2023 – 01 August 2023.

• : Please send your application package via email to bihvacancies@iom.int with clear title: APPLICATION – EVALUATOR DP.0970

This process open to both national and international applicants. Only shortlisted candidates will be contacted.

Documents to submit as a part of the application should include:

• clearly highlighting qualifications and recent experience.
• of the main evaluator and other team members if relevant.
• with titles, year of publication, information on commissioner.
• of at least 1 recent evaluation report. Max. 3 years.
• demonstrating excellent understanding of the assignment and outlining the proposed methodology, data collection plan, quality assurance plan, assessment of timeline, feedback for consideration by IOM when needed.
• financial proposal with detailed budget (all-inclusive, with consultancy fees, travel, accommodation, subsistence, field visits, translation, interpretation, etc.).

All interested applicants are requested to include in their submission detailed costs including:

The evaluator(s) will be required to use their own computers, printers, photocopier etc. It is expected that the evaluator(s) will conduct most interviews remotely, using online platforms (e.g. Skype, Zoom, Google Meet). If travel is required, all travel costs should be planned properly in the technical proposal and included in the financial proposal. Please note that the evaluator(s) are responsible for obtaining entry visa (if necessary) for all its personnel deployed in field research.

Payment is contingent on approval by EM and will be made in two installments:

The evaluator(s) may propose different payment schedules. This will be considered during the assessment of the proposal.

[1] References to Kosovo* shall be understood to be in the context of United Nations Security Council resolution 1244 (1999).

[2] References to Kosovo* shall be understood to be in the context of UN Security Council resolution 1244 (1999).

[3] oecd.org/dac/evaluation/daccriteriaforevaluatingdevelopmentassistance.htm