Result of Service

Outputs

The Consultant will submit a Launch Note which represents the first deliverable, approximately 10 working days after start. This Launch Note contains the evaluation information and initial planning of the review with timeline.

The final deliverable will be a report, no more than 20 pages based on a standard template, due at the end of the contractual period.

The final report should include:

o Executive summary of findings and conclusions;
o Introduction with background to the review;
o Review questions, methodologies and approach, scope, constraints, and limitations;
o Data collection and analysis based on criteria and findings;
o Key findings and conclusions for all areas of questions;
o Annexes, including: TORs, work plan, instruments and protocols, accessed data and charts, references, documentation, etc.

Work Location

UNHQ New York

Expected duration

4 Months

Duties and Responsibilities

Background

The General Assembly, in its resolution 77/255 of 30 December 2022, welcomed the development of the computer-assisted translation and machine translation system (eLUNa), and requested the Secretary-General “to report on updates, including cost-benefit analysis and quality preservation and control, to this system”.

In response to the General Assembly’s request, DGACM intends to undertake an evaluation to help ascertain the contributions of eLUNa and related tools in the areas of effectiveness, efficiency and relevance, including productivity, quality preservation and control, as well as cost-efficiency. The evaluation will be conducted by an independent evaluator, external to DGACM, with the necessary expertise in CAT (Computer Assisted Translation) tools. The evaluation results will be reported to Member States in the next Secretary-General’s report on the pattern of conferences.

Objective

The overall objective is to undertake an evaluation that will cover identification of recycled text, terminology detection and machine translation and their impact on the processing of parliamentary documentation in DGACM, in the six official languages of the United Nations.

Methodology

The evaluation will rely on qualitative and quantitative data, including an initial desk research, followed by data collection and analysis and interviews to expand upon the results highlighted by the desk research and data analysis and explore key issues in more depth.

Server data will be collected and analyzed to quantify the information provided by eLUNa during the translation process. This will include the level of recycled text, automatic term recognition, edit distance for recycled text and machine translation, and the like.

Interviews will be conducted with the following key stakeholders: Programme Executive, Programme Manager, Deputy Programme Managers, Chief Data Officer, Chief of GTT and Technical Team Lead.

The scope of the evaluation will include:

1. To provide an overview of the overall working methods before and after the implementation of eLUNa.
2. To assess the impact of identification of recycled text, terminology detection and machine translation introduced by the eLUNa suite of tools in translation workflows and working methods.
3. To review the benefits anticipated and realized from the implementation of eLUNa.
4. To determine the potential impact of innovation in the future of language professions.
In undertaking the above, the evaluation will attempt to answer the following questions:

1. How does eLUNa contribute to the efficiency, effectiveness and relevance of the Department ?
2. How does eLUNa contribute to cost-efficiency, quality preservation, control and productivity of individual staff?
3. What are the opportunities to further increase cost-efficiency through the use of eLUNa and the collection of additional server data?
4. How is the role of language professionals expected to be impacted by further advances in technology?
5. What measures could DGACM take to remain relevant in the new technology landscape?

The evaluation will adhere to the principle of confidentiality and be gender inclusive.

Qualifications/special skills

Applicants must hold at least a first-level degree or an equivalent qualification in modern languages, translation, interpretation or another relevant subject, such as law, economics, accounting, international relations, science or technology.
A minimum of 15 years of directly relevant professional experience in language operations, project management, and/or monitoring and evaluation, in the area of multilingual documentation for conference services, is required.
Experience in managing translation teams in an international organization is required.
Experience in managing projects for the development and/or implementation of Computer-Assisted Translation or machine translation tools is required.
Experience in leading change management and/or digital transformation processes is required.

Languages

Fluency in English is required. Fluency in another UN language is desired.

No Fee

THE UNITED NATIONS DOES NOT CHARGE A FEE AT ANY STAGE OF THE RECRUITMENT PROCESS (APPLICATION, INTERVIEW MEETING, PROCESSING, OR TRAINING). THE UNITED NATIONS DOES NOT CONCERN ITSELF WITH INFORMATION ON APPLICANTS’ BANK ACCOUNTS.