Terms of Reference

Project Title: Support to Public Finance Management and Institutional Development (SPFMID) Post Title: Short term National Consultant for Final Project Evaluation
Implementing partner: Ministry of Finance
Contract duration: 20 working Days

I. BACKGROUND

In 2016, the Government of Egypt (GoE) has declared the Sustainable Development Strategy Egypt 2030 (SDS) as the base for setting the development objective for all Egyptians. Egypt is also committed to the achievement of the SDGs and has already submitted its first voluntary report to the High-level Political Forum (HLPF).

Two of the key cornerstones in mainstreaming Egypt’s 2030 goals in the public sector practices under the economic dimension are:

a) the administrative reform programme and b) the economic reform programme, which was agreed upon with IMF.

As such, there is a growing need for the Ministry of Finance (MOF) to focus on its

a) Human resources: their capabilities, the work culture and the effectiveness of their productivity within the scope of the MOF’s institutional development framework; as well as,

b) Public financial management core functions. Consequently, the overall objective of this program is to improve the effectiveness of MOF’s performance by instigating a strong inclusiveness organizational culture and a result-focus capacity development of the staff while spearheading the institutionalization of strategic management and supporting the implementation of Egypt’s roadmap for the enhancement of public finance management.

SPFMID Project is based on the groundwork of preceding assistance projects received mainly from IMF, WB, UNDP and British Embassy and will focus on the following four Components:

1. MOF Institutional development
2. MOF PFM reform initiatives
3. The effectiveness of MOF assistance programs
4. Provision of professional assistance to ministerial policy, technical and executive support teams

MOF needs to tackle several challenges such as;

a) critical skills shortages while striving to ensure the competency, efficiency and effectivity of MOF workforce, b) rigidity of the public expenditures structure, high budget deficit and gross domestic debt, c) guarding the emerging employee inclusiveness culture to improve internal coherence, transparency and accountability, and d) reassessment of the government assistance programs in the agricultural commodity sector.1

1

Project Indicative Output(s):
– Institutional development strengthened

– At least 2 key PFM reform initiatives launched

– The effectiveness of MOF assistance programs enhanced – Expert assistance provided to ministerial support teams

2

II. OBJECTIVE

As an integral part of the project implementation cycle, UNDP will commission an independent final project evaluation. The final evaluation will analyze the achievements of the project against its original objectives to provide project partners with an independent review of project outputs and an outlook on opportunities for future cooperation. The evaluation will review technical and managerial aspects and consider issues of effectiveness, efficiency, relevance, impact and sustainability.

The overall objectives of the evaluation are the following:

• Provide an objective assessment of the progress towards the achievement of the project objectives and outcomes as specified in the Project Document and the impact produced so far under the area of intervention, especially under the challenges presented due to the COVID19 pandemic.
• Identify factors that have facilitated and/or impeded the achievement of objectives to generate lessons from experiences in the project respective interventions to inform current and future programming
• Collect and analyze all relevant data to produce quantitative measure relative to project output targets and contribution of the project activities to UNDP Egypt Country Project document (CPD).
• Provide clear and actionable recommendations to suggest effective and realistic future phase of cooperation for UNDP taking into account the challenges and opportunities ahead.
• Upon consultation with the partners, provide an outline for a new project.
  The evaluation time framecovers thedurationof thecurrent project startingfrom2018 – 2022.

3

III.

EVALUATION CRITERIA

The project should be evaluated against the following OECD-DAC evaluation criteria:

• Relevance: the degree to which the project was, and remained relevant in the context in which it is being implemented.
• Efficiency/management: assessing the outputs realized in relation to the inputs provided (project management structure…)
• Effectiveness: the extent to which the project objectives were achieved. Using evaluative evidence, the evaluation will analyze the contributing factors, unintended outcomes (positive or negative, direct or indirect)
• Impact and sustainability: the evaluation should assess the lasting change brought about by the project.
• Gender and social inclusion sensitivity: to what degree was the project sensitive to gender and social inclusion.

  The evaluation must address how the project sought to strengthen the application of the rights-based approach and mainstream gender. (Integrating Human Rights and Gender Equality in Evaluations)

  EVALUATION QUESTIONS
  Evaluation Questions should include at least the following questions:

4

IV.

Relevance:

• −  To what extent was the project in line with the national development priorities, SDS, the UNDP country development programme for Egypt and the SDGs?
• −  To what extent does the project contribute to gender equality, the empowerment of women and the protection and/or promotion of human rights?
• −  To what extent has the project been appropriately responsive to political, legal, economic, institutional, etc., changes in the country?

  Effectiveness:

• −  Were stated outputs achieved? If not, what progress toward the outputs has been made?
• −  What factors have contributed to achieving (or not achieving) intended results?
• −  Were the actions to achieve the outputs and outcomes effective and efficient?
• −  What factors contributed to effectiveness or ineffectiveness?
• −  What unintended change (positive/negative) has the project had on the targeted beneficiaries?
• −  To what extent MOF operational and institutional structures (project management unit and systems) is strengthened to effectively implement activities? To what extent has UNDP support achieved its target and objectives according to the project plan and stated objectives?
• −  Has the project partnership strategy been appropriate and effective? To what extent has the

project been able to build and promote its partnership with other relevant stakeholders for greater results?

• −  What can be done for strengthening the impact/achievements of the project? What are the issues and challenges to be addressed?
• −  To what extent the project has been able to adopt or respond to the situation of COVID-19 pandemic effectively?

  Efficiency:

• −  Did the project make the best use of its resources to achieve its results? Has the project been efficient in implementing its activities?
• −  To what extent have project funds and activities been delivered in a timely manner? Sustainability:

V.

• −  What good practices are emerging from the project? What are the likelihood for good practices to be replicated or scaled up for sustainability purposes?
• −  To what extent the current organizational set up is sustainable (financially and operationally)? What are the existing structures and functions that could ensure sustainability of project outcomes in the targeted areas?
• −  How can the project be improved for its next phase?
• −  How to maximize the cooperation in a future set up (format, design…)?

  Evaluation questions must be agreed upon between UNDP and MOF and accepted or refined in consultation with the evaluator(s)

  METHODOLOGY

  Applicants should not have participated in the project preparation and/or implementation and do not have any conflict of interest with project related activities.

  The evaluator is expected to use suitable methods to obtain data and information for the analysis and draw up of findings, conclusions, lessons learned and recommendations, in consultation with UNDP and MOF.

  Due to COVID-19 travel to the country has been restricted since March 2020 and travel in the country is also restricted. If it is not possible to travel to or within the country for the evaluation then the evaluator should develop a methodology that takes this into account the conduct of the evaluation virtually and remotely, including the use of remote interview methods and extended desk reviews, data analysis, surveys and evaluation questionnaires. This should be detailed in the Inception report and agreed upon.

  If all or part of the evaluation is to be carried out virtually then consideration should be taken for stakeholder availability, ability or willingness to be interviewed remotely. In addition, their accessibility to the internet/ computer may be an issue as many government and national counterparts may be working from home. These limitations must be reflected in the evaluation report.

5

VI.

1.

If a data collection/field mission is not possible then remote interviews may be undertaken through telephone or online (skype, zoom etc.). The consultant can work remotely with support in the field if it is safe for them to operate and travel. No stakeholders, consultants or UNDP staff should be put in harm’s way and safety is the key priority.

Given the strategic relevance of this project, the evaluation is expected to contribute to apply an analytical approach not only to assessing the extent to which objectives have been reached but also identifying the reasons and determining factors behind the observed successes and shortcomings. It is important to take into account the importance of context specific factors affecting project implementation.

The evaluator is expected to suggest a methodology that demonstrates analytical rigor and data accuracy and validation, to support findings and recommendations. The suggested methodology for the evaluation could include:

Documentation review: To review documents such as the project document, project brief, progress reports, Annual Project Reports (APR), board meeting minutes , and minutes from relevant meetings (list of key documents is annexed)

Interviews with MOF Senior Management and staff. The Consultant will obtain initial briefings from UNDP and MOF project on the objectives, scope of evaluation, and clarify any issues as required prior to submitting the inception report. Consult relevant stakeholders/partners of the project.

Collecting relevant information through meetings, consultative sessions, field visits, etc. Obtain other contextual information also as required.

Data review and analysis: The mixed methods of data gathering will be matched with appropriate analytical approaches including qualitative and quantitative analysis and triangulation to ensure a comprehensive and coherent understanding of the data generated by the evaluation

Drafting the evaluation report and making a presentation of findings and recommendations to UNDP and MOF.

Finalizing the report with comments and inputs from various stakeholders and partners. DELIVERABLES

Inception Brief – The methodology that will be used by the evaluator should be presented in the inception brief and it must be agreed to by UNDP and MOF . The brief will include an outline of the main evaluation issues that will be addressed, relevant evaluation questions and the proposed and final methodology that has been agreed upon before the evaluation is set. Evaluation matrix is suggested as a deliverable to be included in the inception report. The evaluation matrix is a tool that evaluators create as map and reference in planning and conducting an evaluation. It also serves as a useful tool for summarizing and visually presenting the evaluation design and methodology for discussions with stakeholders. It details evaluation questions that the evaluation will answer, data sources, data collection, analysis tools or methods appropriate for each data source, and the standard or measure by which

6

7 each question will be evaluated. (the evaluator will need to follow the UNDP standard template

of the inception report) Time frame: 4 days

Table 1. Sample evaluation matrix

VII.

2.

3. 4.

• •

• • • • • •

Draft evaluation report and Power Point Presentation on findings and recommendations: These will be shared with both UNDP and MOF for review A meeting will be held to discuss the draft report to ensure that it meets UNDP quality criteria and expectations as stipulated in the Terms of Reference and the UNDP evaluation guidelines. The comments will be provided in audit trail to the evaluator with the needed changes. The evaluator needs to show how these comments are addressed.

Time frame: 8 days

Final evaluation report (See Evaluation Report Template in Annex 3): The report will be in English and no more than 30 pages in length. Annexes may be added.
The Final report will include the evaluation methodology, a brief executive summary, lessons learned from the project and actionable recommendations for the next phase.

Time frame: 8 days

QUALIFICATIONS

M.A. Degree in economics, development or related discipline.
Demonstrated experience in leading evaluation studies in the above-mentioned field(s); Good command of standard evaluation methods and approach
Expertise in independent policy advice
Excellent English, both written and spoken
Good analytical and writing skills
Good communication skill
Knowledge and understanding of UNDP activities is an asset
Experience in designing institutional development and capacity development projects is an asset

EVALUATION ETHICS

VIII.

This evaluation will be conducted in accordance with the principles outlined in the UNEG ‘Ethical Guidelines for Evaluation’. The consultant must safeguard the rights and confidentiality of information providers, interviewees and stakeholders through measures to ensure compliance with legal and other relevant codes governing collection of data and reporting on data. The consultant must also ensure security of collected information before and after the evaluation and protocols to ensure anonymity and confidentiality of sources

of information where that is expected. The information knowledge and data gathered in the evaluation process must also be solely used for the evaluation and not for other uses with the express authorization of UNDP and partners.

IX. IMPLEMENTATION ARRANGEMENTS

The responsibility for managing this evaluation lies jointly between UNDP and MOF: The UNDP Egypt Country Office jointly with MOF are the operational points for the evaluation and will be responsible for liaising with the stakeholder interviews, arrange the meetings and co-ordinate with other counterparts. The report will be cleared by UNDP and MOF.

Resources and Logistical Support Required: It is expected that the Project Director will introduce the evaluator to stakeholders in order to facilitate the evaluator’s mission. During the evaluation period, the Consultant will require office accommodation that will be provided by MOF as necessary.

Time-frame for the evaluation process: The evaluation will take place over a period of 20 days. The tentative schedule follows:

a. Preparation Phase (4 working days)

UNDP and MOF will provide the expert with a maximum number of relevant documents – such as project documents, quarter progress reports and annual reports. The expert will study the documents as a preparation for evaluation of the project. MOF will give briefing on the general background of the project such as: how the project was developed and how it was implemented.

b. Assessment Phase (12 working days)

The consultant will work at MOF’s premises and will conduct interviews with stakeholders, counterparts and beneficiaries in order to learn about achievements and challenges and assess the impact of activities. The expert will draft a final evaluation report.

c. Reporting Phase (4 working days)

The Consultant will provide an initial draft report to UNDP and MOF one week after finalizing assessment phase. Comments will be provided within 5 working days.

8

IX. EVALUATION OF APPLICANTS

Individual consultants will be evaluated based on the following methodology:

Technical Proposal (70%):

Relevant experience 20

Sample of previous relevant assignment 20 Total 100

Financial Proposal (30%)

Weight

9

To becomputedasa ratio of theProposal’soffer tothe lowestpriceamong the proposals received by MOF.

Applicants receiving a score less than 70% will be technically, disqualified.

X. TERMS OF PAYMENT

• 30% upon submission of Inception Report
• 30% upon submission of Draft Evaluation Report
• 40% upon submission of Final Report, Executive Summary, and Presentation

XI. APPLICATION:

Interested applicants must submit the following:

1. CV and Personal History Form (P11)
2. ProposedMethodologyoutlininghowtheConsultantwillexecutetheassignment

3. Written sample of prior evaluation and/or assessment work
4. Detailed financial proposal in EGP

Applications should be submitted through an email titled “MOF Final Project Evaluation” to the following e-mail address:
dina.gamal@mof.gov.eg
Miral.magdy@mof.gov.eg

reem.elsawy@undp.org

Deadline for applications 8th of August, 2022

10

ANNEX I

All UNDP Programme and project evaluations are to be conducted in accordance with the principles outlined in the UNEG ‘Ethical Guidelines for Evaluation’ and the UNEG Code of Conduct for Evaluation in the UN System. Both documents can be found at the following link:

http://www.uneval.org/document/guidance-documents

Evaluations of UNDP-supported activities need to be independent, impartial and rigorous. Each evaluation should clearly contribute to learning and accountability. Hence evaluators must have personal and professional integrity and be guided by propriety in the conduct of their business

Evaluators:

• Must present information that is complete and fair in its assessment of strengths and weaknesses

  so that decisions or actions taken are well founded.

• Must disclose the full set of evaluation findings along with information on their limitations and have

  this accessible to all affected by the evaluation with expressed legal rights to receive results.

• Should protect the anonymity and confidentiality of individual informants. They should provide maximum notice, minimize demands on time, and: respect people’s right not to engage. Evaluators must respect people’s right to provide information in confidence, and must ensure that sensitive information cannot be traced to its source. Evaluators are not expected to evaluate individuals, and

  must balance an evaluation of management functions with this general principle.

• Evaluations sometimes uncover evidence of wrongdoing. Such cases must be reported discreetly to the appropriate investigative body. Evaluators should consult with other relevant oversight entities

  when there is any doubt about if and how issues should be reported.

• Should be sensitive to beliefs, manners and customs and act with integrity and honesty in their

  relations with all stakeholders. In line with the UN Universal Declaration of Human Rights, evaluators must be sensitive to and address issues of discrimination and gender equality. They should avoid offending the dignity and self-respect of those persons with whom they come in contact in the course of the evaluation. Knowing that evaluation might negatively affect the interests of some stakeholders, evaluators should conduct the evaluation and communicate its purpose and results in a way that clearly respects the stakeholders’ dignity and self -worth.

• Are responsible for their performance and their product(s). They are responsible for the clear, accurate and fair written and/or oral presentation of study limitations, findings and recommendations.
• Should reflect sound accounting procedures and be prudent in using the resources of the evaluation.

  All staff engaged full or part time in evaluation must sign an Evaluation Staff Agreement Form at the start of their contract (see Annex 3).

11

ANNEX 2

Documents to be consulted— This is a list of important documents and Webpages that the evaluator should read at the outset of the evaluation and before finalizing the evaluation design and the inception report.

• Project Documents and any revisions
• United Nations Partnership Development Framework (UNPDF)
• UNDP Country Programme Document(CPD)
• Quarterly Progress Report and detailed activity progress reports
• Project Annual reports
• Minutes of Board meetings
• Presentations and other inputs to Board Meetings and project management meetings Previous

12

ANNEX 3
United Nations Evaluation Group – Code of Conduct for Evaluation in the UN System

Evaluation Consultants Agreement Form

To be signed by all consultants as individuals (not by or on behalf of a consultancy company) before a contract can be issued.

Agreement to abide by the Code of Conduct for Evaluation in the UN System

Name of Consultant:

Name of Consultancy Organization (where relevant):

IconfirmthatIhavereceivedandunderstoodandwillabidebytheUnitedNationsCodeof Conduct for Evaluation.

Signedat( ) on ( )

Signature:

13

ANNEX 4
EVALUATION REPORT TEMPLATE AND QUALITY STANDARDS

This evaluation report template is intended to serve as a guide for preparing meaningful, useful and credible evaluation reports that meet quality standards. It does not prescribe a definitive section-by- section format that all evaluation reports should follow. Rather, it suggests the content that should be included in a quality evaluation report. The descriptions that follow are derived from the UNEG ‘Standards for Evaluation in the UN System’ and ‘Ethical Standards for Evaluations’.

The evaluation report should be complete and logically organized. It should be written clearly and understandable to the intended audience. The report should also include the following:

Title and opening pages—Should provide the following basic information:

• Name of the evaluation intervention
• Time-frame of the evaluation and date of the report
• Countries of the evaluation intervention
• Names and organizations of evaluators
• Name of the organization commissioning the evaluation
• Acknowledgements

14

Tableofcontents—should alwaysincludeboxes,figures,tables andannexes withpagereferences. List of acronyms and abbreviations

Executive summary—A stand-alone section of two to three pages that should:

• Briefly describe the intervention of the evaluation (the project(s), programme(s), policies or other intervention) that was evaluated.
• Explain the purpose andobjectivesofthe evaluation,includingtheaudienceforthe evaluation and the intended uses.
• Describe key aspect of the evaluation approach and methods.
• Summarize principle findings, conclusions, and recommendations.

  Introduction—Should:

• Explain why the evaluation was conducted (the purpose), why the intervention is being evaluated at this

  point in time, and why it addressed the questions it did.

• Identify the primary audience or users of the evaluation, what they wanted to learn from the evaluation

  andwhy,andhowtheyareexpectedtousetheevaluationresults.

• Identifytheinterventionoftheevaluation(theproject(s)programme (s)policies,orother intervention—

  see upcoming section on intervention.)

• Acquaint the reader with the structure and contents of the report and how the information contained in

  the report will meet the purposes of the evaluation and satisfy the information needs of the report’s intended users.

  Descriptionofthe intervention—Providesthebasisforreportuserstounderstandthelogicandasses themeritsof theevaluationmethodologyandunderstandtheapplicabilityoftheevaluation results. The description needs to provide sufficient detail for the report user to derive meaning from the evaluation. The description may include:

• Describewhatisbeingevaluated,whoseekstobenefit,andtheproblemorissueitseeksto address.
• Explain the expected results map or results framework, implementation strategies, and the key

  assumptions underlying the strategy.

• Identify the phase in the implementation of the intervention and any significant changes (e.g., plans,

  strategies, logical frameworks) that have occurred over time, and explain the implications of those

  changes for the evaluation.

• Identifyanddescribethekeypartnersinvolvedintheimplementationandtheirroles.
• Describethescaleoftheintervention,suchasthenumberofcomponents(e.g.,phasesofa project) and

  the size of the target population for each component.

• Indicate the total resources, including human resources and budgets.
• Point out design weaknesses (e.g., intervention logic) or other implementation constraints (e.g.,

resource limitations).

15

Evaluation scope and objectives—The report should provide a clear explanation of the evaluation’s scope, primary objectives and main questions.

• Evaluation scope—The report should define the parameters of the evaluation, for example, the time period, the segments of the target population included, the geographic area included,andwhich components,outputsoroutcomeswereandwerenotassessed.
• Evaluation objectives—The report should spell out the types of decisions evaluation users will make, the issues they will need to consider in making those decisions, and what the evaluation will need to achieve to contribute to those decisions.
• Evaluation criteria—The report should define the evaluation criteria or performance standards used.Thereportshouldexplaintherationaleforselectingtheparticularcriteria used in theevaluation.
• Evaluation questions—Evaluation questions define the information that the evaluation will generate. The report should detail the main evaluation questions addressed by the evaluationandexplain

  howtheanswerstothesequestionsaddresstheinformationneedsof users.

  Evaluation approach and methods—The evaluation report should describe in detail theselected methodological approaches, methods and analysis; the rationale for their selection; and how, within theconstraintsoftimeand money,theapproachesandmethodsemployedyieldeddatathathelped answer the evaluation questions and achieved the evaluation purposes. The description should help the report users judge the merits of the methods used in the evaluation and the credibility of the findings, conclusions and recommendations. The description on methodology should include discussion of each of the following:

• Data sources—The sources of information (documents reviewed and stakeholders), the rationale for their selection and how the information obtained addressed the evaluation questions.
• Sample and sampling frame—If a sample was used: the sample size and characteristics; the sample selection criteria (e.g., single women, under 45); the process for selecting the sample (e.g., random, purposive); if applicable, how comparison and treatment groups were assigned; and the extent to which the sample is representative of the entire target population, including discussion of the limitations of sample for generalizing results.
• Data collection procedures and instruments—Methods or procedures used to collect data, including discussion of data collection instruments (e.g., interview protocols), their appropriatenessforthe datasource,andevidenceoftheirreliabilityandvalidity.
• Performance standards—The standard or measure that will be used to evaluate performance relative to the evaluation questions (e.g., national or regional indicators, rating scales).
• Stakeholder participation—Stakeholders’ participation in the evaluation and how the level of involvement contributed to the credibility of the evaluation and the results.
• Ethicalconsiderations—Themeasurestakentoprotecttherightsandconfidentialityof informants (see UNEG ‘Ethical Guidelines for Evaluators’ for more information).

16

• Background information on evaluators—The composition of the evaluation team, the background and skills of team members, and the appropriateness of the technical skill mix, gender balance and geographical representation for the evaluation.
• Major limitations of the methodology—Major limitations of the methodology should be identifiedand openly discussed as to their implications for evaluation, as well as steps taken to mitigate those limitations.

  Data analysis—the report should describe the procedures used to analyses the data collected to answer the evaluationquestions.Itshoulddetailthevariousstepsandstagesofanalysisthatwere carriedout,includingthe stepstoconfirmtheaccuracyofdataandtheresults.Thereportalsoshould discuss the appropriateness of the analyses to the evaluation questions. Potential weaknesses in the data analysis and gaps or limitations of the data should be discussed, including their possible influence on the way findings may be interpreted and conclusions drawn.

  Findings and conclusions—The report should present the evaluation findings based on the analysis and conclusions drawn from the findings.

• Findings—Shouldbepresentedasstatementsoffactthatarebasedonanalysisofthedata. Theyshould be structured around the evaluation questions so that report users can readily make the connection between what was asked and what was found. Variances between plannedandactualresultsshould beexplained,aswellasfactorsaffectingtheachievement of intended results. Assumptions or risks in the project or programme design that subsequently affected implementation should be discussed.
• Conclusions—Should be comprehensive and balanced, and highlight the strengths, weaknesses and outcomes of the intervention. They should be well substantiated by the evidence and logically connected to evaluation findings. They should respond to key evaluation questions and provide insights into the identification of and/or solutions to important problems or issues pertinent to the decision-making of intended users.

17

Recommendations—The report should provide practical, feasible recommendations directed to the intended users of the report about what actions to take or decisions to make. The recommendations should be specificallysupportedbytheevidenceandlinkedtothefindingsandconclusionsaround keyquestions addressed by the evaluation. They should address sustainability of the initiative and comment on the adequacyoftheprojectexitstrategy,ifapplicable.Recommendationsshouldalso provide specific advice for future or similar projects or programming.

Lessons learnt—As appropriate, the report should include discussion of lessons learned from the evaluation, that is, new knowledge gained from the particular circumstance (intervention, context outcomes,evenaboutevaluationmethods)thatareapplicabletoasimilarcontext.Lessonsshould be concise and based on specific evidence presented in the report.

Report annexes—Suggested annexes include the following to provide the report user with supplemental background and methodological details that enhance the credibility of the report:

• TOR for the evaluation
• Additional methodology-related documentation, such as the evaluation matrix and data

  collection instruments (questionnaires, interview guides, observation protocols, etc.) as

  appropriate

• List of individuals or groups interviewed or consulted and sites visited
• List of supporting documents reviewed
• Project or programme results map or results framework
• Summary tables of findings, such as tables displaying progress towards outputs, targets, and goals

  relative to established indicators

• Short biographies of the evaluators and justification of team composition
• Code of conduct signed by evaluators

18