International Consultancy on Formative Evaluation of the School Readiness Programme Tender

Procurement Summary

Country : Azerbaijan

Summary : International Consultancy on Formative Evaluation of the School Readiness Programme

Deadline : 01 Sep 2019

Other Information

Notice Type : Tender

TOT Ref.No.: 35573195

Document Ref. No. :

Competition : ICB

Financier : United Nations Secretariat

Purchaser Ownership : -

Tender Value : Refer Document

Purchaser's Detail

Purchaser : UNITED NATION CHILDREN`S FUND
24, Neftchilar avenue, Dalga Plaza Baku AZ1095 Tel: +994 12 4923013 (multiline)
Azerbaijan
Email :baku@unicef.org
URL :www.unicef.org

Tender Details

Tenders are invited for Azerbaijan: International Consultancy on Formative Evaluation of the School Readiness Programme.

The purpose of this evaluation is to measure the success of the School Readiness Program against specific objectives SRP and summarize good practices and lessons learned for both UNICEF, the donor and the government counterparts for consideration. The overall purpose of the evaluation is to determine the relevance, effectiveness, efficiency, impact and sustainability of the SRP as an approach to strengthening the access to formal preschool education.

Although it has been 10 years that the program is under implementation it has never been evaluated. Considering this UNICEF had a mutual agreement with the MoE to conduct a formative evaluation of this programme.

The results of the evaluation will provide practical recommendations for improving the quality of SRP and stimulating expansion of other or introduction of new alternative preschool models to cover more preschool-aged children.

The primary audience of the evaluation will be:

Government policy makers, such as the Parliament, the Cabinet of Ministers and the Ministry of Finance will acquire good practices and lessons learned from this programme, and will possibly use the results to contribute to other policy and programme development.
Implementing institutions - the Government, Ministry of Education, local education authorities, and international organizations, as an important source of information for the further scaling-up of this initiative - more specifically to identify lessons learned during the implementation, understand the preliminary impact of the programme in providing equitable access to quality preschool education for all children.
Schools and preschool education institutions and local communities in expanding access to preschool education for all children.
Associations of parents representing the interests of children to further strengthen their monitoring and advocacy efforts.
Civil society organisations for community outreach to parents and families for increasing demand for early childhood education.
Academia for supporting research for expanding alternative preschool education models.
The UNICEF Azerbaijan CO for future programming and support to the implementation of the National Education Strategy, as well as for informing the development of the 2021-2025 Country Programme.
The information presented in the final evaluation report should be easy to understand and comprehensible. Recommendations should be based on evidence and analysis, be relevant and realistic, with priorities for action made clear.

Evaluation Objectives

The overall purpose of the formative evaluation is to measure the extent to which the SRP has achieved the planned results and help ensure that the focus is kept on equity, inclusion and quality in the process of national scale-up of the programme.

The specific evaluation objectives are:

Provide feedback to the UNICEF Azerbaijan Country Office and the Ministry of Education on the soundness (defined as relevance, effectiveness, efficiency and sustainability) and preliminary impact, i.e. longer-term contribution, of the SRP;
Reveal good practices and gaps in approaches; and
Based on the experience from the Programme implementation to extract general lessons learned and recommendations aimed at further enhancement of the initiative; the evaluation outcomes will potentially inform strengthening of other existing or emerging models of preschool education to address inequity in access to preschool education in Azerbaijan;
Assess the potential for replication of the model in other countries by capturing the changes at policy and programmatic levels.
Evaluation Scope

The evaluation should cover the period of SRP implementation (2008 to 2018). The geographical coverage is nation-wide.

Evaluation Framework

The program evaluation questions are formulated as per OECD-DAC evaluation criteria: relevance, efficiency, effectiveness, sustainability and impact. Additional criteria such as coverage, coordination and coherence should also be used in the evaluation.

Below are a set of guiding questions that should be responded by the evaluation. However, it is expected that the international consultant may suggest additional questions or sub-questions, and during the evaluation, additional information that adds substance to the key questions will be collected and included in the final evaluation report.

Relevance

The extent to which the program meets the needs of target groups and right holders.

To what extent does the programme, in terms of its design, scope and achievement of its objectives, address the underlying causes of the low coverage of children (girls and boys) in Azerbaijan by preschool education?
Has the programme been designed according to international norms and agreements on human rights (HR) and gender equality (GE) and in line with national strategies to advance HR & GE?
To what extent is the program design relevant vis-à-vis the overall program goal and the achievement of its objectives in the given period of time?
To what extent did the program minimize the exclusion of the most underserved children?
Effectiveness

The extent to which program activities achieve their outcomes. In evaluating the effectiveness of a program, the following questions should be considered:

To what extent have the planned results/objectives been achieved to date (quantitative and qualitative)?
What were the major factors influencing the achievement or non-achievement of the programme objectives to date? Has the programme provided any additional (not directly planned) significant contributions/outcomes?
Is there a demonstrable improvement in state knowledge and capacity related to coverage of the preschool-aged children?
To what extent the approach was effective in identifying and addressing the different needs/rights issues of boys and girls?
Have legislative acts and policies improved as a result of program implementation?
How satisfied were the right holders from the program?
Efficiency

The extent the management of the SRP ensure timelines and efficient utilization of resources.

When evaluating the efficiency of a programme, the following questions should be considered:

How well have the financial resources been used/were funds managed in a cost-effective manner / what is the correlation between funds utilized and outputs/results achieved / could the same results be achieved with fewer resources?
Did the program ensure co-ordination with other similar interventions to encourage synergy and avoid overlaps?
To what extent were activities implemented as scheduled, how flexible was the program in adapting to changing needs?
Were there other resources made available apart from UNICEF?
Impact

The extend the program increased the system-s capacities to ensure that excluded children benefit from SRP. When evaluating the impact of a programme, the following questions should be considered:

Is there evidence of the programme contributing to the raising of the awareness of the relevant government and parliamentary institutions in relation to the importance of ECE expansion for child development and overall societal growth?
Is there evidence of the programme contributing to an increase in the enrolment of children in preschool education in the country, particularly in relation to children affected by poverty and children living in rural areas?
To what extent have achievements fed into national-level policy dialogue and supported the environment for implementation of child rights?
Sustainability

Sustainability is concerned with measuring whether the benefits of the program are likely to continue after donor funding has been withdrawn.

When evaluating the sustainability of a programme, it is useful to consider the following questions:

To what extent do the newly developed and approved policy (National Education Strategy, Preschool Education Law, and Preschool Education Curriculum) and the level of ownership of the initiative within the Ministry of Education and preschool institutions provide a good ground for a sustained and quality increase in the preschool enrolment rate after the end of external support?
What were the major factors which influenced the achievement or non-achievement of sustainability of the program?
What capacities have been built at the institutional level and were they sustainable?
To what extent has the program promoted the strengthening of already existing partnerships and establishment of new ones and to the strengthening of inter-sectoral and cross-sectoral cooperation both at the national and local level?
In addition to the 5 main evaluation criteria, the evaluation shall also focus on assessing the human rights-based approach and relevant cross-cutting issues, coverage, coordination and coherence.

Cross-cutting:

To what extent are sex and age disaggregated data collected and monitored?
In what ways and to what extent has the program integrated an equity-based approach into the design and implementation of its interventions?
Does the program actively contribute to the promotion of child rights, especially the most vulnerable?
To what extent and how does the program ensure a non-discrimination and equity focus?
Coverage:

Did the program ensured the inclusive coverage?
Which groups have been reached by the program and what is the different impact on those groups?
Have vulnerable children been reached, including children with disabilities?
Coordination:

What was the role of the other government and non-government entities community and other key actors in the design, coordination and implementation of program activities?
Coherence:

What were the areas and ways of cooperation with other UN and donor agencies- in regard to the development of SRP?
How does the program relate to the existing national and/or local policy on preschool-aged children?
Was there coherence across interventions supported by different agencies?
Methodology

The international consultant will be requested to propose a detailed methodology as part of the inception report, which should be guided by the UNICEF-s revised Evaluation Policy[1], the Evaluation Norms and Standards of the United Nations Evaluation Group (UNEG)[2], UNICEF Procedure for Ethical Standards in Research, Evaluations and Data Collection and Analysis[3] and UNICEF-s reporting standards.

It is expected that in the end, the program will reach six inter-related and coherent outcomes, which should be the main building blocks for achieving the goal of the program. Achievement of those six outcomes will be measured with a set of indicators to be provided to international consultant after the contractual arrangements.

[1] UNICEF revised evaluation policy https://undocs.org/E/ICEF/2018/14

[2] UNEG Norms: http://www.uneval.org/document/detail/21, UNEG Standards: http://www.uneval.org/document/detail/22

[3] https://www.unicef.org/supply/files/ATTACHMENT_IV-UNICEF_Procedure_for_E...

The evaluation should include the following steps:

Step 1: Desk review of relevant program documents

The international consultant will review key program documents to understand the program approaches, process and activities since its inception in 2008 to date. The documents could include the relevant national policies, other study reports, program, documents, progress and monitoring reports; review meeting documentation, and international consultants- consultancy and mission reports.

Step 2: Preparation of Inception Report that includes evaluation methodology and tools

The methodology should be prepared to cover all the intended objectives of the evaluation. The evaluation methodology design will be finalized in agreement with the reference group (with UNICEF, MoE) and inception report should be prepared based on the Evaluation Norms and Standards of the United Nations Evaluation Group and submitted to reference group.

Step 3: Data collection

The application of both qualitative and quantitative data collection methods is expected, which should be human rights-based, including child rights-based and gender-sensitive. The data collected should be disaggregated by sex, ethnicity, age, disability, and location. Field visits to schools and communities should employ methods ranging from document review, interviews, focus group discussions, surveys, observation depending on the final methodology.

Step 4: Data analysis

Collected data should be analysed by using the relevant analysis method that should be clearly described in the report.

Step 5: Sharing preliminary findings and recommendations

The international consultant will share preliminary findings and recommendations with the reference group. While feedback will be taken into consideration and incorporated into the draft report, the international consultant is encouraged to guard against validity threats, such as personal bias.

Step 6: Draft report

The international consultant prepares a draft report, with conclusions and recommendations drawn from the data. The report structure should follow UNICEF-s evaluation report guidance.

Step 7: Finalization of the evaluation report

The international consultant will present the final draft evaluation report to the reference group with a powerpoint presentation. Recommendations for the program evaluation report should also be presented. Comments and feedback on the findings and recommendations should be incorporated to finalize the report.

Workplan and Evaluation Management

Description of Deliverables

Time estimate

Assessment criteria

Desk-review & Inception report

15 days

All relevant program documents are reviewed and inception report that includes the result of desk review, consultation meetings and detailed evaluation methodology that is compliant with UNICEF requirements for inception report. The inception report will be assessed based on the Global Evaluation Report Oversight System (GEROS) review criteria.

Among other inception reports should include the following components:

1. Evaluation plan including timelines and activities

2. Methodology

3. Data collection instruments (quantitative & qualitative)

4. Ethical protocols (if relevant)

5. Quality control procedures

6. Training plan

7. Fieldwork plan including team composition, logistics, field monitoring, etc.

8. Plans for data analysis (quantitative and qualitative), report preparation and dissemination

Data collection

15 days (including travels)

Primary data is collected from target groups and partners based on the methodology described in the inception report.

Data analysis and first draft report

20 days

Relevant analysis methods applied to analyse primary and secondary data and a draft report is prepared.

Reconstract the Theory of Change based on the collected data.

Presentation of findings

5 days (including travels)

PowerPoint presentation of findings including practical recommendations is presented to reference group and program partners feedbacks recorded to be considered in the final report.

Final report

10 days

A final report should be between 30-50 pages and structured as per the UNICEF-Adapted UNEG Evaluation Reports Standards[1]

1. Executive summary

2. Object of evaluation

3. Evaluation purpose, objectives and scope

4. Evaluation methodology

5. Findings

6. Conclusions and lessons learned

7. Recommendations

8. Gender and human rights including child rights issues to be consolidated and clearly articulated from all report sections.

9. Annexes

More detailed information of the UNICEF-Adapted UNEG Evaluation Reports standard is provided in the UNICEF Global Evaluation Report Oversight System (GEROS) Review Template, which will be shared at the start of the consultancy. The evaluation report should be adjusted as per feedback of the external quality assurance entity.

[1] UNICEF-Adapted UNEG Evaluation Reports Standards, July 2010

Dissemination

The results of the evaluation will not be circulated to wider-public. The report will be disseminated to a reference group including the MoE in hard and soft copies. Also, the results of the evaluation findings will be disseminated through PowerPoint presentation.

The financial proposal should include a breakdown of the budget amount including fee and number of anticipated working days, travel costs and per-diems. Payments will be processed per the payment conditions aligned with deliverables specified in the ToR, upon satisfactory completion of work assignment as assessed by UNICEF. In-country travel to the SRP implementation sites and interpretation services for the international consultant will be covered by UNICEF.

Qualifications

The key qualifications required for an international consultant are as follows:

Advanced University degree in education, social science, law or relevant area;
8-10 years of professional experience in evaluation and assessment of programs or projects;
Previous experience in evaluating programs or projects for child education and child welfare;
Demonstrated capacity to analyse the data and to write reports, in particular, evaluation reports;
Facilitation skills, particularly the design of stakeholder consultation exercises as well as participatory methods;
General knowledge of UN evaluation policy, norms and standards, including a human rights-based approach to programming and results-based management, including gender equality and child;
Experience in working with UN / UNICEF and government entities.
Excellent mastery of English including in report writing and presentation.
The international consultant must remain in strict adherence with UNEG ethical guidelines and code of conduct. Per the Criteria, for Ethical Review Checklist the evaluation does not need to go through an ethical review board. An international consultant should clearly identify any potential ethical issues and approaches, as well as the processes for ethical review and oversight of the evaluation process in his/her proposal.

Administrative issues

The consultant will work remotely, with travel to Azerbaijan (estimated 25 days in Azerbaijan). Internal travel may be required for data collection and consultations with various stakeholders and will be supported by UNICEF Azerbaijan CO.

During the periods in which the Consultant is working remotely, conference/skype calls will be held as required.

The consultant will make his/her own arrangements for the travel. Travel costs for two trips in this consultancy should be estimated and included in the proposal (lump sum and break down by budget lines) along with the requested daily fee. Travel costs shall be calculated based on economy class travel, regardless of the length of travel; costs for accommodation, meals and incidentals shall not exceed applicable daily subsistence allowance (DSA) rates, as promulgated by the International Civil Service Commission (ICSC).

Meanwhile, UNICEF will provide logistical support for the in-country trips, such as the provision of office space, vehicle for site visits and official meetings, organisation and coordination of meetings, interpretation and translation and support with obtaining visa and registration, once in-country.

Application

Interested individual should submit through e-recruitment: https://www.unicef.org/about/employ

Resume
Technical evaluation proposal with a detailed budget;
cover letter including the following items:
Evidence of matching with the required qualification
References to the previous relevant work
Incomplete applications or missing any of the required documents will not be considered.

UNICEF is committed to diversity and inclusion within its workforce, and encourages all candidates, irrespective of gender, nationality, religious and ethnic backgrounds, including persons living with disabilities, to apply to become a part of the organization.

How to apply:
UNICEF is committed to diversity and inclusion within its workforce, and encourages qualified female and male candidates from all national, religious and ethnic backgrounds, including persons living with disabilities, to apply to become a part of our organization. To apply, click on the following link http://www.unicef.org/about/employ/?job=525672

Documents

 Tender Notice