Monitoring & Evaluation Consultant Tender, Kyrgyzstan - 32394045

Procurement Summary

Country : Kyrgyzstan

Summary : Monitoring & Evaluation Consultant

Deadline : 10 May 2019

Other Information

Notice Type : Tender

TOT Ref.No.: 32394045

Document Ref. No. :

Competition : ICB

Financier : United States Agency for International Development (USAID)

Purchaser Ownership : -

Tender Value : Refer Document

Purchaser's Detail

Purchaser : ACDI/VOCA
Kyrgyzstan
Kyrgyzstan
Email :LMoses@acdivoca.org, NToktorbaeva@acdivoca.kg

Tender Details

Tenders are invited for Kyrgyzstan: Monitoring & Evaluation Consultant.

PROJECT BACKGROUND

ACDI/VOCA in partnership with Helvetas and AIRD have been implementing a four-year USAID funded project titled “Agro Horizon Project” in Kyrgyzstan. The project promoted economic growth by increasing employment in the agricultural sector, particularly for women and youth, while improving the availability of nutritious foods and increasing the market share of the targeted crops in both domestic and foreign markets. Agro Horizon Project is a key component of USAID Kyrgyz Republic-s overall economic development program designed to accelerate diversified and equitable economic growth in Kyrgyz Republic.

The project zone of influence (ZOI) comprise Osh, Batken, and Jalalabad, and Naryn oblasts. With a focus on these four regions the project engaged actors in other locations that have market links to production in the target regions or the potential to strengthen competitiveness of the entire value chain for domestic or regional markets.

To sustainably reduce poverty and promote agricultural economic growth, the project embarked on;

Increasing productivity of agricultural producers and linking them to markets (Task 1)

Increasing productivity and markets for agribusinesses (Task 2)

Improving the enabling environment for agriculture sector growth (Task 3)

Improving the nutritional status of women and children in the zone of influence (Task 4)

The project-s programming employed an approach that was market-driven, private sector led, and promoted inclusive value chains development.

OBJECTIVES

The purpose of the end line evaluation is to document progress made towards project goals, at the same time documenting the contributing factors of the changes in the zone of influence at the population and direct project beneficiaries- levels. The following key outcome and impact indicators will be used (1) Expenditures of rural households, (2) Household dietary diversity Score (3) Poverty rate and (4) the Women-s Empowerment in Agriculture Index (WEAI).

The evaluation will assess the relevance, efficiency, effectiveness, impact, and sustainability of project activities, using a mixed-methods approach. The process will identify lessons learned and challenges faced.

The findings relating to the impact indicators above will be used by USAID and implementing partners to inform future agriculture sector led economic development programming in Kyrgyzstan.

The WEAI measurement in the project ZOI is to capture changes in women-s engagement in agriculture in the following areas; production, resources; income; leadership; and time use. This is in addition to assessing women- empowerment relative to men within households.

The findings of the WEAI measurement will be used by USAID and implementing partners to inform future agriculture sector led economic development programming in Kyrgyzstan that taps into the contributions of both men and women.

SCOPE

Target respondents: Data will be collected at the population level and among direct project participants for comparative purposes for the WEAI, poverty rate, Household dietary diversity score and Expenditures of rural households.

The study will have one set of respondents for poverty rate, Household dietary diversity score and expenditures of rural households and a different set of respondents for the WEAI to avoid respondent fatigue in answering questions.

Implementation period under consideration: The end-line evaluation will consider the whole project intervention period 2015-2019.

Key stakeholders: Activity stakeholders include USAID, ACDI/VOCA, Helvetas, and communities in the zone of influence.

End-line evaluation period: The End-line is planned to be conducted and completed between 20th May - 5th July 2019.

Geographical location: The following zones of influence of the project are the focus of the end-line i.e. Osh, Naryn, Batken and Jalalabad Oblasts (see ZOI map appended). Population of four oblasts as of Jan 2019 according to the National Statistics Committee; Batken - 525, 100, Jalalabad - 1214, 400, Naryn - 287, 000 and Osh - 1341, 900

APPROACH AND METHODOLOGY

The survey will be participatory, drawing on involvement of key project stakeholders and organizations. Quantitative and qualitative methods of data collection and analysis will be adopted to meet the survey objectives. Quantitative methods used during the baseline survey will be adopted and adapted for reasonable comparison of the baseline findings with the End-line findings. Specifically, quantitative methods will be used to quantify the outcome indicators using indicator definitions provided in the project Monitoring and Evaluation Plan for: (1) Expenditures of rural households, (2) Household dietary diversity, (3) Poverty rate and (4) WEAI. The evaluation will assess the relevance, efficiency, effectiveness, impact, and sustainability of project activities, using a mixed-methods approach. This includes the addition of learning-focused questions to the indicator survey and the use of targeted focus group discussions and key informant interviews. Focus group selection strategy will also aim to ensure representative responses from different target groups (e.g. sex, youth). The contractor is expected to audio record data all qualitative data collection activities, transcribe the data into English, and code data via a qualitative analysis platform. Appendix II provides Illustrative learning question topics, which will be used to examine the extent of the projects- impact in thematic areas and demonstrate whether the project achieved it-s key objectives. Learning questions will be finalized in coordination the AHOP-s technical staff during the development of the inception report and endline tools. The process will identify lessons learned and challenges faced. Appendix IV provides a minimum numbers of FI-s and FGD-s expected for examining learning topics referenced in Appendix II.

The qualitative methodology will be tailored in a manner to provide further understanding of quantitative results. The inception report should be explicit on the methodology to be used to answer each of the finalized learning question.

The sample size for the indicators shall be representative of the ZOI, with a confidence level of 95%, a power of 0.8, and a margin of error of 5%, with non-response rates and design effects accounted for. An identified proportion of those sampled should be direct beneficiaries, to allow for comparison of results between direct participant and non- participant groups. Sample unit selection should be based on probability sampling. Quantitative results will further be sample weighted, standard errors and confidence intervals computed to portray the big picture situation in the ZOI.

The consultant will work closely with ACDI/VOCA to ensure all information needs are met. The consultant will select and provide training to enumerators. In addition, the consultant will be required to carry out a field test of the data collection instrument and undertake an analysis of the data before going to the field. This will not only ensure that the enumerators are familiar with the tools, but the questionnaire items are reliable and produce valid results. The consultant will be expected to develop a comprehensive data quality assurance plan including standardized survey protocol. Following data collection, the consultant will compile and analyse quantitative and qualitative data, synthesize survey findings and compute endline project indicators in a reviewed and finalized endline survey report.

Data Collection Tools

The consultant will work closely with ACDI/VOCA and partners to develop the survey instruments. The data collection package will include field manual, data collection instruments such as household questionnaire, FGD/KII guides etc. The use of smart phones and tablets is required.

Enumerator Training

Prior to undertaking the survey, the consultant will provide a training workshop for quantitative enumerators. The purpose of the training is to ensure that everyone involved understands the objectives of the survey, the survey methods, sampling strategy, use of survey tools, and roles and responsibilities in data collection. Participants in the workshop will review every individual survey question for appropriateness, clarity and value of the information provided. The workshop will discuss practical tips on questioning techniques, probing techniques, logical sequences and skip rules. As a part of the training workshop, participants will be given opportunities to interview households as part of a survey pre-test. Pre-testing the questionnaire will provide enumerators with an opportunity to gain practical experience in interviewing, introduce them to the field realities, identify any ambiguity in the questionnaire, and get a sense of the time required to collect household responses.

Data Treatment and Analysis Plan

The consultant must prepare a data treatment and analysis plan to address the following elements:

i. Software to be used for data entry, along with timeline for double data entry and validation to ensure no delays occur in analysis;

ii. Statistical software to be used to calculate secondary variables from primary variables, where appropriate;

iii. Data quality checks and edits (data cleaning) planned to ensure logical consistency and coherence, as well as an indication of the software to be used.

TIME FRAME

It is anticipated that the exercise will take place from 20th May - 5th July 2019. This window covers all activities from field work to production of final report.

ROLES AND RESPONSIBILITIES

The project team will provide the contractor with relevant documents including but not limited to quarterly reports, annual reports, project design narrative, project M&E plan and data, midterm assessment report and baseline report.

ACDI/VOCA M&E team in Washington D.C., project based in-country M&E Director and Project chief of party will interact with the End-line evaluation team throughout the assignment for quality assurance. This includes feedback on sampling methodology, development of tools, training materials of enumerators and coordination in piloting the tools to ensure their efficacy.

The evaluation contractor will be responsible for arranging logistics to facilitate the process. This will include stationary, accommodation, transport and meals for the evaluation team. Hiring and training of data collectors will additionally be done by the evaluation contractor and these should be hired from within project ZOI. The contractor will ensure data collected is reliable and of sufficiently high quality to yield meaningful evaluation conclusions and lessons learned.

PROFESSIONAL QUALIFICATIONS

The contractor needs survey expertise in the following areas; sample size calculations, stratification, clustering, sample selection using multiple stages and unequal probabilities of selection, and sample weight creation. Qualitative data collection experience is also important.

In addition, the contractor would preferably have experience in implementing household and/or beneficiary-based surveys involving the following USAID indicators; Expenditures of rural households, Household Dietary Diversity Score, poverty rate and WEAI.

Key personnel under the contract will be required to be available throughout the survey and these include;

Survey Team Leader will have the overall responsibility for providing guidance and leadership. The following competencies are required;

a. Should possess a Postgraduate degree from a recognized university in development studies, project management, project monitoring and evaluation, or other relevant field of study.

b. Prior experience leading large-scale, complex beneficiary-based or household surveys (preferably in resource-constrained environments).

c. Should possess knowledge of USAID outcome and impact indicators related to agricultural economic development agricultural economic development.

d. Experience collecting and analyzing qualitative data

Field operations manager with;

a. Undergraduate degree from a recognized institution

b. Experience supervising fieldwork for large-scale beneficiary-based or household surveys (preferably in resource-constrained environments)

c. Experience recruiting, training, and managing field supervisors and data collectors

d. Experience coordinating field logistics, schedules, and equipment

e. Experience managing data quality control in the field during survey implementation

f. Strong interpersonal skills, ability to solve problems when confronted with roadblocks during survey fieldwork

g. Prior professional experience in country or region preferred

h. Fluency in English, Russian, Kyrgyz or Uzbek desirable

OUTPUTS EXPECTED OF CONTRACTOR

All reports shall be written in high-quality, professional-standard English. The final evaluation report should be clear and simply written, free of jargon. The main body of the report should not exceed 70 pages and should include an executive summary and recommendations. The consultant should support her/his analysis of project achievements with relevant data and state how this has been sourced. Visuals and graphical representation of data should be included in a robust manner, incorporating standard errors. Recommendations/lessons learned should also include details as to how they might be implemented in future projects.

ACDI/VOCA and the project implementing partners own the report. ACDI/VOCA reserves the right to share and discuss the report findings with any and all stakeholders, including those who are benefiting from the project, and maintains the decision of how to incorporate any resulting changes into the report.

The following deliverables are expected:

Inception report/workplan, including detailed survey implementation plan (DSIP) and results from the pre-test: i) Specifying details for methodology, critical tasks and mitigation measures, anticipated outputs, date-bound timelines, resource needs, and responsible person(s); ii) Composition of a standard field survey team, including expected tasks and responsibilities of each team member, should also be described; iii) Detailing a sampling plan for the quantitative population-based household survey and the qualitative sampling methodology, ethical considerations and gender/age approach, appropriate/sensitivity in data collection; and iv) Survey design and analysis plan (including gender/age lens in the analysis) and pre-test results, and detailed implementation plan reviewed and approved by ACDI/VOCA. This will also include the endline survey plan or a derivation of it.

Quality Assurance Plan: This should include whether the consultant will use computer-assisted personal interviews (CAPI) to reduce and minimizer errors during data collection, regular supervision of data collection, daily verification and validation of data collected before being uploaded into the server, data cleaning and processing as well as training of data collection team. The consultant should include a plan for pilot testing of data collection instruments before the survey kicks off.

Quantitative and qualitative survey instruments/tools: i) Draft and final quantitative survey questionnaires; ii) Draft and final qualitative data collection guides; iii) Draft and final supervisor and enumerator training agenda; iv) Draft and final supervisor and enumerator training materials; and v) Draft and final supervisor and enumerator field guides. All key documents shall be reviewed and approved by ACDI/VOCA.

Data treatment and analysis plan: Data treatment and analysis plan (include gender and age approach for the data analysis) reviewed and approved by ACDI/VOCA.

Raw and cleaned data set, data dictionary/codebook, edit rules, and syntax for data analysis, including syntax for variable transformations Deliverables: i) Raw data set in SPSS and/or CSV formats; ii) Rules applied for cleaning data; iii) data dictionary/codebook; iv) Syntax for all data analysis and variable transformations; v) Final data set that includes cleaned data, sampling weights at each stage, final sampling weights, and all derived indicators; and vi) Sampling weights used to tabulate the results vii) fully transcribed data from recordings of interviews and focus group discussions, along with the sounds files for the recordings. The program files should clearly identify section or module names used in the questionnaire and should follow the same order as the survey questionnaire so the AHOP staff could generate the same results. Include also a list of GPS waypoints collected for each household or value chain actor visited and surveyed.

Draft endline study report i). Clear and concise report not to exceed 70 pages, excluding appendices and annexes, presented in English. ii). Estimates and confidence intervals for all indicators. The endline study report will include at a minimum the following elements, to be reviewed with the consulting team

§ Acknowledgements

§ List of Acronyms and abbreviations

§ Table of Contents

§ List of tables, figures, charts, maps

§ Executive Summary

§ Background/Brief project description, context and rationale

§ Purpose, objectives and expected use of the study

§ Study methodology, survey sampling and data collection techniques

§ Survey team's management and operations

§ Data analysis, management and presentation

§ GIS data and reporting

§ Results and Discussion

o Main findings (to include values of all PMP indicators listed in the objectives section with their estimates and confidence intervals)

o Key observations (to include cultural issues, beneficiary attitudes, etc.)

o Data analysis methods, processes and limitations

o Sampling methodologies employed

§ Lessons Learned

§ Conclusions

§ Recommendation

Annexes:

§ Terms of Reference for the endline study

§ Timetable

§ List of documents, references, and data sets used and aligned data dictionaries

§ Survey instruments: questionnaire, interview guide (s), etc.

§ Field work documentation

§ Description of sampling procedures

§ Data analysis procedures

§ Standard Indicator Table with endline values, targets, and life of project values

Electronic copies of all raw and processed data and GPS waypoints are required to be included with the final report.

Final endline study report: This report will be a revised version of the draft report that incorporates the comments of ACDI/VOCA and USAID. The final report must be presented in English. Final endline study report reviewed and approved by ACDI/VOCA.

A 2-3 page stand-alone brief describing the evaluation design, key findings and other relevant considerations. It will serve to inform any interested stakeholders of the endline and should be written in language easy to understand by non-evaluators and with appropriate graphics and tables.

Briefing to ACDI/VOCA. Multimedia presentation to AHOP Management Team of findings. Presenting findings, conclusions, lessons learned, and recommendations.

EXPECTED ROLES AND RESPONSIBILITIES

ACDI/VOCA responsibilities:

Provide to the consultant in a timely manner all necessary documents in order to adequately conduct survey (Proposal, PMP)

Inform local authorities about the administration of the endline

Provide timely feedback to the consultant on questionnaire, focus group guides, and key informant questions

Plan the data collection phase in coordination with the consultant

Actively participate in the process of the study in all areas of design tools, and data collection

Select AHOP beneficiary participants and market system actors from which sampling will be based

The consultant will have the overall responsibility for the administration of the endline and for all the deliverables specified in the deliverables section, and the bullet points below:**

Review all documentation

Write an inception report; The inception report will provide a refined endline study methodology and implementation plan while respecting ACDI/VOCA-s guidelines (detailed methodology, sampling, data collection logistic, timeline & budget, data quality control/supervision). The study plan document must be approved by ACDI/VOCA before training for team members and the data collection take place

Develop a detailed Implementation Plan, including endline logistics in collaboration with the AHOP team. In addition, a specific endline survey plan is needed to guide the planning and conduct of the actual surveys and data collection.

Develop data collection instruments. The consultant will develop questionnaires/guides in the appropriate format which makes it convenient for enumerators, data entry staff, and others to work with. Ideally the consultant will conduct the survey using tablets or smart phone, and face-to-face interviews. The Consultant should strive to ensure that all other indicators use definitions and questions taken from standardized sources and are developed in coordination with AHOP staff.

Develop, adapt and translate the data collection instruments. The consultant should plan on using tablets or smart phones for data collection.

Develop training manuals/field guides

Train the team members and pilot test the questionnaires and data analysis plan

Supervise data collection fieldwork.

Supervise data entry process (ensure it data is validated).

Conduct (or supervise the conducting) of key informant interview and focus groups

Design data entry templates and database mask that meet the USDA FFP standards. To minimize data entry errors, the consultant should help design a database with appropriate data masks. This could be done either in MS Access, CS Pro, SPSS, or any other appropriate data entry package. The database must allow for data to be entered and to analyze disaggregated groups defined in PMP (region, age, sex, ethnic group)

Clean and analyze collected data

Record and transcribe word-for-word all interviews and focus group discussions for appropriate qualitative data analysis.

Submit a complete standard report with outlines and fully written text in compliance with the detailed Implementation Plan

Give a presentation debriefing AHOP management team

PROPOSAL EVALUATION FACTORS

Proposals will be evaluated using the following criteria

A. Technical Approach/Strategy/Methodology: Maximum Points: 25

B. Key Staff Qualifications: Maximum Points: 25

C. Past Performance/References: Maximum Points: 20

D. Capabilities: Maximum Points: 15

E. Cost Reasonableness Maximum Points: 10

F. Experience in Kyrgyzstan: Maximum Points: 5

How to apply:
PROPOSAL INSTRUCTIONS

The contractor shall prepare the following items for proposal application, in English and submit electronically to the email address below.

Technical Proposal

The technical proposal (Maximum 14 pages) should explicitly spell out how the work will be conducted, what data will be collected, how the data will be collected and compiled, how it will be verified, data quality controls, how the proposed staffing (and how many-key personnel, enumerators, experts, etc.) will be identified, organised and managed.

Indicate progress milestone towards completion of fieldwork, analysis and final report submission

The technical proposal shall also include curriculum vitae of key personnel, each with at least 3 client references. References should have worked with you within the past two years. Two verifiable examples of writing by the primary report writer should be provided.

ACDI/VOCA reserves the right to request any additional technical information.

Interested firms shall submit technical and budget proposals in English no later than 10th May 2019 through email LMoses@acdivoca.org and addressed to Lauren Moses, with copy to NToktorbaeva@acdivoca.kg Kindly note that proposals received after the deadline will not be considered.

Documents

 Tender Notice