I. Background to the GEC Programme and the Project
· The Department for International Development (DFID) leads the UK’s work to end extreme poverty. DFID is tackling the global challenges of our time including poverty and disease, mass migration, insecurity and conflict. DFID’s work is building a safer, healthier, more prosperous world for people in developing countries and in the UK too.
· DFID is working to reach the Sustainable Development Goals (SDGs) by 2030. Progress on girls’ education is critical to the achievement of these targets. SDGs 4 and 5 specifically relate to education and achieving gender parity. SDG 4 specifically notes ‘inclusive and quality education for all and promote lifelong learning’.
· Globally 31 million primary age girls, have never been to school[1]. And the majority of these girls come from the poorest and most marginalised communities in the most disadvantaged locations, ethnic groups etc.[2] Over the last 20 years primary enrolments for girls have improved along with boys but completion rates are equally low for both sexes. At the secondary level the differences between boys and girl’s participation rates really start to show. Significant disparities exist within countries, with the poorest girls from rural areas most severely subject to educational disadvantage - even at the primary level[3].
· The Girls' Education Challenge (GEC) is helping the world’s poorest girls improve their lives through education and supporting better ways of getting girls in school and ensuring they receive a quality of education to transform their future.
· PricewaterhouseCoopers LLP (PwC) and alliance partners have been contracted as the dedicated Fund Manager (FM) and is responsible for the day-to-day operation of the GEC. This includes establishing the recipient tendering process, supporting bidders, sifting and scoring proposals, monitoring Value for Money (VfM) and making project funding recommendations for DFID approval. The FM also manages the relationships with the selected projects and oversees their monitoring, evaluation, and learning operations.
· Through the GEC, DFID provided £355m from 2012-17 to the FM to disburse to 37 individual projects across 18 countries across sub-Saharan Africa and South Asia to help girl’s education.
II. “Biruh Tesfa for All (BTA) – Ethiopia”
Most rural-urban migrants in Ethiopia are adolescent girls, migrating during early adolescence and frequently escaping early forced marriage or poverty. They arrive in low-income urban areas, armed with little in the way of education or preparation for city life. Most enter employment through domestic work, a form of modern slavery. These girls share characteristics with girls with disabilities, as both groups are isolated, confined to the home and out of the public eye. Both categories of girls have negligible prospects for an education or safe and positive livelihoods. With appropriate support, such girls can be mainstreamed into formal education or transition into safe employment.
Objectives of Biruh Tesfa for All
BTA is a partnership between Population Council, Humanity and Inclusion, Plan International and the Ministry of Education. The main objective of the project is to support out-of-school adolescent girls (either drop-outs or those who never attended), aged 10 to 19, to transition into formal schooling or employment and to gain skills to improve the quality of their lives. Specific objectives are:
Biruh Tesfa for All Beneficiaries
The primary beneficiaries of Biruh Tesfa for All are girls aged 10-19 who are out-of-school (OOS) and residing in low income urban areas of three Ethiopian cities: Addis Ababa, Bahir Dar, and Shashamene. Out-of-school girls who satisfy these criteria are largely rural-urban migrants, child domestic workers (CDW) and girls with disabilities (GwD). Through the Biruh Tesfa for All project, we aim to reach 10,500 girls in these three cities. Secondary beneficiaries will be OOS boys with disabilities, community members, teachers and schools.
Outcomes
The main outcomes to be achieved in this 3 ½ year project are: improved literacy skills, improved numeracy skills and successful transitions to formal schooling and/or safer and more productive livelihoods.
Biruh Tesfa for All Implementation Design
Together with downstream partners Humanity and Inclusion and Plan International, we will work with local authorities to select locations in the three project cities that are very low income and receive a large number of rural-urban migrants. Once determined, we will recruit local female leaders from those communities to be trained as mentors. In all, 320 mentors will be recruited across the three cities. Mentors will be trained on the Biruh Tesfa for All life skills curriculum and on the Alternative Basic Education (ABE) curriculum used in Ethiopia.
Following training, BTA mentors will be given geographic areas of assignment in their localities. They will go house-to-house to enumerate girls and boys living in households and identify eligible girls to invite for BTA participation. Girls who are OOS and aged 10-19 will be invited to join the project. At the time of the household visit, mentors will also negotiate for participation of the girl with her employers and guardians. Multiple visits will be undertaken if needed to convince employers/guardians to allow girls to participate. At this time, participating girls with disabilities (GwD) will be assessed to determine the assistive aids they need to attend the BTA programme.
BTA girls are mobilized into age- and education-segmented girls’ groups. Groups meet in existing community facilities which have been upgraded to be inclusive and conducive to learning. BTA groups meet five times per week for about two hours per session. Once in the groups, they receive ABE for four days in a week, and life skills on the 5th day. Upon enrolment in BTA, beneficiaries devise a guidance and counselling plan with their mentor to plan and track progress toward achieving their transition goals, to formal education or safe livelihoods.
In order to improve social norms and attitudes about marginalised girls – including CDW and GwD – BTA will undertake community conversations (CCs) in the project sites among a cross-section of community members. The CCs will include discussions of marginalised girls, their status in communities and the importance of education, development and protection. Through CCs, we will reach over 29,000 community members. In addition, BTA partners will map community-based child protection systems and orient mentors and community leaders on how to provide additional support for girls in need. Finally, 25 primary schools will be made accessible and inclusive through BTA, facilitating girls’ transition to formal schooling.
Timeline: This is a 3 ½ year project, from July 2018 to December 2021.
Additional information on Biruh Tesfa, including past evaluations, can be found on the Council’s website: https://www.popcouncil.org/research/biruh-tesfa-safe-spaces-for-out-of-school-girls-in-urban-slum-areas-of-ethi
III. Evaluation of Biruh Tesfa for All
Rationale for the evaluation
The findings from the evaluation will primarily be used:
Evaluation objective
The project is seeking to procure the services of an independent external evaluator to conduct a mixed-method, gender-sensitive evaluation that is inclusive of persons with disabilities of the Biruh Tesfa for All project over the next 3 ½ years. The evaluation will assess the delivery, effectiveness, value for money (VfM) and impact of the project and report the findings and lessons learnt throughout the process.
Evaluation questions
1. The Evaluation Team will be required to develop an evaluation approach that answers the following overarching questions as a minimum:
Specific project and programme level evaluation questions are
These questions help define the scope and focus of the project evaluation process. The successful bidder will be expected to work with the Project Management Team to review and revise these questions as appropriate at the outset of the project. Project specific context is important in this respect.
Overall evaluation approach
2. The overall evaluation approach requires the Evaluation Team to design, plan and conduct a mixed-methods evaluation that is longitudinal in nature. More details on evaluation approach can be found in section 14 of the FM’s evaluation guidance.
3. A proportionate amount of time and resources should be allocated to the evaluation given the type of project interventions, operational context and the reporting requirements of the GEC.
Research design
4. Comparison groups: bidders are required to outline their approach to evaluating the impact of the project. This should include consideration of the most rigorous approach to establishing a counterfactual. This should enable comparison of the outcomes achieved by a target group who were affected by a project intervention with the outcomes achieved by a group who are similar to the target group, except that they have not been exposed to the intervention i.e. a comparison group. Careful consideration should be given to ethical issues in the use of comparison groups and the approach may be reviewed during contracting phase.
5. Cohort tracking: the project is required to track a learning cohort and a transition cohort – defined as individuals who progress through the project together. Bidders should outline their approach to tracking cohorts in control and intervention areas and in three project towns. Mechanisms to minimize loss-to-follow-up should be suggested, given the high mobility of urban girls. See section 16.4 of the MEL guidance for more information on cohort tracking.
6. Measuring outcomes: bidders are expected to understand the project’s key and intermediate outcomes and suggest the most appropriate data collection approach to evaluate each outcome. This should include a mixture of quantitative and qualitative approaches. Refer to the evaluation sections and appendix A of the grant recipient handbook and section 14 of the MEL Guidance. The Evaluator will be expected to pilot tools that will be used for data collection and refine as necessary.
7. Project sampling framework: The Evaluation Team will be required to help finalise the sampling frameworks for both qualitative and quantitative samples. These should be of a sufficient size and representativeness to allow reasonable:
Refer to the evaluation guidance for further information on sampling and Section 15 of the MEL Guidance.
8. Baseline Study: The Evaluation Team will be required to design and implement a gender-sensitive mixed method baseline study as an integrated part of the overall MEL strategy and plan for the project. This may include pre-baseline data collection to identify the target group and barriers to education. The baseline study should identify the number of beneficiaries with disabilities as well as the type and severity of their disability, following the UN Washington Group methodology[4]. Bidders should set out their approach to the baseline study. See section 16-18 of the MEL Guidance.
Ethical protocols
9. The successful evaluation team will draft a comprehensive protocol and undergo full ethical review as required by the Population Council and in Ethiopia. The evaluation approach must consider the safety of participants and especially children at all stages of the evaluation. The evaluation team will need to demonstrate how they have considered the protection of children through the different evaluation stages, including recruitment and training of research staff, data collection and data analysis and report writing.
10. Research ethics plan: bidders are required to set out their approach to ensuring complete compliance with international good practice, with regards to research ethics and protocols. Particularly with regards to safeguarding children, vulnerable groups (including people with disabilities) and those in fragile and conflict affected states. Consideration should be given to:
Risk and risk management
11. Risk management plan: It is important that the successful bidder has taken all reasonable measures to mitigate any potential risk to the delivery of the required outputs for this evaluation. Therefore, bidders should submit a comprehensive risk management plan covering:
Data quality assurance
12. Quality assurance plan: bidders are required to submit a quality assurance plan that sets out the systems and processes for quality assuring the evaluation and research process and deliverables from start to finish of the project. This plan should include the proposed approaches to:
Existing Information Sources
13. In the first instance, bidders should refer to the DFID GEC website: https://www.gov.uk/guidance/girls-education-challenge for general information concerning the Girls’ Education Challenge.
14. Bidders should refer to the following GEC programme and project documentation: Grant Recipient Handbook, Evaluation guidance, Logframe and workplan guidance, Project Logframe, and Theory of Change.
15. Bidders should also refer to relevant country data and information that is currently available, as required, to prepare the proposal.
Professional Skills and Qualifications
16. Qualifications: bidders are required to identify and provide CVs for all those proposed in the Evaluation Team, stating their roles and responsibilities for this evaluation. Please note that if the enumeration is to be sub-contracted, the evaluator will be ultimately responsible for the enumerators they are subcontracting to. The proposed evaluation team should include the technical expertise and practical experience required to deliver the scope of work and evaluation outputs, in particular, with regards to:
17. Day–to–day project management of the evaluation will be the responsibility of Population Council Country Director, Dr. Annabel Erulkar. The Population Council Project Director, the MEL Officer and Research Officer will participate in providing feedback and direction to the successful bidder.
Deliverables and Schedule
18. Project deliverables: the main deliverables for this project are as follows
Final project evaluation report: design, conduct and submit a final project evaluation report that assesses the effectiveness, impact and VfM of the project.
19. Report requirements: all reports should be submitted in electronic form and should be in English.
20. Detailed work plan: bidders are required to provide a detailed work plan incorporating all relevant tasks and milestones from start to finish of the evaluation study.
21. Project milestones: bidders are required to include in their detailed work plans the milestones set out below. Timelines are illustrative and to be finalized during the inception meeting.
Table 1: Project milestones
The following milestones are illustrative and to be finalized at inception meeting. Timing for subsequent rounds of surveys will be detailed following the completion of the baseline survey.
Typical project milestones /outputs for deliverables | Deadlines |
August 24, 2019 | |
Deadline for receipt of tenders | September 10, 2019 |
Evaluation of tenders and shortlisting completed | September 14, 2019 |
Interviews of shortlisted suppliers held | September 16, 2019 |
Supplier appointed | September 18, 2019 |
1. Inception Phase | Deadlines |
Inception Meeting held | September 20, 2019 |
Review of project documents (proposal, theory of change, child protection policies, risk registers, etc.) | September 24, 2019 |
Research design completed including: sampling framework, data collection strategy, cohort tracking strategy, informed consent protocols, child protection framework, research instruments | October 4, 2019 |
Presentation of Research design and timeline to Evaluation Steering Group | October 7, 2019 |
Final Inception Report and research protocol submitted, based on feedback from Evaluation team | October 14, 2019 |
Research protocols submitted to local and PC ethical review boards | October 16, 2019 |
2. Baseline Study Phase | Deadlines |
Tool piloting and translation Baseline research starts | Bidder to complete Subject to ethical approval |
Baseline research completed | Bidder to complete |
Draft Baseline Study Report submitted for review | Bidder to complete |
Review by Project Management and stakeholders completed /comments provided to Supplier | Bidder to complete |
Final Baseline Study Report submitted, based on comments received | December 30, 2019 |
3. Subsequent Evaluation Phases (to be detailed following baseline) | Deadlines |
3.1 Midterm evaluation | Anticipated May, 2020 |
3.2 Endline evaluation | Anticipated June-Sep, 2021 |
Final Project Evaluation Report received | October 30, 2021 |
Reporting and Contracting Arrangements
22. The Evaluation Team is expected to identify a Project Director and Project Manager for communication and reporting. At the Inception meeting the Evaluation Team Project Manager will submit a full contact list of all those involved in the evaluation.
23. The Evaluation Team will be expected to report to the Evaluation Steering Group and attend all meetings as agreed with the Project Evaluation Manager. The Team will be required to submit to the Project Evaluation Manager bi-weekly progress reports (by email) during the study periods summarising activities /tasks completed to date, time spent etc.
Budget
24. The estimated budget for this evaluation work is £170,000. This budget should cover the data collection, analysis and reporting for three evaluation points. This budget is inclusive of all costs for team member salaries, travel, research costs and any other costs associated the completion of the work including where required costs for reasonable adjustment. Bidders are required to organise and fund their own duty of care arrangements as required.
25. Bidders are requested to provide a fully costed proposal in the form of a price schedule that as a minimum should include:
Bidders may suggest a payment schedule on the basis of milestone payments for the successful delivery of each deliverable.