Call for Applicants: Consultant for External Final Evaluation

Monday 29 October 2018

Terms of Reference for Evaluation 

"Quality Education for All, without Discrimination"

 

Project Summary

Type of evaluation

External Final Evaluation

Name of the project

Quality Education for All, without Discrimination

Project Start and End dates

Phase II: 01.01.2016 – 31.12.2018 – subject to Evaluation

This project is a continuation of Phase I:  01.01.2013 – 31.12.2015

Project duration

36 months

Project locations:

Kosovo: Prishtina, Peja/Pec, Prizren, Ferizaj/Urosevac, Gjakova/Djakovica, Gjilan/Gnjilane, Mitrovica, Mitrovica North and Gracanica.

Thematic areas

Education

Sub themes

Basic Education

Donor

IKEA Foundation

Estimated beneficiaries

15,865

Boys (0-18): 7,800;            Girls (0-18): 7,200;                   

Men (18+): 272;                 Women (18+): 593

 

Overall objective of the project

Fighting discrimination and violence and improve quality of pre-primary and primary education and society at large, and building a culture of inclusive and safe quality learning environments in schools in targeted locations in Kosovo

 

Introduction

This document provides Terms of Reference for the evaluation, project background, intended methodology and timeframes for its implementation, provided in the sections that follow, with regards to the project ‘Quality Education for All, without Discriminationfunded by IKEA Foundation andimplemented by Save the Children in Kosova/o. This project began in January 2016 with the main objective of Fighting discrimination and violence and improve quality of pre-primary and primary education and society at large, and building a culture of inclusive and safe quality learning environments in schools in targeted locations in Kosovo. It is the continuation of the IKEA Foundation funded project of the same name, the first phase of which began in January 2013 with the main objective of Promoting the right to education for the most deprived children in Kosovo. Phase II (herein after ‘the project’) was launched in 16 primary schools in 9 municipalities of Kosovo, namely Prishtina, Peja/Pec, Prizren, Ferizaj/Urosevac, Gjakova/Djakovica, Gjilan/Gnjilane, Mitrovica, Mitrovica North and Gracanica. Interventions included teacher training, and mobilization and awareness raising of communities on the right to education of all children and targeted an estimated 15,865 girls, boys, men and women, with a focus on the most deprived. The project has also worked intensively in advocating to increase governmental budgeting for Inclusive Education (IE), worked closely with non-formal education and service providers such as Handikos and its 8 service provision centres across Kosovo, and was an essential drive in establishing Child-Led-Municipal-Assemblies (CLMAs).

This evaluation is commissioned as one of the final actions of the project, and its objective is to measure the effectiveness of the intervention. The methodology used is to comply with each Key Evaluation Question (KEQ). Some fixed methodology variables include Quality Learning Environment (QLE - Inclusive) toolkit (provided by SCIK) for the target primary schools, Quality of Services (QoS) toolkit (provided by SCIK) for the target Non-Formal Education/service provision centres, as well as focus group discussions with i. parents, ii. project partners, and iii. children (including)CLMA. Remaining methodology features are open to proposal by consultant, as needed in order to answer all KEQ.

 

Project background

Kosovo continues to face numerous challenges, requiring strong political will and the need for structural economic reforms. The high unemployment rate has brought forth a very difficult economic and social situation. The economic situation affects in particular youth, women, and children[1]. Poverty rates are especially high among Roma, Ashkali and Egyptian (RAE) households. Significant causes are the low levels of education within families, while the communities do not support the education of their children. Poor economic conditions, early marriages, child labour and high unemployment among RAE communities also contribute to low school enrolment, particularly girls. Drop-out rates are higher among the RAE communities, due to lack of awareness on education.

Another socially vulnerable group is children with disabilities (CWD). The number of CWD in the school system ranges from 5,000 to 5,500 (6.3 - 7.3 % of the total estimated number of CWD in Kosovo). In the last years, CWD have been included in some public schools[2] even if these schools often lack the competences and suitable conditions to offer access to quality in education.

Moreover, over the past years, there has been a high increase of migration of citizens from Kosovo towards EU countries, with around 50,000[3] reported as illegally migrating, with as many as 5,000 children withdrawn from schools. This migration is notably higher among the RAE communities according to official data from the German government, between January and March 2015, 1,827 Roma from Kosovo submitted new asylum applications.[4]

The Government of Kosovo is committed to developing and implementing inclusive policies and to ensure an environment that promotes increased access to education for all, including most deprived children. Key policy documents include Strategy for Integration of RAE communities in Kosovo 2017-2021[5], Kosovo Education Strategic Plan 2017-2021[6] and the Action Plan for Inclusive Education 2016-2020[7]. However, relevant legislation and strategies are poorly implemented due to a lack of capacity among professionals and insufficient financial resources and support.

Save the Children has been working on inclusive education in Kosovo since 2005, focusing on marginalized ethnic minorities, RAE in particular and CWD. Having built a strong record in this area, it is recognized by the Government authorities at national and local level as a leading organization on inclusive education. Both Save the Children and its local partners have built trust and strong working relationship with communities, schools, municipalities, as well as with the Ministry of Education, Science and Technology (MEST) and Child Rights officers at municipal level.

Through the support provided from IKEA Foundation, Save the Children has implemented the first phase during the period 2013 – 2015, which addressed the exclusion of RAE children and CWD from education. The Realizing the rights of minority and disabled children in Kosovo project (2013-2015), as the first phase of the current project – subject to final evaluation, has been implemented in 18 preschool and primary educational institutions in 9 target municipalities in Kosovo (Prishtina, Peja/Pec, Prizren, Gjakova/Djakovica, Gjilan/Gnjilane, Mitrovica North, Mitrovica South, Gracanica, and Ferizaj). The direct beneficiaries of the project were: children with disabilities and children from the Roma, Ashkali and Egyptian (RAE) communities. Other beneficiaries of the project included teachers, school personnel, school children, municipal staff responsible for working with children, and local CSOs.

Through the interventions delivered as part of this project, SCI Kosovo in partnership with the Ministry of Education, Science and Technology, University of Prishtina, Municipal Education Directorates and implementing partners from civil society (local CSOs), seek to achieve the following outputs for phase II of the project:

The target outcomes of the project were following:

·         Outcome 1:By 2018, 300 children with disabilities (144 girls and 156 boys) and 400 children from RAE communities (192 girls and 208 boys) benefit from improved access to inclusive, safe and good-quality education at pre-primary and primary levels.

·         Outcome 2: By 2018, there will be a significant increase in engagement of targeted communities and active participation of children for the rights to education.

Data on project Key Performance Indicators have been collected by SCiK and will provided to the evaluation team.

 

Project Partners

The project was implemented in 9 target municipalities (Prishtina, Peja/Pec, Prizren, Gjakova/Djakovica, Gjilan/Gnjilane, Mitrovica North, Mitrovica South, Gracanica, and Ferizaj), of which 7 are Albanian majority, and 2 are Serb majority municipalities, with respective 14 Albanian majority primary schools and 2 Serb majority primary schools.

Aside the local and central level governmental institutions and schools mentioned above, SCIK in this project has worked closely with two partner organizations who were sub-granted for implementation. Partner CSO Handikos: (Association of Paraplegics and Child Paralysis of Kosovo) is a local NGO, a leading agency in Kosovo in promoting the rights of persons with disabilities, established in 1983. The Handikos network consists of 25 local offices in different municipalities of Kosovo where peer counselling and other activities are organized and 13 Community Based Rehabilitation centres (CBR) running providing primary physical rehabilitation and game development activities. Partner CSO Putevima Sunca was registered as a local NGO in Kosovo in May 2012 aiming to address and provide assistance in solving the problems faced by minority communities. The organization works with different programs on children’s rights, youth and child protection. The beneficiaries of program are particularly children, young women, youth but also others in need of support. Putevima Sunca is based in Gracanica and its plan and objective is to increase awareness on children’s rights, protection and inclusive education by targeting the children from marginalized communities and supporting them to increase their voice. RROGRAEK (The Network of Roma, Ashkali and Egyptian Women Organizations of Kosovo): is a non-governmental organization, established in 2007. SCiK has built close working relations with RROGRAEK in order to raise their awareness about the discriminatory practices that RAE children and their parents face when accessing educational institutions. Jointly, we work in the municipalities of Peja, Mitrovica, Gjilan, Ferizaj and Prishtina. Nevo Koncepti: is recognized by the communities where they operate and is a very important actor working in education, and the rights of RAE children in Prizren municipality. We jointly work in the municipalities of Prizren and Gjakova.

 

Gender Considerations

Children from RAE communities, especially girls, have poor levels of education and high dropout rates and their right to attend school granted by the Law on Pre-University Education is not being fulfilled. This project focused on continuation of good practice on inclusive education, ensuring gender based equality and gender-sensitive work where possible, as well as non-discrimination.

 

Scope of evaluation

The purpose of this final evaluation (conducted 3 years after phase II of interventions began) is to assess the effectiveness of the approaches piloted during the lifetime of the project, with special focus on establishing the effects of introduced inclusive education support teachers and community mobilization/ partnerships. This assessment aims to measure project results and inform of best practices of the approaches used, by exploring what are the major factors influencing the achievement or non-achievement of the second phase outcomes, by answering the key evaluation questions (KEQ).

This evaluation is commissioned as one of the final actions of the project, and is of formative and outcome- evaluation nature. The evaluation is formative in that it assesses the project design and offers findings to learn from in future project design tackling the same issue; and it is an outcome evaluation, measuring the immediate results or outcomes of the project. Thus, the main objective is to measure the effectiveness of the intervention, as well as the outcome and design of its three major intervention categories:

·         Supporting enrolment of CWD and RAE children in pre-primary and primary education; Transforming school to be safe, inclusive and participatory

·         Working closely with formal and non-formal education and service providers such as Handikos and its service provision centres across Kosovo, to advocate for increasing governmental investments for Inclusive Education (IE),

·         Enable and empower participation of children and community for children’s right

The methodology used is to comply with each Key Evaluation Question (KEQ). Some fixed methodology variables include Quality Learning Environment - Inclusive (QLE) toolkit (provided by SCIK) for the target primary schools, Quality of Services (QoS) toolkit (provided by SCIK) for the target Non-Formal Education/service provision centres, as well as focus group discussions with i. parents, ii. project partners, and iii. children (including)CLMA. Remaining methodology features are open to proposal by consultant, as needed in order to answer all KEQ.

The produced report on the final evaluation will on one hand take place for accountability purposes to the project and donors, it will however also be using as learning material in assessing SCIK approaches in fighting discrimination and violence and improve quality of pre-primary and primary education, and how effect and impact of these approaches can be maximized for future project design. The target audience of this report are thus the donors of the project, as well as the project team and SCIK Programme Development & Quality department, and SCIK project partners and community.

This project has undergone baseline and mid-line evaluations which were simultaneously carried out in Q1 2017 (with some data from Q4, 2016). The final evaluation of phase I also serves as the baseline assessment of phase II, and findings from both assessments have been used in adjusting and improving the project. Both studies, along with other relevant project documentation can be found in the annexes of this TOR.

 

Key EVALUATION Questions

The final evaluation report will be carried out by using SCI tools such as QLE – Inclusive and QoS as well as other methods in answering the Key Evaluation Questions, which are based on OECD DAC Criteria and seek the assessment of the Effectiveness of the project.

 

Key Evaluation Questions

KEQ 1.– Effectiveness:

a. Are the objectives of the project being achieved?

b. What interventions at school level were particularly effective in providing inclusive education in Kosovo, and why?  

c. Are the target primary schools (QLE) and the target service provision centres (QoS) taking appropriate measures to increase inclusiveness?

i. Are target schools attempting to increase the overall quality of learning environments?

ii. Do target service provision centres take into account the needs of different children, children with disabilities, other ethnicities, etc?

QLE Inclusive & QoS assessments to be made for target schools and service provision centres, on the areas that have been tackled by the project, based prior QLE & QoS assessments. Given that not the complete QLE and QoS must be administered, areas of assessments are to be set in coordination with SCIK.

KEQ 2. -  Impact:

a.Has the project contribute to the objective of fighting discrimination and violence and improve quality of pre-primary and primary education and society at large?

 

b.What difference has the project made for the following groups and stakeholders in relation to the overall objective of the project?

i.              Community & Stakeholders (MED/MEST representatives, parents, community members)

ii.             Partners (partner organizations, service providers, target school staff, support teachers)

iii.            Children - CLMAs

 

Overall Objective: Fighting discrimination and violence and improve quality of pre-primary and primary education and society at large, and building a culture of inclusive and safe quality learning environments in schools in targeted locations in Kosovo

KEQ 3. – Sustainability:

a.  Are the positive effects or impacts sustainable?

b.  Has the process of introducing inclusive education support teachers in targeted schools led to sustainability for the objectives?

KEQ 4. – Learning:

a.  What were lessons learned and what recommendations are to be made?

 

 

Evaluation Methodology

The methodology used for the final evaluation shall be participatory, inclusive (consultations with all stakeholders, considering gender, age, disability and other vulnerability considerations), sensitive of social norms and practices, ethical data collection (safety, informed consent, etc).

The main data collection methods to be used are primary. Qualitative data collection through focus group discussions with different stakeholders, such as teachers (as well as relevant education representatives of MED, MEST and primary school authorities), partner CSOs (Handikos and Putevima Sunca) as well as children (CLMA participants). Quantitative data is also to be collected through the Quality Learning Environment - Inclusive (QLE) assessments for sampled target schools, and Quality of Service (QoS) assessments for sampled non-formal education/service providers (Handikos centres). The QLE – Inclusive and QoS toolkits will be made available in local languages by SCIK. Secondary data/ desk research is also to be used in order to measure national/social trends.

Data collection must fulfil the following criteria:  

-       Disaggregated by gender: boys, girls, men and women

-       Other relevant disaggregation (urban or rural location, vulnerability status, etc)

-       Safety and ethics considerations for engaging children in evaluation

-       Data collection methods which are age and gender appropriate

Evaluation Management

Expected Workload:

What

Days of work

By when

Documentation review, desk research

2

13.11.2018

Inception report

1

13.11.2018

Data collection

12

30.11.2018

Data management (coding, transcriptions, data cleaning, etc)

2.5

04.12.2018

First draft report of evaluation & analysis

5

10.12.2018

Validation of evaluation findings and recommendations

(Consultant & SCIK)

0.5

11.12.2018

Final evaluation report

3

15.12.2018

Total

26

 


 

Expected Outcomes

Inception report: One week after selection of the consultant/evaluator, the latter is responsible to submit an Inception Report, which will highlight the methodology and the guiding principles of the evaluation. The report should highlight: Objectives and key questions, Methodology, Data collection methods, sampling considerations, timeline and logistics, etc.

Data collection tools: Data collection tools are available and to be provided by SCIK, thus no data collection tools need to be developed by the consultant/evaluator.

Evaluation Final Report:

-       Table of Contents

-       List of Acronyms

-       List of Tables

-       Executive Summary

-       Background – incl. project goal and outcomes

Scope of Evaluation

-       Methodology

-       Main Findings – KEQ1, 2 and 3

-       Conclusions and Recommendations

-       PPT with visualized findings, incl child friendly visualizations

-       SCI evaluation response plan

-       Annexes

  • Project logframe
  • Evaluation ToR
  • Objectives and key questions
  • Methodology
  • Study schedule
  • List of people involved

Sharing evaluation findings: Evaluation findings will be shared with internal and external stakeholders. Information will also be shared with communities, beneficiaries and children. Findings will furthermore be shared in a child-friendly manner through child friendly data visualization and data sharing in target schools and communities.

Applying evaluation findings: The Evaluation Response Plan will be developed involving representatives of SCIK, stakeholders and the consultant/evaluator. The follow up on implementation of recommendations will take place during future project design tackling inclusive education.

 

 

Evaluator(s)

Eligible evaluators/consultants are selected based on the following criteria:

-       Academic Background (MSc) in Education, Social Studies, Psychology, Public Policy/Administration and similar.

-       Technical competency in Education and Inclusiveness

-       Experience of evaluations, research conduction and analysis of similar complexity

-       Report writing and presentation skills   

Interested applicants should submit the below documents to scik@savethechildren.org  no later than November 7, 2018:

-Technical Proposal - Brief

- Financial Proposal

- CV of Evaluator(s)/Consultant enlisting academic background and relevant professional experience

- Sample of similar previous work, and list of similar publications as applicable

- 2 References

 



[1] Statistical Office of Kosovo, Labor Force Survey for 2009

[2]Save the Children has worked on Inclusion of children with disabilities in preschool and pre-primary.

[3]Kosovo Intelligence Agency (2015)

[4]Lost in Transition STP Report The Forced Migration Circle Of Roma, Ashkali And Balkan Egyptians From Kosovo

[7]Document available upon request to SCIK