Quality data

Producing quality data is of utmost importance as it allows decision-makers to identify issues affecting the education system’s quality. Quality data has the potential to provide a comprehensive picture of context-based challenges affecting the provision of equitable, quality educational opportunities for all (lack or misuse of resources; teacher quality; curriculum and assessments; school infrastructure; etc.). It also allows deeper understanding in the correlation between multiple factors and low educational outcomes. In addition, it facilitates the identification of the most disadvantaged groups within the education system.

Promising policy options

Measuring equity

The starting point for data production should be a consensus-based, clear definition of inclusion and equity (UNESCO, 2017). For instance, the Handbook on Measuring Equity in Education defines equity as ‘a distribution is fair or justified’ (UIS-UNESCO, 2018a: 17).

(For more information on promoting a clear common understanding of those key concepts, consult Policy page Equity and inclusion in policy and legal frameworks).

Instead of a ‘value what we can measure’ culture, an equitable and inclusive education system should be based on a ‘measure what we value’ approach (Ainscow et al., 2003 cited by UNESCO, 2017). Education 2030 Agenda provides a challenging opportunity ‘for creating tools that measure what really counts in education’ (IIEP-UNESCO, 2018: 236). Such as the presence, participation, and achievement of all learners, particularly those who are marginalised or at risk of marginalisation, exclusion or underachievement (IBE-UNESCO, 2008). Data should be produced about all school-age populations whether in school or not (see below).

Averages tend to mask disparities, which is why it is of utmost importance to disaggregate data by income, geographic location, age, sex, race, ethnicity, migratory status, disability, and other pertinent characteristics (United Nations, 2015; Kelcey, Guven and Burde, 2022). Generating gender statistics in educational participation will help policy-makers understand differences and inequalities within the system (click here to learn more on how to produce gender statistics). Additionally, take intersections into consideration as ‘sources of inequity frequently compound one another. It is, therefore, crucial to view child characteristics in conjunction with each other rather than in isolation’ (UIS-UNESCO, 2018a: 100).        

The following concepts should be taken into account when measuring equity in education:

• ‘Meritocracy: educational opportunities are distributed on the basis of merit.
• Minimum standards: educational opportunities must be at least the same for everyone below a certain threshold.
• Equality of condition: educational opportunities must be the same for everyone in the population, regardless of their different circumstances.
• Impartiality: educational opportunities should be distributed equally by gender, ethnicity, religion, language, location, wealth, disability, and other characteristics.
• Redistribution: a mechanism for compensation of initial disadvantage.’ (UIS-UNESCO, 2018a: 127).

While producing national and context-based data and approaches to measuring equity are of utmost importance, countries should keep in mind a shared framework to enhance the international comparability of data. This is essential to monitor progress towards Sustainable Development Goals, particularly SDG4 (UIS-UNESCO, 2018a).

Wide range of data sources

With the advent of the Sustainable Development Goals, the need for data increased significantly (UIS-UNESCO, 2018b). Countries are thus required to collect data from a wider range of sources, such as learning assessments, household surveys, administrative data, census, financial data, among others (UIS-UNESCO, 2017).

A comprehensive, in-depth picture of a situation calls for the cross-checking of multiple quantitative and qualitative data provided by different sources (Raudonyte, 2019). Solely relying on a single data measure will compromise the quality of the information produced. Decision-makers must not only decide on the type of data to collect, but also the amount of data and how it will be stored (Castillo, Adam and Haßler, 2022). Additionally, decision-makers must ensure that appropriate measures capture information from all children, and data is adequately disaggregated. This is key as this type of data allows an understanding of ‘what is currently going wrong, where, and for whom’ (Hinton and Kazmi, 2022: 440). The quality of data –and, more generally, its role in improving the education system– is compromised when it is produced by sources that exclude the most vulnerable and marginalised children (for more information consult Policy page Inclusive assessment systems).  

A number of recommendations should be taken into account concerning learning assessments; household surveys; administrative data and financial data.

Learning assessments can provide relevant information on children’s learning progress. It is essential to ensure background information is given as well. Since they do not take into account out-of-school children, other sources of information such as household surveys must be considered (UIS-UNESCO, 2018a).  For more details consult Policy page Inclusive assessment systems and Student learning assessments. Household surveys can be ‘used to monitor children’s school attendance and achievement rates, as well as examine the factors correlated with being out of school’ (Mont, 2018). This type of survey also provides rich background information.A sample-based household survey generates nationally representative data, without having to survey all households as is the case in a census (for more information on how to design and implement it consult UIS-UNESCO, 2018a). It requires to randomly select the villages and households within districts to survey. It is necessary to provide survey weights to account for the villages’ size. One example of this is the PAL Network citizen-led assessment surveys. 

Household surveys should include a learning assessment. This is essential to ensure that the learning process of out-of-school children is being tracked (UIS-UNESCO, 2018a). As explained by Benavot ‘only through learning assessments carried out in households do we gain some measure of the scale of educational disadvantage experienced by all members of an age cohort’ (IIEP-UNESCO, 2018: 222). Attention should be paid to how the learning assessment is designed so that it comprehensively acknowledges children’s competencies and skills. Avoid floor and ceiling effects: ‘defined by the inability of children to respond to any question in the examination or for most children to respond to all questions correctly’ (UIS-UNESCO, 2018a: 103). Certain issues should be considered when implementing household surveys. For instance, in countries where compulsory school legislation exists, households may be unwilling to admit that their children do not attend school (Lewin, 2015). Additionally, in countries where there is significant stigmatization of persons with disabilities, households may be unwilling to admit that one of the household members has a disability. Gathering data on persons with disabilities is of utmost importance, labelling, categorisation, and derogatory language must be avoided. To acquire information about different disabilities, the Child Functioning module CFM questionnaire can be included in household surveys at a marginal expense (Huebler, 2019; Mont, 2018). (Access the CFM questionnaires at https://data.unicef.org/resources/module-child-functioning/ ), In addition, certain households may be left aside, such as displaced populations, nomadic communities, households living in remote and unrecognized settlements, marginalised minorities (Lewin, 2015). Efforts should be made to reach and capture data on those who are living in informal households.

In addition to household surveys and learning assessments, administrative data is of key importance. A well-functioning Education Management Information System (EMIS) is ‘a critical component of an effective and equitable education system’ (The World Bank, 2016: 25). Through the collection, processing, and dissemination of information, an EMIS is a key component in monitoring the quality of the education system, as well as the extent to which it promotes equity and inclusion.

An EMIS can monitor the student’s attendance and learning progress. Ministries of Education must provide a unique identifying number for every child in order to track their progress throughout their educational life (Lewin, 2015). Child-tracking cards can be developed and integrated to monitor ‘grade progression, age in grade, attendance, and learning achievement’ as well as develop ways to notify at-risk-students (Lewin, 2015: 146).  

Particular difficulties encountered by children must be recorded. For this matter, EMIS can integrate the Child Functioning Module to record the different functional difficulties which children may encounter (seeing, hearing, moving, communicating, among others), without the need for medical categorization (Grant Lewis, 2019; Mont, 2018; UNICEF, 2014).  An EMIS should aim to link data on children’s access and learning to their background information (socioeconomic status, geographical location, age, gender, race, ethnicity, migratory status, disability, and other pertinent characteristics.).

For example, the FIJI EMIS tool can continuously collect and update student-level data on an individual basis as well as on the school environment (for more information consult: https://planipolis.iiep.unesco.org/en/2017/fiji-education-management-information-system-femis-disability-disaggregation-package-guidelines). 

An EMIS should capture information on school facilities, materials, and human resources to ensure quality and inclusion within the education system (Mont, 2018). For example, the UNESCO-led Open EMIS initiative includes information on the school environment.

(For more information on the equitable distribution of human resources and materials consult Policy pages Inequitable distribution of teachers and Inequitable distribution of Teaching and Learning Materials TLM. For more information on the school, facilities consult Policy page School physical infrastructure).

An EMIS should also capture information on safety, resilience and social cohesion (consult IIEP-UNESCO, 2015 to get more information on how to design indicators for that matter).

In addition, collecting data on public and private education spending is essential to understand how the budget is being allocated and if sufficient resources are being provided to those who need it the most. Sources such as National Education Accounts NEAs and Public Expenditure Tracking Surveys (PETS) may be useful (The World Bank, 2016; IIEP-UNESCO Learning Portal, 2019a). Countries should strive to make this information accessible.

Other sources of data to take into account include: school census; school audits; school inspections; school report cards (SRCs); Quantitative Service Delivery Surveys (QSDS); mapping through Geographic Information Systems (GIS); and employer surveys, among others.

(Consult IIEP-UNESCO Learning Portal, 2019a for more information on these various sources).

In addition to quantitative information, it is essential to produce qualitative information (illustrates the “how and why” of issues (Mont, 2018)). This is crucial to provide a comprehensive picture of inclusion and equity issues within the education system. In order to collect qualitative evidence, direct observation systems can be adopted, although challenging to implement, they have the potential to provide insightful information (IIEP-UNESCO, 2019a; Ainscow, 2005) (for more information consult Policy page Classroom practices supervision). 

For all of the different sources, the indicators chosen to collect data should be carefully analysed (Lewin, 2015; Antoninis, Delprato and Benavot, 2016):

• Are they too intrusive or sensitive to be collected?
• Are they too expensive to collect?
• Is it necessary to collect individual information on every child or can the data be acquired on a stratified sample?
• Is the data collected representative of the general population?
• Is the data reliable to track trends over time?

Guarantee data’s quality

UNESCO Institute for Statistics has developed both the Code of Practice (CoP) for education statistics as well as the Education Data Quality Assessment Framework (Ed-DQAF) which provides essential guidelines to guarantee the data’s quality. The following aspects should be taken into consideration (IIEP-Learning Portal, 2019c; UIS-UNESCO, 2017; IMF, 2006; The World Bank and UIS-UNESCO, 2003):

Ensure an enabling institutional environment to guarantee the data’s quality. Clearly specify the actors involved (and their respective responsibility) in the collection, analysis, and dissemination of data. Choose between a centralised or decentralised data collection system. If resources are limited, a centralised system may be more effective. Yet, when resources and capacity are available, decentralised systems have the advantage to speed the process and reduce the risk of error (IIEP-UNESCO, 2015).

Provide sufficient resources by investing in technology and human resources (e.g. ensure adequate training to personnel). ‘Pooling the financial, material, and human resources of different actors and entities involved in the education sector (ministry, non-governmental organisations, bilateral and multilateral, private sector, etc.) should be encouraged’ (IIEP-UNESCO, 2015: 21).

Create a culture of data quality. Statistical work should be informed by quality awareness. Respect the ‘principle of objectivity in the collection, compilation, and dissemination of statistics’ (The World Bank and UIS-UNESCO, 2003: iii). Integrity can be ensured by ensuring that professional principles and ethical standards guide statistical policies and practices, and by ensuring that statistical policies and practices are transparent.

Guarantee statistical procedures’ methodological soundness. Align the concepts and definitions used and the statistical frameworks applied, as well as the scope and coverage, classifications and recording frameworks to internationally accepted standards, guidelines, and good practices.

Guarantee the accuracy and reliability of the data produced. Ensure that the data source and the statistical techniques used portray reality well. The data source should provide an adequate basis for the compilation of statistics. The data and intermediate data should be regularly assessed and validated. Revision processes must be tracked as they are crucial to ensure the reliability of the data produced.

Statistical results should be serviceable, which means that the statistics should cover relevant information, be timely and consistent, as well as allow a predicable revision of policy and practice. The statistics should also be consistent within a dataset and over time, as well as with other major data sets.

They should also be accessible. Present data in a clear, impartial and comprehensive manner. In crisis contexts and areas affected by conflict, prioritization of data to be collected is of crucial importance. The use of technology, such as SMS reporting, could be promoted to collect data from locations that are difficult to access (IIEP-UNESCO, 2015). Metadata should be up-to-date and pertinent. Ensure a prompt and knowledgeable support service. Countries can call upon international organisations with technical expertise for support, such as the Technical Cooperation Group on the indicators for SDG4-EDUCATION 2030 (TCG), the Global Alliance to Monitoring Learning (GAML), the Agency Group on Education Inequality Indicators (IAG-EII), the Working Group on Statistical Capacity Building (WG SCB), and the Working Group on Data Reporting, Validation and Dissemination (WG DRVD). (For more information consult UIS-UNESCO, 2018a and UIS-UNESCO, 2017).

Producing quality data is of utmost importance to guarantee their utility in policy-making processes (Schleicher and Saito, 2005). (For specific information on data use consult Policy page Use of existing data).