It is not enough to produce data only to understand the extent to which issues affect the education system, such as the extent of inequities in educational opportunities and the lack of access, retention, and learning, affecting particularly certain marginalised children, among others (Custer et al., 2018; Abdul-Hamid, 2014). To build an equitable, quality education system, policy-makers must ensure that the data produced effectively informs the decision-making process (i.e. evidence-based policy (EBP)) (Raudonyte, 2019). Based on data, adopted policies can specifically target identified inequities and quality issues within the system. For instance, by ensuring that the education budget is equitably allocated throughout all the administrative levels; by equitably distributing teachers and resources throughout the system; by developing targeted approaches which guarantee that all children –particularly those the most marginalised– can access, participate and learn in school, among many others (Bernard, 2019; UIS-UNESCO, 2018).
* For specific information on how to produce quality data consult Policy page Quality data. This is a crucial step, given that lack of quality will lead policy-makers to disregard available data.
Bernard, E. 2019. How can data be used to revolutionize education? IIEP-UNESCO Learning Portal. Accessed 23 November 2019: https://learningportal.iiep.unesco.org/en/blog/how-can-data-be-used-to-revolutionize-education
Custer, S.; King, E. M.; Atinc, T. M.; Read, L.; Sethi, T. 2018. Towards data driven education systems: Insights into using information to measure results and manage change. Washington, DC: Center for Universal Education at Brookings/AidData. Retrieved from: https://www.brookings.edu/wp-content/uploads/2018/02/toward-data-driven-education-systems.pdf
Raudonyte, I. 2019. Use of learning assessment data in education policy-making. IIEP-UNESCO Working Papers. Paris: IIEP-UNESCO. Retrieved from: https://unesdoc.unesco.org/ark:/48223/pf0000367608?posInSet=2&queryId=ee579ff3-44e7-459d-bfaa-d258fb2a7970
UIS-UNESCO (UNESCO Institute for Statistics). 2018. Handbook on Measuring Equity in Education. Montreal: UIS-UNESCO. Retrieved from: http://uis.unesco.org/sites/default/files/documents/handbook-measuring-equity-education-2018-en.pdf
Promising policy options
Use of data to inform the policy-cycle
Data can be produced and collected from different sources such as household surveys, learning assessments, Education Management Information System (EMIS), school report cards (SRCs), among many others (consult Policy page Quality data). Both quantitative and qualitative quality, aggregated data should be used to inform the entire policy-cycle (UNESCO, 2019; Abdul-Hamid, 2017; Raudonyte, 2019).
The policy-making process is usually complex, yet, for a matter of comprehension, a simplified model will be cited to highlight the multiple ways in which data can inform it (Suteliffe and Court, 2005):
- Agenda-setting: data produced by different sources, such as learning assessments, can be used to ‘create awareness about the magnitude of an identified issue’ (Tobin et al., 2015: 6). For example, in Chile, the correlation between student’s socioeconomic status and their learning outcomes was brought to light by learning assessments’ data. As a result, public awareness on equity issues within the education system surged and called for the development of targeted policies (Meckes and Carrasco, 2010 cited by Raudonyte, 2019).
- Policy formulation: data can be used to design policy options and define strategies by examining potential outcomes (Tobin et al., 2015; Raudonyte, 2019). For instance, in Malawi, geospatial data and administrative data are used to inform teacher allocation throughout the country, particularly in disadvantaged communities (Akyeampong, 2022).
- Policy implementation: data can be used to enhance the effectiveness of initiatives (e.g. the impact of curriculum reform, the implementation of inclusive pedagogies within classrooms, etc.) (Tobin et al., 2015; Raudonyte, 2019). Data is key to understand ‘which solutions work, for whom, and under which conditions’ (Akyeampong, 2022: 36). To support policy implementation, it is key to ‘shorten the distance between data scientists and classroom practice’ (Castillo, Adam and Haßler, 2022: 130). For example, in two remote regions of Peru an initiative known as EduTrac was enforced with the support of community volunteers to improve attendance, increase accessibility to educational material, as well as sharpen the use of local funds (Castillo, Adam and Haßler, 2022: 130). Community members went to the schools to record data, based on a set of monitoring indicators through text messages (Castillo, Adam and Haßler, 2022). The data, processed by a central server, provided routine reports and helped keep track of the project’s implementation. This initiative shows how technology can be leveraged to support community participation in local implementation of projects to enhance their effectiveness.
- Monitoring and evaluation: monitoring and evaluation systems provide information about implemented policies. By analysing their outcomes, the produced data allows determining the effectiveness of implemented strategies (IIEP-UNESCO, 2015). This information should inform future policy-making (Raudonyte, 2019). For monitoring and evaluation processes, it is essential to design context-based, realistic and measurable indicators based on the education system’s specific targets, objectives, and goals (IIEP-UNESCO Learning Portal, 2019b; Lewin, 2015). (For more information on how to develop indicators consult Lewin, 2015).
In addition to national and local policy-makers, teachers, schools, students, and parents can mobilise the data produced:
- Consult Policy page Student learning assessments for details on how learning assessments’ data can be used by teachers, schools, students, and parents.
- Consult IIEP-UNESCO’s research on school report cards at http://etico.iiep.unesco.org/en/report-cards to learn more about ways in which community members and parents can use school data.
- See Annex 1 for a summary of the different ways in which data can be used by different stakeholders.
In order to effectively use data throughout the policy-cycle, the following aspects (see below) should be taken into consideration.
Annex 1
Data use by decision-makers
Source: Gill, B.; Coffee-Borden, B.; Hallgren, K. 2014. A conceptual framework for data-driven decision making. p. 5 Princeton: Mathematica Policy Research. Retrieved from: https://www.mathematica.org/our-publications-and-findings/publications/a-conceptual-framework-for-data-driven-decision-making
Akyeampong, K. 2022. ‘3. Teaching at the Bottom of the Pyramid: Teacher Education in Poor and Marginalized Communities’. In: D.A. Wagner, N.M. Castillo and S. Grant Lewis (Eds), Learning, Marginalization, and Improving the Quality of Education in Low-income Countries, (pp. 77-111). Cambridge, UK: Open Book Publishers. Retrieved from: https://doi.org/10.11647/OBP.0256
Castillo, N.M.; Adam, T.; Haßler, B. 2022. ‘4. Improving the Impact of Educational Technologies on Learning Within Low-Income Contexts’. In: D.A. Wagner, N.M. Castillo and S. Grant Lewis (Eds), Learning, Marginalization, and Improving the Quality of Education in Low-income Countries, (pp. 113-147). Cambridge, UK: Open Book Publishers. Retrieved from: https://doi.org/10.11647/OBP.0256
IIEP-UNESCO Learning Portal. 2019a. Developing a monitoring framework. Accessed 20 November 2019: https://learningportal.iiep.unesco.org/en/issue-briefs/monitor-learning/quality-and-learning-indicators
IIEP-UNESCO Learning Portal. 2019b. Quality and Learning indicators. Accessed 20 November 2019: https://learningportal.iiep.unesco.org/en/issue-briefs/monitor-learning/quality-and-learning-indicators
IIEP-UNESCO. 2015. ‘Booklet 6 – Monitoring and evaluation: How will we know what we have done?’. In: Safety, Resilience, and Social Cohesion: A Guide for Curriculum Developers. Paris: IIEP-UNESCO. Retrieved: https://unesdoc.unesco.org/ark:/48223/pf0000234818/PDF/234818eng.pdf.multi
Lewin, K.M. 2015. Educational access, equity, and development: Planning to make rights realities. Fundamentals of Educational Planning. Paris: IIEP-UNESCO. Retrieved from: https://unesdoc.unesco.org/ark:/48223/pf0000235003/PDF/235003eng.pdf.multi
Raudonyte, I. 2019. Use of learning assessment data in education policy-making. IIEP-UNESCO Working Papers. Paris: IIEP-UNESCO. Retrieved from: https://unesdoc.unesco.org/ark:/48223/pf0000367608?posInSet=2&queryId=ee579ff3-44e7-459d-bfaa-d258fb2a7970
Sutcliffe S.; Court, J. 2005. Evidence-based policymaking: What is it? How does it work? What relevance for developing countries? London: Overseas Development Institute. Retrieved from: https://www.odi.org/sites/odi.org.uk/files/odi-assets/publications-opinion-files/3683.pdf
Tobin, M.; Lietz, P.; Nugroho, D.; Vivekanandan, R.; Nyamkhuu, T. 2015. Using large-scale assessments of students’ learning to inform education policy: Insights from the Asia-Pacific region. Melbourne and Bangkok: ACER and UNESCO. Retrieved from: https://unesdoc.unesco.org/ark:/48223/pf0000235469
UNESCO. 2019. The promise of large-scale learning assessments: Acknowledging limits to unlock opportunities. Paris: UNESCO. Retrieved from: https://unesdoc.unesco.org/ark:/48223/pf0000369697.locale=fr
Promote a data-driven organisational culture
Promoting a data-driven organisational culture is essential to ensure the use of data through all administrative levels (Raudonyte, 2019; Gill, Coffee-Borden and Hallgreen, 2014). One way of doing this is by explicitly acknowledging the importance of quality data since all relevant stakeholders must comprehend the valuable role that data can play in promoting equitable, quality education opportunities for all (Abdul-Hamid, 2017).
A data-driven culture is facilitated by enhancing open communication, information sharing and accountability within the education system (Custer et al., 2018). This makes it paramount to empower stakeholders at the local, regional, and national levels to collect, share, and use data (Abdul-Hamid, 2017). For example, the project EdData II revealed that, in several countries, although data was available, it was ignored by decision-makers who did not have the culture of using it to inform the policy-making processes (Amy et al., 2016). This highlights the importance of fostering a data-driven culture within an organisation.
Financial constraints must be taken into consideration as data collection, analysis and use require sufficient resources.
Amy Mulcahy-Dunn, A.; Dick, A.; Crouch, L.; Newton, E. 2016. Education Data for Decision Making (EdData II). Research Triangle Park: RTI International. Retrieved from: https://globalreadingnetwork.net/sites/default/files/eddata/Core%20Final%20Report_16Dec2016_0.pdf
Custer, S.; King, E. M.; Atinc, T. M.; Read, L.; Sethi, T. 2018. Towards data driven education systems: Insights into using information to measure results and manage change. Washington, DC: Center for Universal Education at Brookings/AidData. Retrieved from: https://www.brookings.edu/wp-content/uploads/2018/02/toward-data-driven-education-systems.pdf
Gill, B.; Coffee-Borden, B.; Hallgren, K. 2014. A conceptual framework for data-driven decision making. Princeton: Mathematica Policy Research. Retrieved from: https://www.mathematica.org/our-publications-and-findings/publications/a-conceptual-framework-for-data-driven-decision-making
Raudonyte, I. 2019. Use of learning assessment data in education policy-making. IIEP-UNESCO Working Papers. Paris: IIEP-UNESCO. Retrieved from: https://unesdoc.unesco.org/ark:/48223/pf0000367608?posInSet=2&queryId=ee579ff3-44e7-459d-bfaa-d258fb2a7970
Build the technical capacity for data analysis
Building the technical capacity of relevant stakeholders to analyse and interpret education data should be a key concern (Raudonyte, 2019). Ministries of Education should ensure that capacity-building opportunities are being provided to units in charge of analysing the data produced by multiple sources.
Concerning EMIS systems, every user should gain the necessary skills to interpret, manipulate, and use the data produced (Abdul-Hamid, 2014).
Raudonyte, I. 2019. Use of learning assessment data in education policy-making. IIEP-UNESCO Working Papers. Paris: IIEP-UNESCO. Retrieved from: https://unesdoc.unesco.org/ark:/48223/pf0000367608?posInSet=2&queryId=ee579ff3-44e7-459d-bfaa-d258fb2a7970
Ensure data analysis is relevant, accessible, and timely available
Data collection processes should be based on a country’s educational goals and policy needs, with the collected data addressing and informing policy concerns (Raudonyte, 2019; IIEP-UNESCO, 2019; Amy et al., 2016; Mählck and Ross, 1990). Particular attention should be paid to the risks of data use, for instance:
- when data is used as a mean to legitimise policy strategies (taken upon other grounds than on the evidence generated by the same data) (Raudonyte, 2019; IIEP-UNESCO, 2019; UNESCO, 2019); and
- when only one type of data is taken into account, rather than multiple sources of information, which provide sufficient qualitative and quantitative information for an in-depth analysis (Tawil and Prince, 2019).
Implement a dissemination strategy to ensure data and findings are accessible and available on time for policy-making at all administrative levels. Data and findings should be generated for and disseminated at a national and sub-national level. Information provided should be locally relevant, otherwise, duplication and inconsistencies may arise (Amy et al., 2016).).
‘Translate’ findings to ensure their use by relevant stakeholders (Raudonyte, 2019; De Chaisemartin and Schwanter, 2017), with findings being presented in a clear, comprehensible manner to the targeted audience (Abdul-Hamid, 2014; Abdul-Hamid, Saraogi and Mintz, 2017). Develop ‘succinct and focused evidence synthesis, making better use of compelling data visualizations and smart technology to inform decision-making processes’ (Hinton and Kazmi, 2022: 441). In this light, ‘information brokers’ can play a key role by mobilising different sources of data to respond to policy-makers initial policy concerns, as well as provide evidence on additional aspects which should be taken into consideration (Saito, 2015).
Use multiple dissemination tools and means adapted to the targeted audience. For instance, official websites, mass media, annual educational statistical yearbooks, electronic databases, bulletins, newsletters, and downloadable documents (Abdul-Hamid, 2014; Raudonyte, 2019).
Ensure timely dissemination of findings in which data analysis is released on time so that it is effectively used through policy-making processes (Raudonyte, 2019; IIEP-UNESCO Learning Portal, 2019c) and provide sufficient resources for the dissemination process (De Chaisemartin and Schwanter, 2017).
Abdul-Hamid, H. Saraogi, N. Mintz, S. 2017. Lessons Learned from World Bank Education Management Information System Operations Portfolio Review, 1998–2014. Washington, D.C.: The World Bank. Retrieved from: http://documents.worldbank.org/curated/en/607441491551866327/pdf/114096-PUB-PUBLIC-PUBDATE-4-6-17.pdf
Amy Mulcahy-Dunn, A.; Dick, A.; Crouch, L.; Newton, E. 2016. Education Data for Decision Making (EdData II). Research Triangle Park: RTI International. Retrieved from: https://globalreadingnetwork.net/sites/default/files/eddata/Core%20Final%20Report_16Dec2016_0.pdf
De Chaisemartin, T.; Schwanter, U. 2017. Ensuring learning data matters. IIEP-UNESCO Learning Portal. Accessed 29 November 2019: https://learningportal.iiep.unesco.org/en/blog/ensuring-learning-data-matters
Hinton, R.; Kazmi, A. 2022. ‘Afterword: The Challenge Ahead for Learning at the Bottom of the Pyramid’. In: D.A. Wagner, N.M. Castillo and S. Grant Lewis (Eds), Learning, Marginalization, and Improving the Quality of Education in Low-income Countries, (pp. 439-442). Cambridge, UK: Open Book Publishers. Retrieved from: https://doi.org/10.11647/OBP.0256
IIEP-UNESCO Learning Portal. 2019c. Using data to improve the quality of education. Accessed 23 November 2019: https://learningportal.iiep.unesco.org/en/issue-briefs/monitor-learning/using-data-to-improve-the-quality-of-education
IIEP-UNESCO. 2019. Webinar – The use of learning assessment data: what have we learnt so far? Accessed 29 November 2019: https://www.youtube.com/watch?v=JUB94glBuAU
Mählck, L.; Ross, K. N. 1990. Planning the quality of education: The collection and use of data for informed decision-making. Paris: IIEP-UNESCO. Retrieved from: https://unesdoc.unesco.org/ark:/48223/pf0000086680/PDF/86680eng.pdf.multi
Raudonyte, I. 2019. Use of learning assessment data in education policy-making. IIEP-UNESCO Working Papers. Paris: IIEP-UNESCO. Retrieved from: https://unesdoc.unesco.org/ark:/48223/pf0000367608?posInSet=2&queryId=ee579ff3-44e7-459d-bfaa-d258fb2a7970
Saito, M. 2015. The use of learning assessments in policy and planning. IIEP-UNESCO Learning Portal. Accessed 20 November 2019: https://learningportal.iiep.unesco.org/en/blog/the-use-of-learning-assessments-in-policy-and-planning
Tawil, S.; Prince, M. 2019. Lessons for education planners: enabling effective use of learning assessment data. IIEP-UNESCO Learning Portal. Accessed 20 November 2019: https://learningportal.iiep.unesco.org/en/blog/lessons-for-education-planners-enabling-effective-use-of-learning-assessment-data
UNESCO Bangkok. 2017. Analyzing and utilizing data for better learning outcomes. Bangkok: UNESCO Bangkok. Retrieved from: https://unesdoc.unesco.org/ark:/48223/pf0000252975?posInSet=52&queryId=22d9d81e-8944-43a3-968f-9176a28e3a78
UNESCO. 2019. The promise of large-scale learning assessments: Acknowledging limits to unlock opportunities. Paris: UNESCO. Retrieved from: https://unesdoc.unesco.org/ark:/48223/pf0000369697.locale=fr