Evaluating Educational Materials
Using LLMs to evaluate pedagogical quality
We’re developing tools to judge the quality of education materials at scale and individually.
Combining data science with a deep knowledge of education, our tools test the quality of education materials against evidence-based rubrics.
Introduction
How can large language models (LLMs) help evaluate educational materials at scale while keeping pedagogical quality at the centre? We took on this challenge, developing tools for evaluating the quality of foundational literacy and numeracy (FLN) content for low- and middle-income countries (LMICs).
Building on existing evaluation criteria frameworks, our education specialists developed pedagogical quality criteria for lesson plans, storybooks and textbooks. We then iteratively tested and refined these criteria to ensure they can be applied reliably by LLMs at scale.
As part of this work, we developed an online tool that helps others explore the approach and generate quality scores for their own materials. We also created a repository of materials through web scraping and crowdsourcing open educational resources and made it available through a dashboard. Our methodology and key lessons learned are documented in an interactive online report.
Using LLMs as a judge.
Scalable quality assurance.
Real-world innovation.
Test your lesson plan, storybook or textbook!
Our evaluation tool – designed by our education specialists – uses AI to provide detailed feedback for foundational literacy and numeracy content for low- and middle-income countries (LMICs).
Teachers, EdTech organisations,
policymakers & NGOs
Whether you create, adopt, or oversee educational content – this tool is built for you.
“Evaluate your lesson plans and teaching materials to ensure they meet quality standards before use in the classroom.”
Related resources
Find out more about our work to solve content curation at scale and use LLMs to judge the quality of education materials.
Interactive Guide
How can LLMs help evaluate educational materials at scale while keeping pedagogical quality at the centre? This interactive guide explains what we did, what we learnt and how you can replicate it for yourself.
BETA Tool
We created a repository of materials through web scraping and crowdsourcing open educational resources. You can find these materials through this BETA open-source dashboard.