Trackosaurus and Stellenbosh University: Game-based assessments using speech recognition

With our support and working alongside Stellenbosch University, Trackosaurus integrated speech recognition into their assessment games for South Africa's isiXhosa and Afrikaans languages. They fine-tuned automatic speech recognition (ASR) models using transcribed child speech. The games help teachers monitor children's progress in oral narrative skills and provide access to maths games for pre-readers.

Why this is important
Large class sizes make it hard for teachers to assess students well. Speech recognition can automate assessments, allowing access for student who are pre-reading and providing rapid information for teachers about learning gaps and interventions needed.

Key Learnings

1. Speech recognition models work better if trained on closely related languages - Our model (Whisper) worked better on Afrikaans than isiXhosa probably due to similarity with training languages. Pretraining models in African languages may yield benefits for related ones.
2. Test a range of different models initially before selecting one - Investigating different models more widely during early development can avoid unnecessary time spent fine tuning models which don't fit your needs.
3. Consider developing with at least two languages at the same time - Language differences between isiXhosa and Afrikaans were significant. Beginning with both early on might have meant learning specific to each could be recognised and accounted for.

Grant we provided: $56,345