Measuring the impact of learning with AI in Sierra Leone and beyond
A new controlled trial in Sierra Leone tests what happens when generative AI acts as a Socratic tutor instead of a glorified answer key....
Most headlines about artificial intelligence in education are exhausting. You get breathless PR notes about magic tutors or, conversely, apocalyptic warnings that kids will never learn to think again. It is all noise. That is why a recent randomized controlled trial in Sierra Leone caught my attention. Done in partnership with the local ministry and Fab AI, researchers actually measured how Gemini affected math progress for nearly eighteen hundred junior secondary students over eight weeks. No vague theories. Just hard data.
The core problem with generative models in a classroom is simple. If you hand a student an instant answer generator, they use it to bypass the hard cognitive friction required to actually learn something. But the trial used a specific pedagogical framework designed to do the exact opposite. Instead of spitting out solutions, the system relied on scaffolding and Socratic questioning. The telemetry is fascinating. Across more than a hundred thousand interactions, students sought conceptual understanding over ninety percent of the time, while the AI doled out direct answers in only two percent of its messages.
Look at those mechanics. When technology is engineered to withhold the easy shortcut, it forces the human brain to do the heavy lifting. Students using this guided approach jumped by a quarter of a standard deviation in math scores compared to the control group. That translates to well over a year of typical academic progress crammed into an eight-week window. More importantly, teachers didn't get sidelined; they stayed at the center of the room, shifting from traditional lecturers into active facilitators while using the tool to sharpen their own lesson prep.
We need to see more rigorous field experiments like this one and far less hand-waving from Silicon Valley boardrooms. Technology doesn't fix broken systems on its own. But when small teams build tools with actual pedagogical restraint, and when local educators retain complete agency over how those tools get deployed, things change. AI can be a legitimate lever for human capability. We just have to stop designing it for pure laziness and start building it for genuine comprehension.
