Is the Sobral Education Miracle Real?

20 years ago, Finland stood at the centre of the international education world. A string of outstanding results on international assessments created a whole cottage industry, made up of study tours and school visits as a steady stream of educators, researchers, and policymakers from around the world came to discover Finland’s secret sauce. In poorer countries examples of real success are few and far between, but one contender in recent years has been the city of Sobral in the state of Ceará in Brazil. Despite being a poorer region of the country, Sobral and Ceará shot up the ranking on domestic tests. A World Bank report from 2020 hailed the “world-class education” in the city of Sobral, and another held up the State of Ceará as “a role model for reducing learning poverty”.

Nevermind Finland, the study tours are now headed for North-East Brazil, including from CGD. Attention came from the international media, and researchers at Oxford and Harvard set about exploring the practices underlying for the perceived success.

But what if the apparent success wasn’t real? New data raises doubts. The 2023 TIMSS survey was released on 4 December 2024, and showed that Ceará fell below average for Brazilian states, a very long way from world-class. The average Ceará score on maths and science in both grade 4 and grade 8 is below the low international threshold for basic competence.

Ceará topped all Brazilian states on the 2023 domestic tests for grade 5 (the Basic Education Development Index (IDEB) combines average test scores on maths, Portuguese, and the school’s rate of grade progression). By contrast it falls to just 14th place out of 27 states on their average score for grade 4 maths and science in the TIMSS survey.

So what’s happening? First, we should remember that much of the international attention has been on the city of Sobral rather than the wider state of Ceará. The new TIMSS data doesn’t let us look at individual cities, but there is plenty of other data showing the city of Sobral outperforming the state. On the optional “PISA for schools” test, 16 of 18 Sobral secondary schools took part in 2017 and scored similarly to the national Brazil average in Maths and Science, and 20 points above average in Portuguese. Brazil as a whole participated in the main PISA survey in 2022, the other major international test. But results for that survey were only available by region rather than by state—and the north-east region that contains Ceará also has eight other states. On that test, the north-east region lagged the rest of the country, unsurprisingly given the higher levels of poverty.

The city of Sobral aside, there is still a real fall in rankings for Ceará state. One potential part of the explanation could be testing different subjects. Much of the focus of reforms in Sobral and Ceará has been on reading, whereas the new TIMSS test is a maths and science test. But we might expect better reading performance to translate into better maths too. Further when we look at the subject breakdown of national tests, in the most recent data (2019) Ceara ranked 6th in Brazil in both reading and maths.

Another explanation could of course be Goodhart’s law—that when a measure becomes a target, it becomes a poor measure. There could easily have been some subtle “teaching to the test” on the national exams, such as teachers familiarising students with the format of the exam, if not necessarily the kind of outright manipulation seen in India.

Ultimately these new results do take the shine off Ceará state’s stellar domestic test scores. Without city-level results, it’s unclear if Sobral defies the broader trend. For now, it’s probably worth remembering that all too often in education, if something looks too good to be true, it just might be.