Your donation will help us produce journalism like this. Please give today.

A new working paper could give educators powerful new motivations to invest in summer programs, which seem to stem the tide of learning loss due to the COVID-19 pandemic — at least in math.

The paper, from CALDER at the American Institutes for Research, looked at the academic progress of students who attended summer school in 2022 across eight school districts, most of them urban or suburban, in seven states.

Here are four key takeaways:

1. Finally, a bit of positive news about post-pandemic interventions.

The new paper represents what could be the first encouraging findings coming out of post-pandemic interventions, said CALDER’s Dan Goldhaber. Earlier research has been discouraging, he noted, with few positive effects. “One of my takeaways is that it’s nice to be able to say, ‘Hey, there is something that school systems can do to help kids get back on track,” he said, “even if it’s only making incremental change.”

2. Summer school works for math … but for reading: not so much.

The researchers found that summer programs had sizable positive effects on students’ math achievement, potentially closing about 2% to 3% of districts’ total learning losses in math, but not in reading. The math gains were “positive and significant,” said Goldhaber, large enough for researchers to suggest that districts consider offering summer math programs to many more students in the future. Reading scores improved in just one of the eight districts.

He noted that research has established that so-called “math effects” due to school interventions are more likely than reading effects.

“The ‘math but not reading’ is consistent with education research writ large,” he said. Simply put, schools have a more significant impact on kids’ math skills than on reading skills, probably because kids read and write outside of school, but don’t necessarily do math.

3. Post-COVID summer programs are at least as good as those schools operated pre-COVID — and they’re targeting kids who need them most.

Researchers compared impacts of current summer programs to those operating before the pandemic and found that they’re having “about the same kind of impact as summer school programs pre-pandemic,” Goldhaber said. That’s encouraging, since in many districts, summer programs have grown in scale but haven’t suffered in quality, according to the new findings. Any time educators push to scale up interventions, he said, it’s harder to maintain quality. “So it’s encouraging” to see quality stay high.

He also said the programs they examined typically targeted students who were struggling and actually needed the extra help.

4. The good news about summer math learning is tempered by the fact that so few students are getting it at the moment.

The CALDER researchers estimated that only 15% of eligible students in 2022 were receiving summer math instruction. That means schools last year were under-utilizing what could have been a powerful, effective intervention.

Eamonn Fitzmaurice/LA School Report/Getty Images

The positive effects, Goldhaber said, “are kind of dwarfed by the magnitude of the COVID learning losses,” with the small number of students in effect disguising its potential effect on achievement. What could be a game-changer for the moment shows a “pretty small” effect on achievement as a result.

The new study is part of a larger “Road to COVID Recovery” partnership between researchers at the American Institutes for Research, Harvard University, NWEA and 11 school districts in total. Just eight supplied data to this study.

Researchers used value-added models to estimate the effect of each of the eight summer programs on MAP Growth test scores, with Spring 2022 as the baseline and Fall 2022 as the outcome. Summer sessions ran from three to six weeks, depending on the program, and daily classes ranged from 45 minutes to two hours.

CALDER’s findings could scarcely come at a better time, with recent NAEP scores suggesting that COVID had a “cataclysmic” impact on K–12 education, coming on the heels of a decade of stagnation.

Other recent research from Michigan showed that the pandemic slowed students’ math achievement over the three-year period from spring 2019 through spring 2022, with achievement growth “substantially lower” than that of comparable students in the three earlier years.

As with the CALDER findings, the Michigan researchers found that scores for English language arts, which include reading and writing, were small and generally not statistically significant.

In December, researchers from CALDER, NWEA and Harvard University, looked at achievement in a dozen mid-to-large sized school districts, enrolling more than 600,000 students across 10 states, and found that between fall 2021 and spring 2022, schools had basically put an end to student achievement declines in math and reading relative to pre-pandemic levels — but that average test score gains during the 2021-22 school year hadn’t moved past pre-pandemic levels.

Students in a few elementary grades improved substantially in math, but beyond a few areas, researchers didn’t find a lot of compelling evidence of recovery in other subjects or grades.

Aaron Dworkin, CEO of the National Summer Learning Association, said he was encouraged by the new CALDER findings, adding that summer programs can often try different strategies “that you might not be able to always utilize” during the school year.

He noted one successful free program in Detroit called Math Corps, created by Wayne State University’s math department, that uses an unusual model: College math majors get paid to teach high school students, who get paid to teach middle school students.

The program maintains a fun, playful high-energy atmosphere that catches students’ attention, especially in the summer. “Hundreds of kids and families love it,” Dworkin said. “And they’re so supportive.”

---

This article was published in partnership with The 74. Sign up for The 74’s newsletter here.