Your donation will help us produce journalism like this. Please give today.

This is the last in a three-part series examining California’s approach to education data and school accountability. Part One surveyed how the state’s skepticism of test-based accountability starts at the top with Gov. Jerry Brown, who successfully took on the federal government; Part Two explored how the elimination of certain data systems has limited educational research in one of the country’s most consequential states.

California is hoping to redefine school accountability in the “California Way.”

While state officials are hard at work designing a system in line with the oversights demanded by the Every Student Succeeds Act (ESSA), the new federal K-12 education law, they also want to remain true to the state’s ethos of de-emphasizing test scores and focusing on helping — rather than “punishing” — struggling schools.

“We have had now basically three years without a functioning accountability system and we’re approaching the moment when we actually have to put something in place,” said David Plank, a Stanford professor and executive director of the research group Policy Analysis for California Education (PACE).

A report released in May by a task force convened by the state superintendent lays out a series of metrics — beyond standardized test scores — for judging schools, but says little about what happens to the ones that persistently score poorly.

The system will also be crucial to the future of the state’s public schools, which grapple with limited funding and poor data systems that make it difficult to track areas where improvement is needed. California students post some of the lowest test scores in the country on federal exams and while, fairly wealthy, the state spends among the least on K-12 education.

As other states face similar tensions, California’s approach may point to the future of accountability across the country as one of ESSA’s pillars is devolving power from the federal government. The state appears on the verge of designing a system that moves beyond a narrow focus on math and reading test scores while including measures of equity, such as suspensions and expulsions, which fall disproportionately on special education students and those of color.

At the same time, it’s not clear that there’s the political will to give accountability any teeth — to intervene, and even, in some cases, sanction schools that are low-performing year after year by, for instance, labeling them as failing or dismissing staff.

Accountability but then what

The state must balance an array of competing interests in creating a new system for evaluating schools. California has to meet still-to-be-finalized federal guidelines under ESSA, while creating coherence between those new rules and its already-planned system for district-focused accountability.

Many want the state to continue to avoid a focus on test scores and accountability that imposes strict fixes; yet a number of advocates say California needs to intervene in schools that flounder year after year.

The plan is for school-based accountability to be underway in the 2017-18 school year — but many details need to be worked out before then.

Last month, the “accountability and continuous improvement” advisory task force, convened by state Superintendent Tom Torlakson, put out a list of metrics for judging schools, many of which were backed by the state Board of Education. The group was co-chaired by Eric Heins, head of the California Teachers Association, the state’s largest teachers union, and Wes Smith, executive director of the Association of California School Administrators.

In interviews, several task force members praised the group’s recommendations and collaborative approach.

Samantha Tran, of the nonprofit Children Now, was one of them. She described the task force’s two main camps: the “traditional education community” and a “number of equity advocates.” In the former, she included administrators groups, teachers unions and school board associations, among others, and in the latter, she included herself and others concerned with ensuring measures of equality were included.

“We really pushed on each other in terms of (our) bottom line — what were key things that people wanted to make sure were reflected in the report, so at the end of the day, we could all sign on to it,” Tran said.

The task force agreed that schools should be judged on a variety of measures, including tests scores, rates of suspension/expulsion, chronic absenteeism, college and career readiness, and high school graduation rates.

But the report was much less clear on what interventions look like when schools are deemed low-performing. The task force’s explanatory graphic speaks to the complex nature of the undertaking and in some cases a lack of clarity.

“The idea that large proportions of schools would be held accountable was taken off the table pretty quickly,” said University of Southern California professor Morgan Polikoff, also a task force member. Polikoff said the focus was largely based on what measures to use, “not what would happen based on those measures.”

“In the first meeting…I said, ‘There’s a quite robust (research) literature that suggests that consequential accountability improves outcomes for kids,’ and that comment was not well received,” Polikoff said, referring to studies showing that No Child Left Behind and state accountability systems have had positive impacts on student achievement.

There is little discussion about what, if anything, will happen to struggling schools in the state report. It promises to “ensure significant, sustained, evidence-based interventions in priority … schools” through “mandatory … (technical assistance) and supports that build (local educational agency)/school capacity to sustain improvement over time” for schools “that need more comprehensive and intensive supports to make major improvements in performance and/or growth.”

What, precisely, a “significant, sustained, and evidence-based intervention” is remains unstated. There has been much talk of reducing “punitive” sanctions against schools — but one person’s interventions may be another’s punishment.

“The ‘what happens’ question has not been answered,” said Peter Birdsall, task force member and the executive director of the California County Superintendents.

ESSA requires intervention in certain schools — those in the bottom 5 percent statewide; high schools where fewer than two-in-three students graduate — and specifically that those steps be “evidence-based.” But it largely leaves it up to the states, like California, to decide what the steps are.

But in echoes of recent history, California’s vision of accountability may in some cases run headlong into Washington, D.C.’s approach. Proposed federal guidelines for ESSA say that schools must be assigned a single, summative performance rating. California’s task force recommendations — released prior to the federal rules — suggests a “dashboard” approach that gives schools scores across multiple dimensions but avoids assigning them an aggregate rating.

California’s lack of robust state data systems may also make implementing the accountability system challenging since it will rely on a variety of new measures that must be accurately collected and reported.

“That came up repeatedly in conversations, where folks were in essence saying, ‘We don’t really have the capacity or systems to do this now,’” said Polikoff, who hopes that putting multiple measures into the rating system will force the state to create the necessary data infrastructure.

Accountability applied with nuance

Even those pushing for state oversight and accountability with consequences say they don’t want to return to what some see as an overemphasis on test scores, which led to inflexible sanctions imposed on poorly performing schools.

“There’s a desire … to think about a continuum that really does start with support,” said Tran of Children Now. “Education is a human endeavor. The construct of we’re going to fire the principal and half the teachers — that may make sense in a certain school, but what if you just hired this phenomenal principal and things are starting to turn around, but there’s another dynamic that’s at play?

“You almost need a little bit more judgment to go in and still apply that pressure because that pressure is important,” she added, “but have a more nuanced response based on the context of the school and the district.”

John Affeldt, of the nonprofit law firm Public Advocates, has been critical of accountability aspects that are too focused on test scores and said the measures being discussed now in California move in the right direction. However, he said, “I’m not supportive of no accountability and no consequences. … If we’re going to do support and assistance, then let’s make sure it’s working.”

Putting in place any sort of sanctions for struggling schools may still face an uphill political battle, considering the long-standing opposition of Gov. Jerry Brown and politically powerful state teachers unions.

“I’m concerned. We are seeing progress in the conversation around creating a more meaningful accountability system. However, we need to know clearly what supports and interventions our students are going to receive,” said Ryan Smith of Education Trust – West, which advocates for the achievement of students of color and those living in poverty.

Others argue that sanctions for low-performing schools do more harm than good.

“I do think that schools that are so-called ‘low performing’ should get the carrot rather than stick approach,” said Joshua Pechthalt, president of the California Federation of Teachers, the smaller of the two statewide teachers unions. “I taught at an inner-city high school in Los Angeles for many, many years … and people at that school worked their tails off. … We needed additional resources; we didn’t need to be sanctioned.”

A California Teachers Association spokesperson said in an email that the union was “providing input” on accountability but declined to comment beyond that.

“It’s more important to get it right, than to do it fast,” Heins, the CTA president and task force co-chair, wrote in a letter to the state Board of Education.

Brown’s spokesperson directed a request for comment to the state school board, whose spokesperson then pointed back to Brown’s budget message this year, which said the state plans “to establish an accountability system that provides a more accurate picture of school performance and progress than the past system.”

Meanwhile, a bill laying out metrics for accountability recently passed the state Assembly and is now in the Senate. Assemblywoman Shirley Weber, the bill’s sponsor, said her proposal shares significant overlap with the task force recommendations, which Weber said relied partially on her bill. But both are needed, she said, suggesting that task force recommendations don’t always translate into real action.

“We feel very strongly that our bill still needs to exist, because we know how task forces go,” she said.

Even Weber’s bill though does not specify interventions for struggling schools.

Rewards vs. sanctions

Many in California believe that past accountability and intervention systems have not worked. Research on California school interventions is limited — perhaps because the data needed to study the state’s education system is hard to come by — but there is some evidence from which to draw.

It shows that intervention and accountability systems can make a difference — often for good, but sometimes for ill.

A specific assistance and intervention program for certain low-performing districts — led by California and mandated by No Child Left Behind — showed positive results as measured by math test scores; so did a 2009 federal school turnaround grant that awarded up to $2 million each to some of California’s lowest-performing schools. The initiative was particularly successful in schools that dismissed their principal and half the teaching staff.

A 2013 paper, though, found that achievement decreased in California schools that were sanctioned for not hitting targets for certain disadvantaged groups, such as students of color or low-income students.

But a national study found that racial achievement gaps shrunk in states like Mississippi and Oklahoma when more stringent accountability pressure was based on the performance of vulnerable student groups. In general, No Child Left Behind — and its state-level precursors — produced gains in student achievement but also led to unintended consequences, like cheating and teaching to the test.

One of the most thorough studies of accountability examined the impact of Texas’ approach, which sanctioned persistently struggling schools while rewarding high-performing ones. The research found students in schools facing pressure to avoid a low rating scored higher on standardized tests, and, most importantly, graduated college more frequently and earned more money as adults. However, in high-performing schools eligible for extra recognition, struggling students were less likely to graduate college and made less money at age 25.

Despite questions of its effectiveness, California’s task force recommendations seem to place a greater emphasis on rewarding and recognizing high-achieving schools rather than putting pressure on low-performing ones to improve.

“Every single draft (of the recommendations) that came out … I kept copying and pasting the same thing, which was, ‘I don’t know why we’re spending so much space on awards; I know of virtually no evidence that those help kids,’” said Polikoff.

A leap of faith

California is known for a lot of things: vibrant and beautiful cities, an innovative economy and a top-notch state university system.

But when it comes to its K-12 public schools, California has far less to brag about. Its students score among the worst in the country on federal fourth- and eighth-grade math and reading tests. Across states, California ranked 46th, only slightly better than Alabama and a bit worse than Mississippi, according to one careful analysis that adjusted for demographic differences, like student poverty and English proficiency.

The reasons are complex and difficult to pin down, but a 2007 report by some of the state’s top researchers identified three major issues: a complex, inequitable and inadequate school funding system; the difficulty in firing ineffective teachers, and a lack of infrastructure to collect and use data. A 2012 follow-up report said that many of these problems remained.

Since then, there has been significant progress on school funding equity: Gov. Brown’s Local Control Funding Formula is designed to drive more money to schools serving disadvantaged students, while also enhancing local flexibility to use that money. The amount of money though remains an issue, and there are numerous concerns that the new funding is not reaching the students it’s meant to help.

By all estimates, California had for years spent among the least per pupil of any state in the country: $7,348 in 2013 vs. New York’s $16,726, according to the most recent comparable data. Voters passed a tax increase raising billions to better fund education in 2012, which appears to have helped significantly but still has not completely closed the gap between California and other states. In April, a state appeals court rejected a lawsuit backed by the nonprofit law firm Public Advocates arguing the state was inadequately funding schools; an appeal was filed to the state Supreme Court earlier this month.

In other respects, the state has seemingly made less progress. The weak data infrastructure remains, though a renewed push to improve it has cropped up. The well-known Vergara lawsuit claiming it’s all but impossible to fire an ineffective tenured teacher has been appealed to the state Supreme Court, after being thrown out by the Court of Appeal in April.

Political transitions also loom. By 2019, term limits guarantee that a new governor and state superintendent will take office — considering how fundamentally Brown’s philosophy of local control and distaste for data has shaped California’s education policy, the next governor may make a big difference.

This comes amid the backdrop of the state’s efforts to redesign the accountability system, which still has numerous unanswered questions. What will happen to struggling schools? How will the state merge its district-focused accountability system with ESSA’s school-focused one? How aggressive a role will the federal government play? Will the state legislature step in?

For now, at least, California continues to tread its own path — one other states, newly empowered by the end of No Child Left Behind and the dawning of ESSA, may follow.

“There’s a quite radical move away from ‘test-and-punish’ to creating a culture of ‘continuous improvement’ and that is a leap of faith,” said Plank of the research group PACE. “We have no idea whether we can make this work or not.”

---

This article was published in partnership with The 74.