This article was originally published in The Notebook. In August 2020, The Notebook became Chalkbeat Philadelphia.
“Strong progress, but a long way to go,” is how the summer’s news about test scores in the School District has often been reported.
But this neat sound bite does not adequately describe the current data about student performance in Philadelphia.
With a three-year track record now available on Philadelphia’s nationally watched “experiment” in school reform and privatized school management, a picture of school improvement trends in Philadelphia is emerging that is complex and uneven.
For example, improvement in K-8 math performance has been impressive. But high school students’ performance in reading and math has actually deteriorated as measured by the state PSSA exam.
A local researcher cautioned that three years of results is still “too limited a window to make conclusive comments” about the success of different reform models and initiatives.
Eva Gold of Research for Action, a local organization that has been monitoring school reform efforts in Philadelphia, noted, “The test scores do show gains, and this is good news. This type of progress is very consistent with the test score gains that are often seen in an initial phase of a reform effort.”
But she added, “What is very important is to watch if the current reform effort continues to deepen itself and continues to positively impact teaching and learning so that test scores don’t plateau after a few years.”
Capturing headlines
In September, the news was not all good: 132 Philadelphia schools met their targets for Adequate Yearly Progress (AYP), compared to 160 in 2004. But this year’s state performance targets for AYP in reading and math were higher than last year’s and so a drop in the number of schools making AYP had been predicted.
District reports of a continued rise in overall scores on both the TerraNova and PSSA exams captured headlines earlier this summer.
At an August 22 press conference, local and state officials greeted the news that overall state test scores had climbed by offering accolades for the School District’s reform program and for the private providers running over 40 Philadelphia schools.
State Rep. Dwight Evans, one of the architects of the 2001 state takeover of Philadelphia schools, was one of those proclaiming success. “The results announced today show that we are clearly on the right track,” he said.
But as is the case in many urban districts, Philadelphia’s high schools do not seem to be on the right track.
The average PSSA scores of 11th graders in both reading and math are lower now than they were in 2001 when the takeover was being engineered.
The percentage of Philadelphia 11th graders earning a mark of Below Basic, the lowest possible score on the PSSA, has grown from 44 to 54 percent in reading and from 56 to 60 percent in math during that period.
District CEO Paul Vallas expresses hope that better prepared ninth-graders, recently implemented curriculum reforms, and an initiative to create more small high schools will start to boost high school scores.
In contrast to the grim high school results, among fifth and eighth-graders across the District, gains in math performance seem to be accelerating, to a point where close to half of Philadelphia fifth-graders are scoring proficient or advanced in math.
While the three-year trend in K-8 reading scores has been positive, improvements in reading have been more modest than those in math.
Management models
Meanwhile, two conference papers about Philadelphia schools presented this summer pointed out that while the schools managed by outside education management organizations (EMOs) are receiving extra funds for their work, they are producing results no better than comparison groups.
A review of the test scores shows that at schools that have remained under the management of the School District for the last three years, test score gains have outstripped those of all of the private management models.
Schools managed by the School District’s Office of Restructured Schools have shown particularly strong gains. But this office, which had been overseeing reform at the lowest-scoring District-run elementary and middle schools, was dismantled recently as part of a District reorganization.
“Preliminary analysis of both the TerraNova and the PSSA raises questions about the diverse provider model,” Gold commented about the paper by Research for Action. “Using a rigorous statistical analysis, there doesn’t seem to be evidence that EMOs are accelerating improvement significantly more quickly than the District.”
She added that a more thorough analysis of the data is planned, using a “value-added” approach that monitors the progress of the same groups of students over several years.