This article was originally published in The Notebook. In August 2020, The Notebook became Chalkbeat Philadelphia.
Your browser does not support the audio element.
An influential school reform group is urging Philadelphia District leaders to approve every charter applicant that can effectively run schools serving a high percentage of economically disadvantaged students.
Last week, District officials heard pitches from operators hoping to open 40 new charter schools across the city.
To back up its advice on charters, Philadelphia School Advocacy Partners – an arm of the Philadelphia School Partnership – has released a report that sorts the city’s public school landscape into two systems: "high impact" and "underperforming."
"For poor and minority students in Philadelphia, there really are two kinds of schools: those that work and those that don’t," said the report. "Students who end up in the effective schools achieve significantly better outcomes."
Of the 22 schools on PSAP’s "underperforming" list, 86 percent are District schools.
Of the 17 on its slate of "high-impact" schools, 70.5 percent are charters.
"The best schools serving disadvantaged students are disproportionately charters," says the report, which calls for schools on the "underperforming" list to be closed or turned over to new managers.
PSAP’s report relies on Pennsylvania’s School Performance Profile index – a rating system that is driven almost entirely by student achievement on standardized tests.
PSAP concedes the SPP has flaws.
"We note that the SPP is not a perfect tool for school evaluation; there is no perfect tool," the report says. "But whether one uses … any other combination of assessment measures, the story is the same: there is a vast chasm between the performance of schools serving similar populations of students in Philadelphia."
SPP issues aside, Research for Action, a Philadelphia-based nonprofit education research organization, issued a stinging critique of the PSAP report, saying:
"For a study to be credible, it must meet a set of quality standards that ensure rigor, consistency, and transparency. Our examination of the PSAP document raises questions about whether the analysis meets these criteria."
The data wars
In an interview with WHYY/NewsWorks this week, Mark Gleason, executive director of the Philadelphia School Partnership, responded to the criticism that PSAP has received for its methodology. (Listen to WHYY/NewsWorks’ extended interview with Mark Gleason by clicking the play button above.)
"What I would say to Research for Action is, ‘What’s your plan to do better by these students?’ Because the evidence is plowing more money into those schools that are not getting the job done is not going to be effective," Gleason said.
The PSAP report hinges on the idea that the schools in the two systems serve similar students, but Research for Action argues that the demographic data shows that the two systems have more differences than the report assumes.
Research for Action released a reaction brief arguing that "the data presented by PSAP are not nearly sufficient to support their sweeping conclusions."
According to state data, the 17 "high impact" schools, serve a student population that’s 73 percent black, 13 percent Latino, 88 percent economically disadvantaged, 6 percent English language learner, 13 percent special education, and 48 percent male.
The 22 "underperforming" schools serve a student population that’s 73 percent black, 15 percent Latino, 91 percent economically disadvantaged, 7 percent English language learner, 23 percent special ed, and 54 percent male.
In essence, the "underperforming" schools serve more black and Latino students, more low income students, more English language learners, more boys, and significantly more special education students.
Gleason says that, other than the special-education data, the numbers are "not statistically significant."
Research for Action says the difference in the poverty numbers would be more significant if not for what appears to be a data reporting error.
The state website on which the report relies lists Universal Audenried, a Renaissance charter, as being only 23 percent disadvantaged. In 2012-13, the school was listed as being 100 percent disadvantaged.
"Using the 2013-14 SPP figure dramatically deflates the percentage of economically disadvantaged students in the ‘underperforming’ sample," wrote RFA authors Lucas Westmaas and John Sludden.
A spokesman for the state Department of Education said the information came from Audenried itself. Universal Audenried couldn’t be reached for comment.
If one assumes Audenried’s economically disadvantaged figure to be 100 percent, the "underperforming" list jumps 3.5 percentage points in that category.
RFA’s response also says PSAP’s use of special-ed data is marred by "a significant omission."