This article was originally published in The Notebook. In August 2020, The Notebook became Chalkbeat Philadelphia.
The flexibility that charter schools have in choosing whether to fill vacated seats with new students, also known as “backfilling,” has long been the subject of controversy. Critics of charters argue that a school’s statistics can be improved by choosing not to replace students who leave and point out that neighborhood public schools do not have that option because they have to accept every student in the catchment area who shows up wanting an education.
“We have heard estimates that there are 1,400 empty seats in charter schools during any given year, even while many charters claim to have long wait lists,” said Councilwoman Helen Gym in a statement this month, citing data obtained from the District Finance Office. Having “empty seats in charters — whether they open up because students are forced to leave or leave on their own — is a major point of contention for those who care about improving accountability and leveling the playing field between charter and public schools.”
KIPP Philadelphia cites a college-graduation rate – 35 percent for students who graduated from its original city middle school – far exceeding the 9 percent rate for low-income students nationally. But the statistic has come under fire from education blogger Gary Rubinstein, a public school math teacher in New York City, who called it “nothing more than a lie generated by KIPP’s PR department” because students who left between 5th and 8th grades were not replaced.
Although Rubinstein’s language was hyperbolic, data from the Pennsylvania Department of Education confirm his statement about KIPP middle schools in Philadelphia — at least until recently.
KIPP’s 35 percent college graduation rate was for the graduating class of 8th graders in 2007 — the first cohort of KIPP students who graduated from the KIPP Philadelphia middle school.
That cohort contained just 33 students in 8th grade, but when the same cohort entered 5th grade at the school, it had 86 students. In other words, most of the original students either left or were removed from KIPP Philadelphia’s middle school before 8th grade and so were not counted in KIPP’s graduation statistic. KIPP could have replenished its enrollment by using the lottery process to add students in higher grades at the beginning of each school year, but it did not do this for its inaugural Philadelphia class.
Presumably, this practice leaves a school with a cohort of the students with the best academic and behavioral records. The 9 percent graduation figure refers to all students coming from households in the lowest income quartile.
If KIPP had used the number from the 5th-grade cohort, of 86 students, KIPP’s college graduation rate would drop from 35 percent to 19 percent — assuming that the 53 students who left KIPP graduated at the 9 percent rate.
“There will always be students who arrive in the middle of the school year or students who need to start in a new high school in 10th or 11th grade. It is unfair that the burden for serving students in these situations falls so heavily on the District’s neighborhood schools,” Gym said in her statement, which did not refer to KIPP specifically. “Charters that only fill empty seats when they feel like it are rigging the system, and claims of higher graduation rates or higher student achievement at such schools must take into account that they benefit from such practices — while public schools lose.”
Why students left KIPP in the early years can’t be directly traced, but two out of the three KIPP Philadelphia schools with available Charter School Evaluations were criticized by the Charter Schools Office for harsh discipline policies — or failing to follow federal guidance on “clear and proportional consequences for misbehavior.”
“The school’s code allows expulsion for minor or non-violent behavior,” the evaluation reads. It goes on to give examples of this behavior: “failure to complete assignments, use of obscene or abusive language, [violation of] students’ dress code.”
When a school’s policy allows students to be removed for failure to complete assignments, the students who remain naturally tend to have better academic records and outcomes.
But the state of affairs across KIPP Philadelphia schools is not so simple today, because most of its schools over the last several years have begun to fill vacant seats.
Enrollment by grade at KIPP Philadelphia Preparatory Academy
KIPP Philadelphia’s original middle school didn’t backfill for the first seven graduating middle school classes. But with the incoming 5th-grade class of 2010-11 and every incoming class thereafter, KIPP began replacing students who left. (Older classes already proceeding through the school that year were not affected by this new policy, the numbers would suggest.)
KIPP West Philadelphia Preparatory has a nearly identical pattern, just delayed by one year.
“Ten years ago, KIPP CEO Richard Barth asked all KIPP schools to report their student attrition numbers every year, and to commit to bringing them down,” said a national KIPP spokesperson in an email. “In the intervening time, we have seen KIPP’s attrition numbers [drop] from 16 to 11 percent from 2007-2011.”
Barth elaborated on this process in an interview with Ed Week, in which he said that KIPP commissioned a study by the research firm Mathematica, which found that KIPP’s middle schools backfill “at similar rates to district schools” after Barth asked schools to report their attrition rates in 2007.
A more recent Mathematica study commissioned by KIPP found that “average impacts [on test scores] across the middle schools in the network declined somewhat since 2007,” lending credence to the idea that KIPP’s academic statistics benefited from the policy of not backfilling.
KIPP’s effect on students’ standardized test scores over time
The Mathematica study examined the “impacts” of KIPP on its middle school students’ standardized test scores in math and reading. The study defined impacts as growth in standardized test scores compared to a control group of students with similar “demographic characteristics.”
Although that methodology led the study to conclude that KIPP always had some impact — more in math than in reading — those impacts declined significantly from 2007 onward, a period of time that corresponds both with KIPP’s new policy of backfilling and with the steady expansion of KIPP schools across the country.
KIPP had been expanding for years prior to 2007, the year that backfilling began, indicating that the new backfilling policy may have had a larger impact.
The national KIPP organization is not the charter operator for KIPP schools in Philadelphia. Instead, it oversees a network of over 30 independent Charter Management Organizations (CMOs) in more than 20 states, including KIPP Philadelphia. Although the national organization provides advice and oversight, the local CMOs ultimately determine their schools’ policies in areas, including discipline, that could lead to higher attrition rates among students.
Although the data for KIPP West Philadelphia Preparatory show the same pattern with backfilling starting one year later than at their other middle school, KIPP Philadelphia’s new high school — KIPP DuBois — is still not backfilling its classes between years, according to state enrollment data.
DuBois only has two years of enrollment data available from the state, but that data show 12th-grade classes roughly two-thirds the size of the incoming 9th-grade classes.
“The District’s recent charter agreements mandate that charter schools fill seats when they open up,” Gym said. “The District must hold charters accountable to the same basic standards of public schools. Otherwise, school ‘choice’ will remain a rigged game.”
Unlike the national organization, KIPP Philadelphia did not respond to requests for comment on the policy.