This article was originally published in The Notebook. In August 2020, The Notebook became Chalkbeat Philadelphia.
Some critics of the federal No Child Left Behind Act (NCLB) have argued that large schools and schools with a diverse student population are penalized by the law’s provisions. School performance results from 2004 in Philadelphia appear to bear out this charge.
Few of the School District’s larger and more diverse schools achieved what the state considers "adequate yearly progress" toward achievement goals set under NCLB. Some now argue that smaller and more homogeneous schools may be getting off easy while some relatively successful large or diverse schools may be getting unfairly labeled.
"The research nationally makes it very clear that there is what amounts to a diversity penalty," said Monty Neill, executive director of FairTest, a Massachusetts-based organization that has been a strong critic of NCLB.
The law as a result "gives an inadequate identification of which schools really need extra help," Neill stated.
Schools must meet all targets
"Adequate yearly progress" (AYP), which is defined by each state based on specifications in the federal law, has become the most closely watched measure of school performance. To "make AYP," all schools and school districts receiving federal funds must meet all their targets for test scores and test participation for the overall student population and also for demographic "subgroups."
High schools must also meet targets for graduation rates, and in Pennsylvania, elementary and middle schools must meet targets for student attendance.
This year, 160 of 265 District schools (or 60 percent) met all their AYP targets set by the Pennsylvania Department of Education. That represented almost a threefold increase over last year’s results. Public attention has focused on these significant gains.
Lost in the celebration is the fact that some schools had many more targets to reach than others, and schools with more targets to meet were far less likely to meet all their targets as required by NCLB.
The reason schools have different numbers of targets is the provision of the law requiring that test performance and participation be broken down by "subgroups." The provision was an attempt to promote equity in the educational outcomes for different populations within a school.
Every subgroup of children is expected to achieve both proficiency and test participation targets in reading and in mathematics. Potential subgroups include different racial and ethnic groups, students with limited English proficiency, economically disadvantaged students, and special education students. But in Pennsylvania, a subgroup target only applies to a school if the subgroup contains 40 or more students.
Smaller and less diverse schools are not likely to reach the 40-student threshold for most subgroups and can essentially fly under the AYP radar, escaping any accountability for the performance of subgroups of students that may be disadvantaged.
In Philadelphia, several large public schools had to meet the test score and participation targets for each of as many as six or seven subgroups, while some schools had no subgroups to report and only had to meet targets for the overall student population.
More subgroups, less success
Of District schools that had four or more subgroups reporting, only one-fifth (four schools out of 20) made all their AYP targets – excluding those schools with selective admission criteria. These schools with four or more subgroups had to meet 21 or more targets.
But Philadelphia schools reporting one subgroup or none had much less trouble getting to AYP. Of these schools, 24 out of 25 made adequate yearly progress – when schools with special admissions criteria were excluded. These schools had to meet nine or fewer targets.
While these results in Philadelphia do not prove that having more subgroups is what prevents schools from making AYP, the findings are consistent with two recent research studies in California that show that the more subgroups a school has, the less likely it is to make AYP.
Researcher Robert Balfanz of Johns Hopkins University, who has analyzed data on Philadelphia middle schools, argues that a key determinant of whether high-poverty middle schools here failed to meet AYP was the number of subgroups reporting and specifically whether schools had to report the progress of special education students or limited English proficiency students.
Few Philadelphia schools had success at meeting targets among subgroups of special education students (see What does ‘AYP’ mean for students with disabilities?) and limited English proficiency students.
But close to 90 percent of Philadelphia schools were not held accountable for the performance of their special education students because they had fewer than 40 of those students tested. Among the 25 neighborhood schools with the fewest subgroups, all but one of which achieved AYP, not one of them had to count their results for special education or limited English proficiency students.
District had to meet 41 targets
Similarly, at the district level, Philadelphia experiences challenges in making AYP that face large and diverse districts. Despite increases in proficiency in nearly every area, the School District fell short on some of its districtwide AYP targets for the sixth consecutive year. That result earned the District the lowly status of "Corrective Action II, second year." This status could prompt a state takeover under NCLB, if one hadn’t already taken place.
Philadelphia as a district has to count its results for nine subgroups. It met its AYP targets for five subgroups, including economically disadvantaged students, and overall, it met 31 of 41 targets. The District came up short with four subgroups: Black, Latino, special education, and limited English proficiency students.
Yet the vast majority of districts in Southeastern Pennsylvania – 80 percent – had to worry about five or fewer subgroups compared to Philadelphia’s nine. No other district had to meet more than 33 targets overall, and most had far fewer.
Under NCLB, the highly regarded Radnor Township school system in Delaware County was not required to report on the progress of its Black students or economically disadvantaged students. It claimed only three subgroups – Whites, Asians, and special education students. In suburban Jenkintown’s school system, the only subgroup reported was White students.
Charters faced fewer AYP targets
A review of subgroup reports shows that Philadelphia charter schools, many of them smaller and less diverse than traditional public schools, faced fewer hurdles than District schools. All but three of the 43 charters tested in Philadelphia had two or fewer subgroups. Despite this advantage, charter schools have lagged behind District schools, with less than half making AYP this year.
Only one of 43 Philadelphia charters, Philadelphia Academy Charter School, had a special education subgroup, and it failed to meet its targets there. Not a single Philadelphia charter reported a subgroup of limited English proficiency students.
Critics of the NCLB "diversity penalty" say that it creates incentives for schools and districts to segregate their students. It may also create an incentive to underreport or reclassify students and avoid having to count the data for a subgroup.
Neill noted that with categories such as special education and limited English proficiency students, "you can choose to label them somewhat differently" – and schools can thereby stay below the reporting threshold and improve their odds of making AYP.
"Schools can shuffle kids around, push them out, label and re-label kids, or not count them in testing," Neill commented. "We’re going to see more and more of that," he predicted.