Facebook Twitter

Lots of data, but basis for EMO decision remains murky

This article was originally published in The Notebook. In August 2020, The Notebook became Chalkbeat Philadelphia.

As the School Reform Commission gets ready to decide how or whether to continue the nation’s largest experiment in school privatization here in Philadelphia, it will have a mountain of data, as well as recommendations from two high-level reports, from an in-house review, and from CEO Paul Vallas.

Even so, neither Vallas nor the SRC has yet clarified their criteria for determining if contracts with six private managers should continue, end, or change significantly.

“I support keeping what works and throwing out what doesn’t work,” Vallas said in a February press briefing.

Said SRC chairman James Nevels: “There are a number of factors that have to be taken into account.”

The six private managers – the for-profits Edison Schools, Inc. and Victory Schools; nonprofits Foundations, Inc., and Universal; and Penn and Temple universities – have operated schools since 2002 under terms of the state takeover of the District. These providers are also known as education management organizations, or EMOs.

So far, the studies – one from the RAND Corporation and Research for Action (RFA), the other from a state-funded panel of experts — have cast doubt on the effectiveness of the private managers in driving student achievement beyond what the District could accomplish on its own.

The RAND/RFA study found that students in 41 privately managed schools had academic gains that were no different from those in the rest of the District. The schools that did stand out above the pack, particularly in math, were overseen by the District’s Office of Restructured Schools from 2002 to 2005. Those schools were kept under District control while getting supports including teacher coaches and extra time for planning, as well as more intensive focus on reading and math.

More specifically, the two reports raised questions as to whether the extra cost to the District of privatized school management – which has averaged about $20 million a year – can be justified, especially in light of a looming shortfall in the 2007-08 budget that has been estimated to be as high as $140 million.

The panel of experts, called the Accountability Review Council (ARC), noted that the RAND/RFA report – which it helped fund – “saw limited evidence to support the claim that private management as implemented in Philadelphia can turn around Philadelphia’s low-performing schools.” It urged the SRC to “rethink the appropriateness and the magnitude” of privatization as a reform strategy and urged “greater transparency” as it makes decisions about how to allocate limited resources among the private managers and other District schools.

In its evaluation, the ARC – headed by James E. Lyons, now the superintendent of public instruction for the state of Maryland – suggested a possible cap on the privately managed schools “until there is stronger evidence of their measurable academic success.” It also encouraged the SRC to “monitor the kinds of instructional benefits for students that additional state aid was able to purchase in EMO-managed schools.” The state has been giving the District about $25 million a year to hire outside school management providers.

The District’s own study, headed up by Chief Accountability Officer LaVonne Sheffield, is due near the end of the month. Sheffield said that her review would provide a snapshot of each school that, besides test scores, will include such information as student and teacher attendance, suspensions, and serious incidents.

Her team, which includes representatives from the legal, finance, special education, curriculum, and English-language learners departments, is trying to determine how the providers spent their money and whether they fulfilled the terms of their contracts, she said. The data will also be broken down provider by provider.

As part of the review, the District is holding seven community meetings between February 27 and March 1, each one devoted to feedback on a specific school manager. It has also conducted focus groups of teachers and principals from the privately managed schools and a telephone survey of randomly selected parents.

Concerned that the District’s plan for community input is not sufficient, two community groups – We Overcome and Southeast Pennsylvania Network for Family Health, Education and Welfare – have scheduled another set of sessions for feedback on the EMOs.

Sheffield said she has been under no pressure to do anything but a thorough, dispassionate review. “We’re going to tell them what the data tells us,” Sheffield said. But she doesn’t yet know how specific her team’s recommendations will be, or what form they will take. “We are planning to make a range of recommendations,” she said. “We’re going to be upfront… but at the end of the day it’s the leadership that makes the decision,” she said.

The Education First Compact – a local alliance of educational experts, advocates, and researchers convened by the Philadelphia Education Fund (PEF) – is urging the SRC to be clear about what is most important to them in evaluating the providers’ performance.

“I think it’s important that the criteria the SRC uses to make a decision about EMO contracts are known to the public,” said Carol Fixman, PEF executive director. “I don’t think the criteria are known yet.”

In testimony prepared for delivery at February 14th’s cancelled SRC meeting, Compact spokesperson Brian Armstead of PEF said the group wants the SRC to “state the specific findings upon which they base their decision on each [provider.]” And any new contracts should include “clear performance measures, including test scores, attendance and school climate data, and measures of parent, teacher, and student satisfaction.” The providers should also be required to annually report financial and performance data, the statement said.

Jolley Bruce Christman of RFA, who co-authored the RAND/RFA report, agreed that the SRC should consider more than just test scores in making decisions. “But the key is that … they need to come to the data with an open mind and not be driven by a particular ideological slant or political pressures,” Christman said. ”For schools that didn’t exceed the gains of other District schools, a very compelling case would need to be made on the basis of strong evidence from the other indicators for them to continue with their provider.”

The providers themselves, especially Edison Schools, Inc., were quick to question the results of the RAND/RFA study. Edison officials asserted that the overall test score improvement in Philadelphia was due to introduction of competition and privatization. They also noted that the providers were given the worst-performing schools and said their improvement shouldn’t be compared to the District’s as a whole. Nevels echoed that concern.

“I don’t think that’s fair, because these were the worst schools,” he told The Inquirer. “There was a determination by the Commonwealth that in the worst-achieving schools they wanted to see a new way of approaching things, and that’s exactly what happened.”

The RAND/RFA researchers maintain that their comparison between EMO and District schools is fair because their analysis compared how much progress each school made from their different starting points.

Nevels and Vallas both have signaled that they expect contract renewal decisions will be made on a school-by-school basis, with providers perhaps retaining some schools and losing others.

Vallas has also said, however, that in any contract extensions, he favors giving the privately managed schools more autonomy to fully implement their programs. When the providers were brought in, it was under a so-called “thin management” arrangement, under which they had limited ability to recruit and hire staff or change schedules.

“If they need to be more flexible to be effective, they should be given more flexibility,” Vallas said. He said he favors a model in which the providers are given the full school budget so they can “prioritize how the budget dollars should be spent.” He also said he favors “full site selection” of teachers.

“I’m not talking about the schools being non-union, but EMOs should have full site selection, should be able to determine who they want to hire and who they don’t want to hire,” he said.

As far as the extra dollars given to EMO schools – as much as $750 per student – Vallas said that if contracts are extended, “We might want to revisit how those EMOs are compensated.”

Vallas – whose future at the District may be in doubt due to SRC unhappiness with the budget crisis – has identified the contracts as a potential source of savings in next year’s budget. But he also pointed out that savings from EMO schools’ extra funds may prove limited because the District may have to pick up the cost of afterschool programs and other interventions in those schools.

“I believe in equity in school funding,” he said. “It’s not unfair [that EMO schools are] getting what they’re getting, but it’s unfair other schools aren’t getting it. All our schools are underfunded. It’s unfair we can’t get $750 extra [per student] for all our schools.”

In the meantime, providers are lobbying to keep their schools and carry on what has become the most watched experiment in school privatization in the nation.

Kent McGuire, dean of the Temple College of Education, stated that the community wants Temple to remain in its “partnership schools,” which include Meade, Duckrey, Dunbar, and Ferguson. But the university and its education school – which just took the reins of the initiative in January from the president’s office – are still considering their options.

“Whether we want to be in this particular relationship with the School District depends on where we come out of our own analysis, and the things we think we need, and whether the District wants to be in agreement with us. I couldn’t predict where things will come out,” he said.

Edison, on the other hand, has already said it wants to keep all 20 of its schools and provide districtwide services as well, such as afterschool programs and teacher training. John Chubb, the company’s vice president for education, said that Edison is open to making changes in any future contracts to help the company fully implement its model. Some of the company’s contracts elsewhere have rigorous performance standards and annual financial reviews, he said.

“It’s fair for the District to ask for whatever it thinks it needs as assurance it’s getting its money’s worth,” he said. “There’s no controversy here. Nothing that’s being asked for is something we don’t have experience with.”