This article was originally published in The Notebook. In August 2020, The Notebook became Chalkbeat Philadelphia.
Nearly everyone agrees that high-quality pre-K can make a big difference for kids. But how should states measure quality?
That question is at the heart of a new brief from Research for Action, the Philadelphia-based nonprofit group that studies education. The answer will be critical to cities like Philadelphia as they try to expand pre-K coverage that actually delivers lasting benefits.
The problem is there’s no straightforward answer.
Shooting for the STARS
The RFA brief looks specifically at Quality Ratings and Improvement Systems. It’s a clunky term, but it refers to the rubrics that states use to rate early childhood programs. Pennsylvania’s is called Keystone STARS (for "standards, training, assistance, resources and support"), and it’s one of the oldest in the country.
There has been scads of QRIS research. Most of it, however, looks at whether a given QRIS has been implemented faithfully. The research is comparatively thin on whether ratings systems actually do a good job measuring quality.
In other words, a QRIS might rate one center a 2 and another center a 4, but it’s not clear whether there’s a big difference between level 2 and level 4 centers in terms of student achievement.
“They have been studied. It’s just a matter of figuring out how to best study the relationship between QRIS scores and student outcomes,” says Katie Englander, co-author of the RFA brief. “And so it’s a mixed bag so far.”
One study on California’s QRIS suggested the system did a good job measuring criteria that have been linked with positive child outcomes. Another examined Colorado’s QRIS and found that students at highly rated centers in the state fared no better than those from their low-rated counterparts.
A November 2015 study from the University of Pennsylvania’s Child Research Center gave mixed reviews to Keystone STARS, which uses a four-star rating scale. It found that children who attended a 3- or 4-star center did measurably better on assessments than children who went to 1- or 2-star facility. But it also found little difference between 1- and 2-star centers, and little to distinguish 3-star centers from 4-star ones.
That’s in part why Philadelphia’s new pre-K expansion is geared toward funding centers with either a 3- or 4-star rating. The city also has a provisional program for centers that are attempting to earn 3-star status.
City officials have essentially anchored their expansion efforts to this notion that Keystone STARS can indeed discern quality. And they’ve done so at a time when STARS itself is evolving.