This article was originally published in The Notebook. In August 2020, The Notebook became Chalkbeat Philadelphia.
The School District of Philadelphia has launched a new website to display the results of a survey offered to parents, students, teachers, and principals at every District and charter school.
The survey was designed to measure and publicize stakeholder feedback, which recent research has found helps provide a more complete evaluation of a school – as opposed to relying solely on standardized test scores.
The site allows users to view responses from individual schools in five categories: leadership, parent/guardian-community ties, professional capacity, climate, and instruction.
For each question asked, users see a graph representing the range of responses from each category of stakeholders at the school, next to a graph of the average responses in the District to this question.
Instructions on how to use the surveys (Credit: School District of Philadelphia)
“We hope the lessons found in the data help to build trust within school communities,” said Superintendent William Hite. “Administrators, educators, students and families should be involved in conversations about improving academic outcomes.”
The survey was created and administered in partnership with the University of Pennsylvania Graduate School of Education (GSE), with support from an Institute of Education Science grant. In January 2016, a partnership between the District and GSE known as Shared Solutions will host a conference to discuss the results and how they can be used to improve schools in Philadelphia.
Tonya Wolford, deputy at the District’s Office of Research and Evaluation, said the surveys can “help put information into perspective. If a school is deemed to have a difficult climate, is it the neighborhood or in the building? Is the issue in the hallways or in the classrooms? Depending on the answers to these questions, you would need a different approach to addressing climate issues.”
Although the opportunity to take the survey was given to every stakeholder, it was voluntary, and response rates vary depending on the school and category of stakeholder. While 70 percent of principals and 53 percent of teachers filled out the survey, only 33 percent of students and 7 percent of parents responded.
Low response rates are particularly problematic at individual schools, because the website has a minimum number of responses needed to display a result.
For example, users are unable to view the responses of parents at Samuel Fels High School to the statement “My child’s school provides me with regular feedback about my child’s progress” because only 7 parents took the survey – a less than 1 percent response rate. Instead, users see a message: “There are not enough responses to display results for this question.” The same message is displayed for every question on the parent/guardian survey for Fels.
The parent/guardian surveys have a minimum threshold of 5 percent of a school’s enrollment, or at least 25 parents – whichever is greater. With a District average parent response rate of 7 percent, Fels is not alone in having incomplete survey results.
The student minimum is much higher, requiring at least 25 percent of students or 50 students – whichever is greater – for the District to display any results of the student surveys.
The larger the portion of responders, the more accurate the survey is at representing the opinions of a stakeholder group.
A few dozen schools report a student response rate of over 90 percent, but three schools got no student responses – Widener Memorial, Elkin, and Barton – and many others are in the single digits.
“The entire process of administering it – pulling students into computer labs – was something that had to be worked into their plan,” said Wolford.
Overall, charter school student response rates were worse than for District schools. At 17 charter schools, not a single student responded to the survey, and many more schools have near-zero response rates.
While charter manager Universal Companies has student response rates of at least 25 percent at every school, other providers like ASPIRA that operate multiple schools have zero student responses. Mastery Charter has three of its 11 schools reporting no student responses, and the other eight are near zero – well below the 25 percent minimum needed to display results.
Shared Solutions made phone calls last spring to school leaders and sent email reminders to school staff, but the schools themselves were responsible for informing students and parents about how to take the survey. Shared Solutions conducted the survey between April 20 and June 19.
The primary method of administering the survey was online, using the SurveyMonkey program. Each school was given paper copies of the student and parent surveys for households that lack Internet access.
However, schools were only provided a number of paper copies equal to 5 percent of the number of students enrolled at the school. The District’s technical report states that “photocopying or otherwise duplicating these surveys was prohibited.” This and a lack of Internet access may have contributed to the 7 percent average response rate on the parent surveys.
In 2015, a census called America Community Survey reported that 36 percent of homes in Philadelphia lack Internet access. But this disproportionately affects certain neighborhoods. In Kensington, 90 percent of homes were reportedly without access.
The District plans to keep conducting comprehensive surveys every school year to better inform strategies for school improvement. The 2015-16 survey is being designed, and the goal is to start distributing it in March.
“I think the online results should help with the response rates,” said Wolford. “Now people can really see what it means to have stakeholders take the surveys.”
Wolford said the District will also be extending the administration window, raising the minimum threshold for parents to at least 20 percent, and raising the student threshold to 50 percent. This will make the results more accurately represent stakeholders at each school, but schools will also need an even greater response rate for results to be displayed.
And she said the paper surveys will be distributed differently next year. “We are going to ask schools to opt in for paper surveys and request a specific amount.”
Reflecting the opinions of all stakeholders is the goal, but whether response rates can be boosted remains to be seen.