A Response to “Correcting the Facts about the One Newark Plan: A Strategic Approach To 100 Excellent Schools”

Full report here: Weber.Baker.OneNewarkResponsewithexecsum

Mark Weber & Bruce Baker

Summary

This brief is a response to the Newark Public Schools rebuttal of our analysis of the district’s schools restructuring plan, One Newark. In this response, we find:

• The consequences of the One Newark plan are racially disparate, creating a possible legal challenge for both the families of students and staff. NPS, however, has not acknowledged this part of our analysis.
• NPS uses scale scores from state tests, averaged across grade levels, in their rebuttal. We find these measures to be seriously flawed, and certainly no better than the measures we used in our initial report.
• Even using these flawed measures, we still find the classifications of schools under One Newark to be arbitrary and capricious when accounting for student population characteristics.
• Even when using scale scores, we find no evidence that the student population of Newark will do better under schools run by charter management organizations. Further, the patterns of student cohort attrition in some charter schools and other behaviors lead us to question the validity of One Newark’s charter takeover strategy.
• The statistical models used by NPS in their rebuttal are fundamentally flawed: specifically, the author(s) did not account for collinearity within the NPS model, biasing the results towards NPS’s favored position.

Introduction

On March 11, 2014, the Newark Public Schools (NPS) released a response to our policy brief of January 24, 2014: “An Empirical Critique of One Newark.”[1] Our brief examined the One Newark plan, a proposal by NPS to close, “renew,” or turn over to charter management organizations (CMOs) many of the district’s schools. Our brief reached the following conclusions:

•  Measures of academic performance are not significant predictors of the classifications assigned to NPS schools by the district, when controlling for student population characteristics.
• Schools assigned the consequential classifications have substantively and statistically significantly greater shares of low income and black students.
• Further, facilities utilization is also not a predictor of assigned classifications, though utilization rates are somewhat lower for those schools slated for charter takeover.
• Proposed charter takeovers cannot be justified on the assumption that charters will yield better outcomes with those same children. This is because the charters in question do not currently serve similar children. Rather they serve less needy children and when adjusting school aggregate performance measures for the children they serve, they achieve no better current outcomes on average than the schools they are slated to take over.
• Schools slated for charter takeover or closure specifically serve higher shares of black children than do schools facing no consequential classification. Schools classified under “renew” status serve higher shares of low‐income children.

In its response[2], NPS questions both our methodology and our data sources. We are pleased to engage NPS in a thoughtful dialogue about One Newark; however, their rebuttal unfortunately confirms many of our conclusions about the plan, and refuses to even acknowledge many of our critiques.

Rather than answer NPS’s criticisms point-by-point, we take this opportunity to focus on the larger issues NPS raises about our brief, addressing specific arguments within the body of this response. It is our intention here to further the dialogue about One Newark in the hopes that NPS will move toward a position of transparency and engagement with stakeholders, both in and out of Newark.

Conclusions

We are pleased that “An Empirical Critique of One Newark” has generated a response from the Newark Public Schools administration. We have watched over the last few months as the topic of the One Newark plan has generated strong reactions from stakeholders both in and out of Newark. Given the changes that One Newark will bring – changes that even NPS agrees are profound and far-reaching – a measured, careful analysis of the rationale and consequences of these changes is clearly necessary.

Our conclusions are informed by public data using standard statistical methods. We labor to make our results replicable and understandable: we believe it is a testament to our work that NPS was able to respond to “An Empirical Critique” without any questions as to why we reached the conclusions that we did, even if they disagreed with those conclusions.

We believe it is time for NPS to make a similar commitment to transparency in their own formulations of policy. Despite their protestations, we are still no closer to understanding how NPS classified particular schools than we were before. We still do not know NPS’s rationale for why three particular schools are being taken over by two particular CMOs. We still do not know why staff at particular schools face an employment consequence while staff at other schools do not. We don’t know why NPS proposes to divest particular facilities to particular parties.

Backwards-engineering a rationale for One Newark does not contribute to transparency. Using flawed measures like averaged scale scores does not increase stakeholders’ faith in NPS’s ability to justify its plan. Engaging in poor statistical practice does not lead to confidence in NPS’s judgments. And failing to fulfill legal obligations to release data in a timely manner does not encourage a candid exchange of views.

We agree that the educational outcomes of Newark’s students are not acceptable, and that change is needed in the lives of Newark’s deserving children. Whether that change can come solely, or even primarily, through the policies of a state-run school district is an open question. We heartily agree, however, that school policies certainly matter, and Newark should constantly strive to make its schools better, even in the face of seemingly insurmountable problems whose solutions lie outside the purview of the public schools.

But no change can come unless and until an open dialogue about education takes place in front of a well-informed public, where all stakeholders have access to the inner working of the mechanisms that generate policies. If our briefs have compelled NPS to begin to engage in this dialogue, we will consider our time analyzing One Newark to have been well spent.

---

[1] https://njedpolicy.wordpress.com/2014/01/24/new-report-an-empirical-critique-of-one-newark/

[2] http://onewark.org/wp-content/uploads/2013/12/StrategicApproach.pdf

Advertisement