“Beating the Odds”: A Comparison of the Demographics and Performance of Charter Schools to District Schools in Jersey City

Ajay Srikanth, PhD Student, Rutgers Graduate School of Education

Bruce Henecker, EdD Student, Rutgers Graduate School of Education

February 5, 2016

Note: All opinions here are those of the authors and do not reflect those of their employers, the NJEA, Rutgers GSE, or their professors and advisors there.

PDF Policy Brief: Srikanth_Henecker.JC.Feb5_2016

Executive Summary

On May 13, 2015, the Jersey City Council passed a resolution urging the Governor and State Legislator to provide equitable funding for the charter schools in Jersey City. The council argues that the charters in Jersey City are some of the highest performing schools in the city and also serve demographically-similar students. Both of these assertions are inaccurate. On average, charter schools serve significantly lower percentages of students eligible for Free Lunch, lower percentages of Special Education students, and substantially lower percentages of English Language Learners. With respect to student achievement, charter schools do not outperform district schools in Language Arts or Math once you control for demographics.

In response to the council’s resolution, we propose the following:

  1. Recommend to the Jersey City Council that they pass a resolution requiring that charter schools operating in Jersey City hold weighted lotteries that increase the rate of students receiving free lunch, students who receive special education services, and students classified as Limited English Proficient (based on a recommendation stated in Weber and Rubin, 2015).
  1. Recommend that the New Jersey Department of Education develop an enhanced charter funding formula that takes into account the increased cost of educating students across the range of Special Education classifications so

that charters receive a reimbursement rate commensurate with type of special education students they serve.

  1. Recommend that the New Jersey Department of Education develop an enhanced charter funding formula that reduces the base-funding amounts charter schools receive if they do not accept English Language Learners at a rate consistent with that of the host district.
  1. Recommend that policymakers account for demographic differences when comparing school performance within and across sectors- district vs. charter.

Research Note: On Average, Are Children in Newark Doing Better?

Bruce D. Baker & Mark Weber, Rutgers GSE

PDF of Policy Brief:Baker.Weber.NewarkBetterOff.NJEPF.11_15_15

In this research note, we estimate a series of models using publicly available school level data to address the following question:

Q: Did students in Newark (combined district and charter) make gains on statewide averages (non-Newark) on state assessments, controlling for demographics?

Specifically, we evaluate changes in mean scale scores on state assessments (NJASK) for language arts and math grades 6 to 8.

Newark Reforms Since 2009

Schools in the city of Newark have undergone a series of disruptive reforms since 2009, including substantial increases in the numbers of children served in charter schools, adoption of a unified enrollment system, ratification of a performance based teacher contract, and school closures, reconstitutions and reorganization.[i] Some of these reforms were instituted following the much publicized gift of Facebook founder Mark Zuckerberg, chronicled in Dale Russakoff’s The Prize.[ii]

A commonly asked question in the aftermath of these disruptions is whether students in Newark on the whole are better off than they were before these reforms? That is, were the disruptions and resulting political turmoil worth it? Some have chosen to speculate, based largely on anecdotal evidence, that children in Newark must be better off today than before these disruptive reforms.

Chris Cerf, former NJ Commissioner of Education and current State Superintendent of Schools for the Newark Public Schools, asserts that the past few years have brought significant positive changes for Newark’s schools:

Whether the measure is graduation rates, improved instructional quality, last year’s improvement in the lowest-performing schools targeted for special intervention, a nation-leading new collective-bargaining agreement, the addition of many new high-quality public schools, increased parental choice, or a material increase in the proportion of effective teachers, the arrow is pointed decidedly up in Newark.

“To be sure, as is always the case, the evidence of improvement is textured and in some respects uneven. The many positive indicators and trend lines, however, paint a picture of hope and progress that is completely at odds with the pessimism that has made its way into the standard storyline.”[iii]

Tom Moran, Editorial Page Editor of the Star-Ledger and a consistent supporter of the Newark reforms, writes: “The growth of charters has not damaged the kids in the traditional system. In fact, they’ve made modest improvements.”[iv] In a post on his Facebook page, Mark Zuckerberg, whose $100 million gift was the catalyst for the NPS reforms, writes: “No effort like this is ever going to be without challenges, mistakes and honest differences among people with good intentions. We welcome a full analysis and debate of lessons learned. But it is important that we not overlook the positive results.”[v]The chief-of-staff for Cory Booker, former mayor of Newark and current U.S. senator who was instrumental in secure Zuckerberg’s donation, states: “Newark students are quite simply better off now than they were five years ago.[vi]

In these conversations, “better off” is often reduced to whether or not, on average, across district and charter schools, student test scores for children in Newark have improved. That is, are students achieving more than they otherwise would have, had there been no such disruptions? It remains far too soon to measure longer term outcomes, including graduation rates, college attendance or economic outcomes.

While we are unable to compare against what might have been in the absence of reforms, we can at least evaluate whether children in Newark have made progress when compared to statewide averages, controlling for student population characteristics.

Conclusions

In a recent interview, Russakoff stated that she did not believe Newark’s students are better off today than they were five years ago: “…it feels like a wash.”[i] The analysis herein, while admittedly narrow in scope and short in time frame perspective, finds that Russakoff is correct. Average state assessment scores in grades 6, 7 and 8 are pretty much right where they were – relative to non-Newark students – in 2009.

Follow-up analyses are certainly warranted, but limited by changes in state outcome measures.

[i] http://www.nj.com/opinion/index.ssf/2015/09/author_dale_russakoff_discusses_new_book_on_newark.html

[i] Weber, M. (2015) Empirical Critique of One Newark: First year update. New Jersey Education Policy Forum. https://njedpolicy.wordpress.com/2015/03/12/empirical-critique-of-one-newark-first-year-update/

[ii] Russakoff, Dale (2015) The Prize: Who’s in charge of America’s schools? New York: Houghton Mifflin Harcourt

[iii] http://www.njspotlight.com/stories/14/12/01/op-ed-dispelling-five-falsehoods-about-newark-s-school-system/

[iv]http://www.nj.com/opinion/index.ssf/2015/09/newark_moran.html

[v] https://www.facebook.com/zuck/posts/10102461248065941?fref=nf

[vi] http://www.nj.com/essex/index.ssf/2015/11/newark_power_brokers_back_zuckerbergs_school_donat.html

 

The (Mis-)Use of Data in Dale Russakoff’s The Prize

Mark Weber, PhD candidate, Rutgers University, Graduate School of Education

Full Review: Weber_RussakoffFINAL-1

Executive Summary

Dale Russakoff’s The Prize is one of the most discussed books on education policy in recent memory. Russakoff tells the story of Facebook founder Mark Zuckerberg’s $100 million donation toward education “reform” in Newark, NJ, and the subsequent changes in the city’s schools.

Russakoff cites several data points in the course of the book in an effort to generalize the individual stories she recounts. Unfortunately, many of these points rely on proprietary data, rendering them unverifiable. Several are presented without proper context, creating a false picture of the reality of schooling in Newark. Others are contradicted by publicly available and uniformly reported sources.

Russakoff, for example, states that a single charter school has more social workers per pupil than a single NPS school. State data, however, shows Newark’s public district schools have more educational support personnel of many types per student than Newark’s charter schools. Russakoff also compares Newark’s per pupil spending to that of entire states, without accounting for differences in student populations or regional wage differences.

A central theme of The Prize is that Newark’s public district schools suffer from budgetary bloat; state data, however, shows that district schools spend less than charters on administration. Plant spending at district schools is not excessive compared to charters.

Russakoff does acknowledge differences in student population characteristics between district and charter schools. She does not adequately explore, however, how these differences, and differences in resources, affect student outcomes. She also repeats the contention of many of her book’s protagonists that teacher quality is unduly poor in district schools without examining how teacher characteristics have changed in Newark during the last several years of “reform.”

Policy makers should approach The Prize with caution: while an interesting and compelling narrative about the politics of school “reform,” the book’s misapplication of data to uphold its theses makes it an inadequate analysis of education policy in Newark and elsewhere.

 

Camden’s “Transformation” Schools: Racial & Experience Disparity in Staff Consequences

Mark Weber

Doctoral Student, Rutgers Graduate School of Education

Policy Brief: Weber_CamdenTransformationsFINAL

Executive Summary

This brief examines the 2015 “transformation” plan of the Camden City Public Schools, which will transfer five district schools to charter management organizations. When using New Jersey’s growth measures, adjusted for student demographics, I find that these are not the “most struggling schools” in Camden, as the district asserts.

Staff at these schools must reapply for their positions, but are not guaranteed employment. I find that this consequence affects Camden’s black teachers more than its white teachers, even when controlling for school-wide growth measures and student body characteristics. Using a logistic regression model, black staff are 1.6 times more likely to face an employment consequence than white staff. Similarly, staff with 5 to 24 years of experience are between 2.3 and 3.4 times more likely to face this consequence than staff with less than 5 years of experience.

Previous research suggests the loss of experienced teachers and teachers whose race aligns with students could negatively impact student achievement. At the same time, there is little evidence to suggest the charter management organizations taking over transformed schools will fare any better at improving test-based student outcomes.

CCPS should immediately release its methodology for identifying the transformation schools as the “most struggling” in the district, and justify the potential loss of experienced and black staff under its plan.

 

One Newark: Choosing “Great” Schools, or Merely Segregated Ones?

Full Length Brief: Weber_One NewarkSegregation-1

Mark Weber, PhD candidate, Rutgers University, Graduate School of Education

Executive Summary

This brief provides a preliminary analysis of the potentially segregative effects of One Newark, the school choice plan implemented by the Newark Public Schools (NPS) in 2014. School choices appear to be influenced by ratings assigned by NPS; however, these ratings are correlated to student population characteristics, such as race and economic status. Newark families, therefore, may be the choosing schools – inadvertently or otherwise – that are more segregated.

“Popular” schools under One Newark – the ones chosen most often by families – enroll fewer students eligible for the federal free-lunch program, a proxy measure of economic disadvantage. Popular charter schools also enroll relatively large proportions of black students compared to all of the city’s publicly-funded schools, even as popular district schools enroll relatively small proportions.

While popular schools show better performance on statewide assessments, their “growth” scores, which are intended to take into account differences in student populations, are more mixed. Because test scores are correlated to student population characteristics, families that choose higher-performing schools under One Newark may be selecting schools that are more segregated.

There are notable differences between popular district and popular charter schools: the popular charters have higher suspension rates and more inexperienced teachers than the popular district schools. Whether families are aware of these discrepancies is unknown.

Although the limited data released by NPS is inadequate for a full analysis, I find these results to be sufficient evidence to warrant the release of the full set of One Newark application data.

Empirical Critique of “One Newark”: First Year Update

Testimony before the Joint Committee on the Public Schools

PDF Version (Introduction below): Weber Testimony

New Jersey Legislature

Mark Weber

INTRODUCTION

Good morning. My name is Mark Weber; I am a New Jersey public school teacher, a public school parent, a member of the New Jersey Education Association, and a doctoral student in Education Theory, Organization, and Policy at Rutgers University’s Graduate School of Education.

Last year, I was honored to testify before this committee regarding research I and others had conducted on One Newark, the school reorganization plan for the Newark Public Schools. Dr. Bruce Baker, my advisor at Rutgers and one of the nation’s foremost experts on school finance and policy, joined me in writing three briefs in 2014 questioning the premises of One Newark. Dr. Joseph Oluwole, a professor of education law at Montclair State University, provided a legal analysis of the plan in our second brief.

I would like to state for the record that neither myself, Dr. Baker, nor Dr. Oluwole received any compensation for our efforts, and our conclusions are solely our own and do not reflect the views of our employers or any other organization.

Our research a year ago led us to conclude that there was little reason to believe One Newark would lead to better educational outcomes for students. There was little empirical evidence to support the contention that closing or reconstituting schools under One Newark’s “Renew School” plan would improve student performance. There was little reason to believe converting district schools into charter schools would help students enrolled in the Newark Public Schools (NPS). And we were concerned that the plan would have a racially disparate impact on both staff and students.

In the year since my testimony, we have seen a great public outcry against One Newark. We’ve also heard repeated claims made by State Superintendent Cami Anderson and her staff that Newark’s schools have improved under her leadership, and that One Newark will improve that city’s system of schools.

To be clear: it is far too early to make any claims, pro or con, about the effect of One Newark on academic outcomes; the plan was only implemented this past fall. Nevertheless, after an additional year of research and analysis, it remains my conclusion that there is no evidence One Newark will improve student outcomes.

Further, after having studied the effects of “renewal” on the eight schools selected by State Superintendent Anderson for interventions in 2012, it is my conclusion that the evidence suggests the reforms she and her staff have implemented have not only failed to improve student achievement in Newark; they have had a racially disparate impact on the NPS certificated teaching and support staff.

Before I begin, I’d like to make a point that will be reiterated throughout my testimony: my analysis and the analyses of others actually raise more questions than they answer. But it shouldn’t fall to independent researchers such as me or the scholars I work with to provide this committee or other stakeholders with actionable information about Newark’s schools.

Certainly, we as scholars stand ready to provide assistance and technical advice; but the organization that should be testing the claims of NPS and State Superintendent Anderson is the New Jersey Department Of Education. The students and families of Newark deserve nothing less than a robust set of checks and balances to ensure that their schools are being properly managed.

One Newark can be thought of as containing four components: the expansion of charter schools; a “renewal” program for schools deemed to be underperforming; a system of consumer “choice,” where families select schools from a menu of public and charter options; and continuing state control of the district.

This last component is clearly a necessary precondition for the first three. Given the community outcry against State Superintendent Anderson and One Newark, it’s safe to say that none of the other three components would have been implemented were it not for continuing state control.

The critical questions I ask about these components are simple: do they work, are there unintended consequences from their implementation, and is One Newark being properly monitored and evaluated? Let me start by addressing the expansion of charter schools in Newark.

Research Note: Resource Equity & Student Sorting Across Newark District & Charter Schools

Bruce D. Baker

PDF: BBaker.NJCharters.2015

Executive Summary

In this brief, I present preliminary findings that are part of a larger, national analysis of newly released federal data, a primary objective of which is to evaluate the extent to which those data yield findings consistent with findings arrived at using state level data sources. In this brief, I specifically explore variations in student characteristics and resources across schools in Newark, NJ.

I begin by reflecting on my most recent policy brief on charter and district school performance outcomes – growth percentile data from 2012 and 2013 – noting that on average, Newark Charter schools remain relatively average in student achievement gains given their student populations. But as noted on previous occasions, Newark Charter school student populations are anything but average.

Next, I use longitudinal data from the NCES Common Core of Data, public school universe (the source of underlying demographic data for the newly released federal data) characterizing changes in Newark Charter market share (share of children served in Charter Schools) and the share of low income children served in Newark Charter schools.

Next, I explore what the newly released (albeit already dated) federal data say about Newark Charter school demographics, compared to district schools serving similar grade distributions.

Next, I explore resource distributions and teacher characteristics across Newark schools, charter and district. The question at hand here is whether across district and charter schools, those schools serving needier and more costly student populations also have more (or fewer) resources with which to serve those children. Further, whether among schools serving similar student populations, resource levels are similar.

Forthcoming analyses of charter schools in New York City found that those schools tended to serve less needy populations (than district schools) and were able to do so with substantially more resources that district schools serving similar populations. Because the share of children in the district served by charters remained small, their disruptive effect on equity remained small. By contrast, in Houston, charter schools both served more comparable student populations, and did so, on average, with more comparable resource levels, resulting in less disruption of equity. In each case, the more interesting story, however, was the extent of variation among charter schools, both in students served and in resource levels.

Here, I explore similar questions in the City of Newark, first with the newly released Federal data and then with the most recent four years of available state data (2010 to 2014).

Conclusions & Policy Implications

To summarize:

  • Recently released federal data, confirmed by more recent state data indicates that student population differences between Newark district and charter schools persist.
    • Newark charter schools continue to serve smaller shares of children qualified for free lunch, children with limited English language proficiency and children with disabilities, than do district schools serving similar grade ranges.
    • While charter school market share has remained relatively small (through 2013), the effect of charters underserving lower income students on district school enrollments has remained relatively modest.
  • Charter school total staffing expenditures, either as reported in federal data or as compiled from state data appear to fall in line with student needs in charter schools.
    • Charter schools serve less needy populations and do so with relatively low total salary expense per pupil.
    • But, there exists significant variation in resources among charter schools, with some outspending otherwise similar district schools and others significantly underspending otherwise similar district schools.
  • Charter school wage competitiveness varies widely, with some charters paying substantially more than district schools for teachers of specific experience and degree levels. But these wages do not, as of yet, substantially influence total staffing costs.
  • Charter schools have very high concentrations of 1st and 2nd year teachers, which lowers their total staffing expenditure per pupil but only to the point where those staffing expenditures are in line with expectations (not lower, as one might expect for schools with so many novice teachers).

Finally, comparisons between the newly released Federal data collection and updated state data sources appear both relatively stable over time and relatively consistent across sources even as the charter sector rapidly grows and evolves and as the district continuously morphs.

Two issues require consideration by policymakers and local officials if reliance on charter schooling and expansion of charter schooling are to play a significant role in the future of schooling in Newark. The first is the active management of the potential deleterious effects of student sorting on district schools – that is, as market share increases and the tendency remains for charters to enroll (or keep) fewer of the lowest income children, district schools may be more adversely affected.

An appropriately designed centralized enrollment system can partially mitigate these issues. But (at least) two factors can offset the potential benefits of such a system. First, individual choices of differently motivated and differently informed parents influence who signs up to attend what schools, leading to uneven distribution of initial selections. Second, centralized enrollment affects only how students are sorted on entry, but does not control who stays or leaves a given school.

Perhaps more importantly, however, it may be the case that some charter schools are simply not cut out to best serve some students (as with the district’s own Magnet schools). It would likely be a bad policy choice to create a centralized enrollment system that requires schools to serve children they are ill-equipped to serve.

The second issue requiring consideration is whether the staffing and expenditure structure of charter schools is sustainable and/or efficient. As I’ve shown in my previous report, charter schools are a relative break-even on state achievement growth outcomes, given their resource levels and student characteristics.[1] But, the current staffing expenditure levels (which are merely average, not low) of charters in Newark depend on maintaining a very inexperienced workforce. Again, current novice teacher concentrations may be a function of recent enrollment growth.

As growth slows, these schools will either have to a) shed more experienced teachers to maintain their low-expense staff, b) lower their wages, potentially compromising quality of recruits, c) reduce staffing ratios, potentially compromising program quality or d) increase their spending levels. If charter operators choose “a” above – relying on high attrition, it remains questionable whether the supply of new teachers, even from alternative pathways, would be sufficient to maintain the present model at much larger scale.

[1] https://njedpolicy.files.wordpress.com/2014/10/research-note-on-productive-efficiency.pdf

Follow

Get every new post delivered to your Inbox.

Join 6,457 other followers