Empirical Critique of “One Newark”: First Year Update

Testimony before the Joint Committee on the Public Schools

PDF Version (Introduction below): Weber Testimony

New Jersey Legislature

Mark Weber

INTRODUCTION

Good morning. My name is Mark Weber; I am a New Jersey public school teacher, a public school parent, a member of the New Jersey Education Association, and a doctoral student in Education Theory, Organization, and Policy at Rutgers University’s Graduate School of Education.

Last year, I was honored to testify before this committee regarding research I and others had conducted on One Newark, the school reorganization plan for the Newark Public Schools. Dr. Bruce Baker, my advisor at Rutgers and one of the nation’s foremost experts on school finance and policy, joined me in writing three briefs in 2014 questioning the premises of One Newark. Dr. Joseph Oluwole, a professor of education law at Montclair State University, provided a legal analysis of the plan in our second brief.

I would like to state for the record that neither myself, Dr. Baker, nor Dr. Oluwole received any compensation for our efforts, and our conclusions are solely our own and do not reflect the views of our employers or any other organization.

Our research a year ago led us to conclude that there was little reason to believe One Newark would lead to better educational outcomes for students. There was little empirical evidence to support the contention that closing or reconstituting schools under One Newark’s “Renew School” plan would improve student performance. There was little reason to believe converting district schools into charter schools would help students enrolled in the Newark Public Schools (NPS). And we were concerned that the plan would have a racially disparate impact on both staff and students.

In the year since my testimony, we have seen a great public outcry against One Newark. We’ve also heard repeated claims made by State Superintendent Cami Anderson and her staff that Newark’s schools have improved under her leadership, and that One Newark will improve that city’s system of schools.

To be clear: it is far too early to make any claims, pro or con, about the effect of One Newark on academic outcomes; the plan was only implemented this past fall. Nevertheless, after an additional year of research and analysis, it remains my conclusion that there is no evidence One Newark will improve student outcomes.

Further, after having studied the effects of “renewal” on the eight schools selected by State Superintendent Anderson for interventions in 2012, it is my conclusion that the evidence suggests the reforms she and her staff have implemented have not only failed to improve student achievement in Newark; they have had a racially disparate impact on the NPS certificated teaching and support staff.

Before I begin, I’d like to make a point that will be reiterated throughout my testimony: my analysis and the analyses of others actually raise more questions than they answer. But it shouldn’t fall to independent researchers such as me or the scholars I work with to provide this committee or other stakeholders with actionable information about Newark’s schools.

Certainly, we as scholars stand ready to provide assistance and technical advice; but the organization that should be testing the claims of NPS and State Superintendent Anderson is the New Jersey Department Of Education. The students and families of Newark deserve nothing less than a robust set of checks and balances to ensure that their schools are being properly managed.

One Newark can be thought of as containing four components: the expansion of charter schools; a “renewal” program for schools deemed to be underperforming; a system of consumer “choice,” where families select schools from a menu of public and charter options; and continuing state control of the district.

This last component is clearly a necessary precondition for the first three. Given the community outcry against State Superintendent Anderson and One Newark, it’s safe to say that none of the other three components would have been implemented were it not for continuing state control.

The critical questions I ask about these components are simple: do they work, are there unintended consequences from their implementation, and is One Newark being properly monitored and evaluated? Let me start by addressing the expansion of charter schools in Newark.

Research Note: Resource Equity & Student Sorting Across Newark District & Charter Schools

Bruce D. Baker

PDF: BBaker.NJCharters.2015

Executive Summary

In this brief, I present preliminary findings that are part of a larger, national analysis of newly released federal data, a primary objective of which is to evaluate the extent to which those data yield findings consistent with findings arrived at using state level data sources. In this brief, I specifically explore variations in student characteristics and resources across schools in Newark, NJ.

I begin by reflecting on my most recent policy brief on charter and district school performance outcomes – growth percentile data from 2012 and 2013 – noting that on average, Newark Charter schools remain relatively average in student achievement gains given their student populations. But as noted on previous occasions, Newark Charter school student populations are anything but average.

Next, I use longitudinal data from the NCES Common Core of Data, public school universe (the source of underlying demographic data for the newly released federal data) characterizing changes in Newark Charter market share (share of children served in Charter Schools) and the share of low income children served in Newark Charter schools.

Next, I explore what the newly released (albeit already dated) federal data say about Newark Charter school demographics, compared to district schools serving similar grade distributions.

Next, I explore resource distributions and teacher characteristics across Newark schools, charter and district. The question at hand here is whether across district and charter schools, those schools serving needier and more costly student populations also have more (or fewer) resources with which to serve those children. Further, whether among schools serving similar student populations, resource levels are similar.

Forthcoming analyses of charter schools in New York City found that those schools tended to serve less needy populations (than district schools) and were able to do so with substantially more resources that district schools serving similar populations. Because the share of children in the district served by charters remained small, their disruptive effect on equity remained small. By contrast, in Houston, charter schools both served more comparable student populations, and did so, on average, with more comparable resource levels, resulting in less disruption of equity. In each case, the more interesting story, however, was the extent of variation among charter schools, both in students served and in resource levels.

Here, I explore similar questions in the City of Newark, first with the newly released Federal data and then with the most recent four years of available state data (2010 to 2014).

Conclusions & Policy Implications

To summarize:

  • Recently released federal data, confirmed by more recent state data indicates that student population differences between Newark district and charter schools persist.
    • Newark charter schools continue to serve smaller shares of children qualified for free lunch, children with limited English language proficiency and children with disabilities, than do district schools serving similar grade ranges.
    • While charter school market share has remained relatively small (through 2013), the effect of charters underserving lower income students on district school enrollments has remained relatively modest.
  • Charter school total staffing expenditures, either as reported in federal data or as compiled from state data appear to fall in line with student needs in charter schools.
    • Charter schools serve less needy populations and do so with relatively low total salary expense per pupil.
    • But, there exists significant variation in resources among charter schools, with some outspending otherwise similar district schools and others significantly underspending otherwise similar district schools.
  • Charter school wage competitiveness varies widely, with some charters paying substantially more than district schools for teachers of specific experience and degree levels. But these wages do not, as of yet, substantially influence total staffing costs.
  • Charter schools have very high concentrations of 1st and 2nd year teachers, which lowers their total staffing expenditure per pupil but only to the point where those staffing expenditures are in line with expectations (not lower, as one might expect for schools with so many novice teachers).

Finally, comparisons between the newly released Federal data collection and updated state data sources appear both relatively stable over time and relatively consistent across sources even as the charter sector rapidly grows and evolves and as the district continuously morphs.

Two issues require consideration by policymakers and local officials if reliance on charter schooling and expansion of charter schooling are to play a significant role in the future of schooling in Newark. The first is the active management of the potential deleterious effects of student sorting on district schools – that is, as market share increases and the tendency remains for charters to enroll (or keep) fewer of the lowest income children, district schools may be more adversely affected.

An appropriately designed centralized enrollment system can partially mitigate these issues. But (at least) two factors can offset the potential benefits of such a system. First, individual choices of differently motivated and differently informed parents influence who signs up to attend what schools, leading to uneven distribution of initial selections. Second, centralized enrollment affects only how students are sorted on entry, but does not control who stays or leaves a given school.

Perhaps more importantly, however, it may be the case that some charter schools are simply not cut out to best serve some students (as with the district’s own Magnet schools). It would likely be a bad policy choice to create a centralized enrollment system that requires schools to serve children they are ill-equipped to serve.

The second issue requiring consideration is whether the staffing and expenditure structure of charter schools is sustainable and/or efficient. As I’ve shown in my previous report, charter schools are a relative break-even on state achievement growth outcomes, given their resource levels and student characteristics.[1] But, the current staffing expenditure levels (which are merely average, not low) of charters in Newark depend on maintaining a very inexperienced workforce. Again, current novice teacher concentrations may be a function of recent enrollment growth.

As growth slows, these schools will either have to a) shed more experienced teachers to maintain their low-expense staff, b) lower their wages, potentially compromising quality of recruits, c) reduce staffing ratios, potentially compromising program quality or d) increase their spending levels. If charter operators choose “a” above – relying on high attrition, it remains questionable whether the supply of new teachers, even from alternative pathways, would be sufficient to maintain the present model at much larger scale.

[1] https://njedpolicy.files.wordpress.com/2014/10/research-note-on-productive-efficiency.pdf

New Jersey Charter School Law, Race and Equal Protection

New Jersey Charter School Law, Race and Equal Protection

PDF: CharterSchoolPolicyBrief(RaceandEqualProtection)

Joseph O. Oluwole, Montclair State University

Introduction

This brief addresses the constitutional import of the New Jersey Charter School Program Act’s race-conscious student enrollment mandates. The New Jersey Charter School Program Act provides that charter schools must not discriminate on any “basis that would be illegal if used by a school district.”[1] It also provides that charter schools must “be open to all students on a space available basis.”[2] If the school gets more applicants than the spaces available, the charter school must use a random selection process to determine admittees.[3] Despite these non-discrimination and space-available admission requirements, the law allows schools to exercise preferential treatment in admissions: a charter school can restrict admission to specific grade levels or to the school’s subject areas of concentration.[4] These subject areas include the arts, sciences or mathematics.[5]

Additionally, despite the non-discrimination requirement, the Charter School Program Act mandates certain discriminatory practices which are not readily subject to legal challenges. For instance, the Charter School Program Act dictates that students in the resident district of the charter school must be given enrollment preference over other prospective students.[6] If more resident students apply than space available, these students will then be subjected to the random-selection process as well.[7]

Another preference in the law deals with the enrollment of continuing students. The Charter School Program Act provides that, as long as a charter school has the requisite grade level at the school, it must give enrollment preference to students who attended the school in the immediate prior school year.[8]

The Charter School Program Act also authorizes but does not require enrollment discrimination on the basis of family ties. In particular, the law states that “[a] charter school may give enrollment priority to a sibling of a student enrolled in the charter school.”[9] Charter schools are also authorized but not required to craft and incorporate into their charter, criteria that is reasonable for evaluating prospective students.[10] However, such criteria must not discriminate against prospective students because of their intellect, handicap status, athleticism, English language proficiency, aptitude measures, achievement measures, or other ground that would be legally invalid for a school district to use.[11]

Notwithstanding the non-discrimination and space-available admission requirements, the Charter School Program Act requires charter schools to incorporate race in their enrollment decisions in order to ensure that a cross section of the community is represented in the school’s enrollment. Specifically, the law provides:

The admission policy of the charter school shall, to the maximum extent practicable, seek the enrollment of a cross section of the community’s school age population including racial and academic factors.[12]

This provision is important because of the tendency of charter schools to become one-race charter schools. There are also many charter schools in heavily-minority districts furthering segregation. This makes it important to ensure that diversity rather than segregation persists in those districts.

This brief examines the constitutionality of this and other race-conscious mandates of the New Jersey Charter School Program Act under the United States Equal Protection Clause. It also analyzes the mandates under the New Jersey Equal Protection constitutional provision.

Race-Conscious Mandates in the New Jersey Charter School Program Act

As noted earlier, even though the Charter School Program Act prohibits discrimination on any ground that would be legally invalid if employed by school districts, it also provides that:

The admission policy of the charter school shall, to the maximum extent practicable, seek the enrollment of a cross section of the community’s school age population including racial and academic factors.[13]

The language of this mandate suggests that it is a race-conscious provision rather than a racial quota. It is evident that it is not a quota since it does not require reservation of specific number of seats for a particular race(s).[14]

There are other race-conscious mandates in the state laws governing charter schools. For instance, the New Jersey Department of Education regulations provide that:

Prior to the granting of the charter, the Commissioner shall assess the student composition of a charter school and the segregative effect that the loss of the students may have on its district of residence.[15]

Further, the regulations state that:

On an annual basis, the Commissioner shall assess the student composition of a charter school and the segregative effect that the loss of the students may have on its district of residence.[16]

If the Commissioner finds that the charter school has segregative effect, the Commissioner can impose a remedy.[17] According to the Superior Court of New Jersey Appellate Division, if a charter school has already been approved, a school district must wait until the school actually has a segregative effect on the district before seeking judicial or administrative remedial action for the segregative effect.[18]

The New Jersey Department of Education regulations require that charter schools seeking to be regional schools include in their application a “plan to ensure the enrollment of a cross section of the school-age population of the region of residence, including racial and academic factors.”[19]

The regulations empower the commissioner to deny or grant charter renewals based on the “annual assessments of student composition of the charter school.”[20] The regulations also provide that:

No later than January 15 of subsequent school years [after the initial recruitment period for a charter school], a charter school shall submit to the Commissioner the number of students by grade level, gender and race/ethnicity from each district selected for enrollment from its initial recruitment period for the following school year.[21]

What is evident from the above provisions is that New Jersey places a premium on the racial composition of its charter schools.

The United States Equal Protection Clause and New Jersey’s Charter School Laws’ Race-Consciousness

The Equal Protection Clause of the Fourteenth Amendment of the United States Constitution provides:

No State shall … deny to any person within its jurisdiction the equal protection of the laws.[22]

The Equal Protection Clause is designed to protect people from discrimination on the basis of various characteristics including race.

The scope of the Equal Protection Clause with respect to voluntary race-conscious mandates was muddled for many years because the United States remained silent on the constitutionality of such mandates. Unlike in desegregation cases which seek to remedy legally-sanctioned segregation,[23] race-conscious cases involve voluntary efforts to promote diversity. In 2007, in Parents Involved in Community Schools v. Seattle School District No. 1,[24] the United Supreme Court finally ruled on the constitutionality of race-conscious measures. Even though the Parents Involved case involved race-conscious measures at public schools, the reasoning in the case applies to New Jersey charter schools since they are public schools.[25]

New Jersey’s Charter School Program Act appears to send mixed messages to school districts. On one hand, the law declares that charter schools must not discriminate on any basis that would be illegal for school districts;[26] on the other hand, the law requires charter schools to use race-conscious measures.[27] In essence, charter schools have to ensure that they only use race-conscious measures in the same way that school districts can constitutionally use them. Consequently, if Parents Involved authorizes public schools to use race-conscious measures, then charter schools can.

In Parents Involved, the Supreme Court iterated that “when the government distributes burdens or benefits on the basis of individual racial classifications, that action is reviewed under strict scrutiny.”[28] Student admission on the basis of race is a distribution of benefits under this rule. Therefore, the New Jersey mandate to enroll a cross section in a race-conscious manner is subject to strict scrutiny under the United States Supreme Court Equal Protection Clause jurisprudence.

Since the various charter school provisions highlighted above also facially involve decisions based on race, those provisions will also attract strict scrutiny. According to the Supreme Court, “racial classifications are simply too pernicious to permit any but the most exact connection between justification and classification.”[29] Unfortunately, the Supreme Court requires both invidious and beneficial uses of race to be reviewed under strict scrutiny.[30] Justice Kennedy pointed out that:

Absent searching judicial inquiry into the justification for such race-based measures, there is simply no way of determining what classifications are ‘benign’ or ‘remedial’ and what classifications are in fact motivated by illegitimate notions of racial inferiority or simple racial politics.[31]

The strict scrutiny standard demands that the means used in a racial classification must be narrowly tailored to a compelling interest.[32] In other words, the government must establish two things in order to satisfy strict scrutiny: (i) a compelling interest justifying the racial classification; and (ii) the means used in pursuing the compelling interest is narrowly tailored to achieve the compelling interest.

New Jersey’s compelling interest for its race-conscious charter school mandates is diversity. The state seeks to ensure that students learn in diverse educational settings. Five Supreme Court Justices in Parents Involved recognized a compelling interest in diversity at the K-12 level.[33] Consequently, the race-conscious mandates in New Jersey’s Charter School Program Act as well in the New Jersey Department of Education’s regulations would survive the compelling interest prong of strict scrutiny.

The Supreme Court has warned against use of race-conscious plans with sweeping mandates to remedy past general discrimination in society.[34] Therefore, neither the state nor charter schools can rely on an interest in remedying societal discrimination as justification for race-conscious measures. Instead, the state and charter schools should singularly focus on the compelling interest in diversity.

The challenge for the compelling interest lies in how it is implemented under the charter school law mandates. If the means chosen does not satisfy the narrowly-tailored requirement pursuant to Parents Involved, it will not survive Equal Protection Clause review.

While New Jersey charter schools must incorporate race-consciousness into their enrollment decisions in order to ensure a community cross-section enrollment, the language of New Jersey’s race-conscious mandate is a bit amorphous. For instance, the main race-conscious mandate uses the terms “practicable” and “seek”[35] which qualify the obligation such as to not make it necessarily a quota which the Supreme Court abhors. The mandate also states that charter schools must consider a variety of factors “including racial and academic factors” in seeking to enroll a cross section of the community.[36] In essence, the law uses a race-plus approach. This is important as it aligns with what the Supreme Court has approved. As Justice Kennedy stated:

In the administration of public schools by the state and local authorities it is permissible to consider the racial makeup of schools and to adopt general policies to encourage a diverse student body, one aspect of which is its racial composition.[37]

New Jersey leaves it up to each charter school to come up with a race-conscious plan that satisfies the state’s student-enrollment mandate. Therefore, individual charter schools need to be fully aware of what they can (or cannot) do in targeting race-conscious enrollment of a cross-section of the community. The Parents Involved narrowly-tailored analysis reveals some guidance.

A majority of the Supreme Court[38] in Parents Involved ruled that, in order to satisfy narrow tailoring, the means chosen must have more than a minimal effect in achieving the compelling interest.[39] Consequently, charter schools must ensure that any race-conscious plan they implement can be shown in court to have more than a minimal impact on diversity. The Commissioner must also ensure that any remedy granted in pursuit of diversity in addressing segregative effect, as required in the New Jersey Department of Education regulations, will have more than a minimal impact.

Expounding on the import of the “minimal impact” principle, the Supreme Court stated that:

While we do not suggest that greater use of race would be preferable, the minimal impact of the districts’ racial classifications on school enrollment casts doubt on the necessity of using racial classifications.[40]

According to the Supreme Court majority, if a race-conscious plan has only marginal impact, it implies that there are other means that would be more effective than the means used by the school.[41]

In order to satisfy narrow tailoring, charter schools must also document that they evaluated workable non-race-conscious means for attaining diversity before settling on the race-conscious means.[42] Such evaluation of race-neutral means must be done seriously and in good faith.[43] The districts in Parents Involved faltered on this requirement because they could present no evidence that they engaged in good faith and serious evaluation of race-neutral means before adopting race-conscious measures.[44]

New Jersey would be wise to include in its Charter School Program Act the requirement of documented consideration of race-neutral means before adoption of race-conscious means; and before race-conscious Commissioner decisions. This might increase the chances that the state’s race-conscious mandate would pass narrow-tailoring muster under the requirement of serious good-faith evaluation. Charter schools are not, however, required to evaluate every existing race-neutral alternative.[45] As long as they document that they seriously considered some race-neutral alternative before concluding that race-consciousness was the effective approach to diversity, they should be fine. This requirement would apply to both the race-conscious enrollment decisions as well as the Commissioner’s remedy of the segregative effect of a charter school.

Charter schools also must be very clear in documented evidence about the need for the race-conscious plan; as must the Commissioner in ordering remedial action to address segregative effects. This is because the plurality of the Supreme Court[46] insisted that:

If the need for the racial classifications embraced by the school districts is unclear, even on the districts’ own terms, the costs are undeniable.[47]

Charter schools should build a logical stopping point into their race-conscious plans.[48] This is essential as several Justices expressed concern about an open-ended race-conscious plan that would effectively cause the “definition of racial diversity” to fluctuate with demographic changes.[49]

Justice Kennedy was the pivotal vote in the 5-4 decision so his opinion about narrow tailoring in the case is defining. According to Justice Kennedy, narrow tailoring requires “inquiry into less restrictive alternatives.”[50] This inquiry “requires in many cases a thorough understanding of how a plan works.”[51] Charter schools must document the details of how they use race in making race-conscious enrollment decisions about individual prospective students.[52] Justice Kennedy considers this a “threshold mandate” of narrow tailoring.[53] The Commissioner should similarly document for actions to remedy segregative effect of a charter school.

Justice Kennedy is highly particular about the need for detail and precision in race-conscious plans. A plan that simply references the statutory requirement that charter schools enroll a cross section of the community using race-consciousness will not suffice.[54] Neither would a plan that merely includes sweeping language, without specific guidelines;[55] nor would a plan that merely encourages cooperative efforts to achieve the race-conscious cross-section enrollment.[56] A narrowly-tailored plan cannot be “broad and imprecise” in describing “how and when” racial classifications are used.[57] The plan must be very clear about each of the following: (i) who makes the race-conscious decisions about each student;[58] (ii) how the school ensures accountability and monitoring of the decisionmaking process;[59] (iii) the specific situations in which enrollment decisions will be made based on race; [60] (iv) the specific situations in which enrollment decisions will not be race-based;[61] and (v) how the school will decide which of two students similarly situated should be enrolled based on race.[62]

Justice Kennedy noted that, the more complex a race-conscious plan is, the more susceptible it is to internal inconsistencies and ambiguities.[63] This is an additional reason for charter schools to ensure that their race-conscious plans are comprehensively precise. As Justice Kennedy emphasized, “[w]hen a court subjects governmental action to strict scrutiny, it cannot construe ambiguities in favor of the State.”[64]

In order to be deemed narrowly tailored, the charter school must use racial categories reflective of the diversity of races in the resident district (or regional district for a regional charter school).[65] Justice Kennedy opined that the use of the racial categories white versus non-white would not suffice if there is a diversity of races in the district. [66] If the school chooses to use such binary categories despite the diversity of races in the district, it must clearly justify why the binary categories (as opposed to the categories reflective of the diversity of races) provide the best means of achieving the compelling interest in diversity.[67] This requirement was evident in Justice Kennedy’s conclusion regarding one of the districts in the Parents Involved case:

The district, nevertheless, has failed to make an adequate showing in at least one respect. It has failed to explain why, in a district composed of a diversity of races, with fewer than half of the students classified as ‘white,’ it has employed the crude racial categories of ‘white’ and ‘non-white’ as the basis for its assignment decisions. … Far from being narrowly tailored to its purposes, this system threatens to defeat its own ends, and the school district has provided no convincing explanation for its design. [68]

Justice Kennedy endorsed certain race-conscious plans which charter schools should consider as part of their comprehensive approach to achieving diversity. These plans include deliberative choice of location for the charter school; dedicating resources to special programs that will attract a diverse student population; targeted recruitment outreach to prospective students and faculty.[69] Justice Kennedy indicated that these plans would satisfy the strict scrutiny standard of review.[70]

Charter schools as well as the state are also authorized to track student statistics based on race; including such data such as performances and enrollment by race.[71] This supports the conclusion that the New Jersey Department of Education regulations identified above which call for tracking data by and for the Commissioner, while race-conscious, satisfy strict scrutiny.

Charter schools’ race-conscious plans must not individually type students by race.[72] In this spirit, Justice Kennedy declared that:

If school authorities are concerned that the student-body compositions of certain schools interfere with the objective of offering an equal educational opportunity to all of their students, they are free to devise race-conscious measures to address the problem in a general way and without treating each student in different fashion solely on the basis of a systematic, individual typing by race.[73]

If the charter school uses individual typing by race, it must only be as a very last recourse after it is clear that no other viable option exists.[74]

Relative to the plurality and Justice Kennedy, the dissenting Justices were more receptive to giving schools discretion with respect to the means used to achieve the compelling interest in diversity.[75] Writing for the dissent, Justice Breyer noted that:

School authorities are traditionally charged with broad power to formulate and implement educational policy and might well conclude, for example, that in order to prepare students to live in a pluralistic society each school should have a prescribed ratio of Negro to white students reflecting the proportion for the district as a whole. To do this as an educational policy is within the broad discretionary powers of school authorities.[76]

Given that the dissent embraces a race plan akin to a quota, it is evident that the New Jersey Charter School Program Act’s non-quota race-conscious mandate would a fortiori readily pass muster under this perspective. Besides, the dissent would apply a standard less stringent than strict scrutiny to review race-conscious plans designed to include (as opposed to exclude) minorities in equal education opportunity.[77] This further buttresses the conclusion that the liberal-leaning Justices would support New Jersey’s race-conscious mandate.

Even if strict scrutiny is applied, a race-conscious plan would satisfy these Justices if certain factors are satisfied: (i) The use of race is a single part of the plan that is principally dependent on non-racial factors;[78] (ii) the use of race is not burdensome to a substantial number of people;[79] (iii) the plan reflects “the results of local experience and community consultation”;[80] (iv) the plan is “the product of a process that has sought to enhance student choice.”[81] New Jersey’s charter school race-conscious mandate satisfies at least two of the factors. As noted earlier, it calls for incorporation of a variety of factors in race-conscious plans; and does not call for the centrality of race. Additionally, given that a driving point for charter schools is choice, the fourth factor should easily be satisfied. As for the third factor, the state and the individual charter school would have to ensure that the race-conscious plan is developed after consulting the local community and local experiences.

In order to satisfy the second factor, the state and school must ensure that the race-conscious plan is not so drastic that it is disruptive to a significant number of people. The Justices also indicated that it would help a race-conscious plan to show that other plans that use race less explicitly would not effectuate the compelling interest in diversity.[82]

New Jersey’s Equal Protection Constitutional Provision and the Charter School Race-Conscious Mandate

New Jersey’s Equal Protection constitutional provision states:

All persons are by nature free and independent, and have certain natural and unalienable rights, among which are those of enjoying and defending life and liberty, of acquiring, possessing, and protecting property, and of pursuing and obtaining safety and happiness.[83]

Unlike the federal constitution which relies on a three-tier standard,[84] New Jersey analyzes its constitutional provision using a balancing test which considers the following factors: (i) the nature of the right impacted by the government action;[85] (ii) the extent to which the government action infringes the right; [86] and (iii) the public necessity for the action.[87] The nature of the right implicated in New Jersey’s race-conscious mandate is the fundamental right of every student to have an opportunity to pursue the same education quality.[88]

The argument goes that, if race is considered in enrollment decisions or in the Commissioner’s remedy of the segregative effect of a charter school, then those denied the opportunity to attend a charter school are being denied the fundamental right to quality education as others similarly situated. While it is true that this right is fundamental, the argument fails because the race-conscious plan only minimally affects the right to a quality education. After all, as Bruce D. Baker found:

Charter schools do not vary substantively on measures of student growth from other schools in the same county or city when controlling for student characteristics and resources.[89]

Given this, New Jersey’s race-conscious mandate should satisfy the first two factors of the balancing test.

The need for diversity of schools provides the linchpin for the third factor. Certainly, one-race charter schools, and charter schools that leave district schools highly segregated, are not in the public interest. Such schools undermine the promise of Brown v. Board of Education for integrated education.[90] They are also reminiscent of education in the era of segregated schooling, albeit that that era mostly featured de jure segregation. The public necessity for desegregation in education as well as the academic benefits of integrated education should be viewed as important reasons for ensuring the enrollment of a cross section of the community that accounts for race, among other factors. Similarly, the Commissioner’s remedy of segregative impact of charter schools on district schools is justifiable on the basis of academic benefits of integrated education.

One-race charters might also violate another provision of the New Jersey Constitution which states:

No person shall be denied the enjoyment of any civil or military right, nor be discriminated against in the exercise of any civil or military right, nor be segregated in the militia or in the public schools, because of religious principles, race, color, ancestry or national origin.[91]

This constitutional provision makes it even more critical for the Commissioner to remedy the segregative effects of charter schools. It makes it even more imperative to enforce the Charter School Program Act requirement that charter schools enroll a student population reflective of a cross section of the community. For those denied enrollment on the basis of race, however, this constitutional provision could also form the basis for a challenge to race-conscious plans since it prohibits denial of a right on the basis of race. To help minimize their legal exposure, charter schools would be wise to ensure that they consider a composite of factors as required in the Charter School Program Act mandate; otherwise, they risk more provable racial discrimination challenges under this constitutional provision.

Conclusion

Charter schools implementing New Jersey’s charter school race-conscious mandates should survive Equal Protection challenges under the federal constitution if they comply with the Supreme Court’s dictates in the Parents Involved decision. Charter schools must ensure that they do not use race-conscious plans that principally rely on race. Instead a race-conscious plan that considers race as a plus factor should be used.

Furthermore, New Jersey courts need to recognize a public necessity, under the New Jersey constitution, to prevent one-race charters. Additionally, New Jersey courts as well as the Commissioner need to enforce the requirement that charter schools enroll a cross section of the community that is reflective of race. Finally, the state must encourage the establishment of more regional charter schools that span both suburban and urban districts so that the regional charters can effectively and appropriately reflect a cross section of a community that would ensure diversity in the schools.

[1] N.J. Stat. Ann. § 18A:36A-7 (2014).

[2] Id.

[3] N.J. Stat. Ann. § 18A:36A-8(a) (2014).

[4] N.J. Stat. Ann. § 18A:36A-7 (2014).

[5] Id.

[6] N.J. Stat. Ann. § 18A:36A-8(a) (2014); N.J. Admin. Code tit. 6A, 6A:11–2.10(b) (2014).

[7] Id.

[8] N.J. Stat. Ann. § 18A:36A-8(b) (2014); N.J. Admin. Code tit. 6A, 6A:11–2.10(b) (2014).

[9] N.J. Stat. Ann. § 18A:36A-8(c) (2014); N.J. Admin. Code tit. 6A, 6A:11–2.10(b) (2014).

[10] N.J. Stat. Ann. § 18A:36A-7 (2014).

[11] Id.; N.J. Admin. Code tit. 6A, 6A:11–2.10(b) (2014).

[12] N.J. Stat. Ann. § 18A:36A-8(e) (2014).

[13] N.J. Stat. Ann. § 18A:36A-8(e) (2014).

[14] “Properly understood, a ‘quota’ is a program in which a certain fixed number or proportion of opportunities are ‘reserved exclusively for certain minority groups’” (Parents Involved in Community Schools v. Seattle School District No. 1, 551 U.S. 701, 846 (2007) (citing Richmond v. J.A. Croson Co., 488 U.S. 469, 496 (1989)).

[15] N.J. Admin. Code tit. 6A, 6A:11–2.1(j) (2014).

[16] N.J. Admin. Code tit. 6A, 6A:11–2.2(c) (2014).

[17] In re Charter School Appeal of Greater Brunswick Charter School, 753 A.2d 1155, 1164-65 (N.J.Super.A.D. 1999); N.J.S.A. 18A:6-9 (2014).

[18] In re Charter School Appeal of Greater Brunswick Charter School, 753 A.2d at 1164-65 (N.J.Super.A.D. 1999).

[19] N.J. Admin. Code tit. 6A, 6A:11–2.1(b)(4)(ii) (2014).

[20] N.J. Admin. Code tit. 6A, 6A:11–2.3(b)(8) (2014).

[21] N.J. Admin. Code tit. 6A, 6A:11–4.4(a) (2014).

[22] U.S. Const. amend. XIV.

[23] Brown v. Board of Education of Topeka, 347 U.S. 483 (1954); Griffin v. County School Board, 377 U.S. 218 (1964); Swann v. Charlotte–Mecklenburg Board of Education, 402 U.S. 1 (1971); Keyes v. School District No. 1, Denver, Colorado, 413 U.S. 189 (1973).

[24] 551 U.S. 701 (2007).

[25] N.J.S.A. 18A:36A-2 (“The Legislature finds and declares that the establishment of charter schools as part of this State’s program of public education”); N.J.S.A. 18A:36A-3(a) (“A charter school shall be a public school operated under a charter granted by the commissioner”).

[26] N.J. Stat. Ann. § 18A:36A-7 (2014).

[27] N.J. Stat. Ann. § 18A:36A-8(e) (2014).

[28] Parents Involved, 551 U.S. at 720 (2007) (citing Johnson v. California, 543 U.S. 499, 505-06 (2005)).

[29] Parents Involved, 551 U.S. at 720 (2007) (citing Gratz v. Bollinger, 539 U.S. 244, 270 (2003)).

[30] Regents of University of California v. Bakke, 438 U.S. 265 (1978); Johnson v. California, 543 U.S. 499, 505 (2005).

[31] Parents Involved, 551 U.S. at 783 (2007) (quoting Richmond v. J.A. Croson Co., 488 U.S. 469, 493 (1989).

[32] Adarand Constructors, Inc. v. Peña, 515 U.S. 200 (1995); McConnell v. FEC, 540 U.S. 93, 205 (2003); Johnson v. California, 543 U.S. 499, 505 (2005).

[33] The Justices were the four-liberal leaning Breyer, Ginsburg, Souter and Stevens as well as the unpredictable Justice Kennedy (unpredictable because he sometimes takes conservative positions when not expected while taking liberal positions when not expected).

[34] Parents Involved, 551 U.S. at 731-32, 794 (2007).

[35] N.J. Stat. Ann. § 18A:36A-8(e) (2014).

[36] Id.

[37] Parents Involved, 551 U.S. at 788 (2007) (emphasis added).

[38] This majority was comprised of Justices Thomas, Scalia, Kennedy, Alito and Chief Justice Roberts.

[39] Parents Involved, 551 U.S. at 733 (2007).

[40] Id. at 734.

[41] Id.

[42] Id. at 735.

[43] Id.

[44] Id.

[45] Grutter v. Bollinger, 539 U.S. 306, 339 (2003).

[46] The plurality was comprised of Chief Justice Roberts and Justices Alito, Scalia and Thomas.

[47] Parents Involved, 551 U.S. at 745 (2007).

[48] Id. at 731.

[49] Id.

[50] Id. at 784.

[51] Id.

[52] Id.

[53] Id.

[54] Parents Involved, 551 U.S. at 785 (2007).

[55] Id.

[56] Id.

[57] Id. at 784-85.

[58] Id. at 785.

[59] Id.

[60] Parents Involved, 551 U.S. at 785.

[61] Id.

[62] Id.

[63] Id.

[64] Id. at 786.

[65] Id.

[66] Id. at 786-87.

[67] Parents Involved, 551 U.S. at 787 (2007).

[68] Id. at 786-87.

[69] Id. at 789.

[70] Id.

[71] Id.

[72] Parents Involved, 551 U.S. at 789 (2007).

[73] Id. at 788-89.

[74] Id. at 790.

[75] Id. at 823.

[76] Id. at 804-05 (quoting Swann v. Charlotte–Mecklenburg Board of Education, 402 U.S. 1, 16 (1971)).

[77] Parents Involved, 551 U.S. at 837 (2007).

[78] Id. at 846.

[79] Id. at 847.

[80] Id. at 848.

[81] Id.

[82] Id.

[83] N.J. Const. art. I, ¶ 1.

[84] The federal Equal Protection Clause is reviewed using the following three-tier standard: rational basis; intermediate scrutiny; and strict scrutiny.

[85] In re Grant of Charter School Application of Englewood on Palisades Charter School, 727 A.2d 15, 48 (N.J.Super.A.D. 1999).

[86] Id.

[87] Id.

[88] Id. at 48.

[89] Baker, D. (2014), Research Note: On Teacher Effect vs. Other Stuff in New Jersey’s Growth Percentiles, available at https://njedpolicy.files.wordpress.com/2014/06/bbaker-sgps_and_otherstuff2.pdf.

[90] 347 U.S. 483 (1954).

[91] N.J. Const. art. I, ¶ 5.

Research Note: On Student Growth & the Productivity of New Jersey Charter Schools

Bruce D. Baker, Rutgers University, Graduate School of Education

October 31, 2014

PDF: Research Note on Productive Efficiency

In June of 2014, I wrote a brief in which I evaluated New Jersey’s school growth percentile measures to determine whether factors outside the control of local schools or districts are significantly predictive of variation in those growth percentile measures.[1] I found that this was indeed the case. Specifically, I found:

Student Population Characteristics

  1. % free lunch is significantly, negatively associated with growth percentiles for both subjects and both years. That is, schools with higher shares of low income children have significantly lower growth percentiles;
  2. When controlling for low income concentrations, schools with higher shares of English language learners have higher growth percentiles on both tests in both years;
  3. Schools with larger shares of children already at or above proficiency tend to show greater gains on both tests in both years;

School Resources

  1. Schools with more competitive teacher salaries (at constant degree and experience) have higher growth percentiles on both tests in both years.
  2. Schools with more full time classroom teachers per pupil have higher growth percentiles on both tests in both years.

Other

  1. Charter schools have neither higher nor lower growth percentiles than otherwise similar schools in the same county.

 

On the one hand, these findings raise some serious questions about the usefulness of the state’s growth percentile measures for characterizing school effectiveness. At the very least, if one wishes to compare the growth percentiles of one school to another, one should use a statistical approach that first corrects for those factors that are a) outside of the control of local school officials and b) substantively influence growth.

While I’ve been critical of the growth percentile data produced by the state, most notably for their failure to more completely address these issues, the growth percentile measures are certainly more useful than performance level measures which are even more highly correlated with differences in demographics and other contextual variables.

Here I use the growth percentile measures as the outcomes of interest in a set of models wherein I attempt to estimate the relative efficiency of production of outcomes across New Jersey schools. Given the findings of my previous analyses, if I wish to compare school growth percentiles and make assertions about how well one school versus another achieves growth, I must account for several factors.

I must, for example, account for a) initial performance levels, b) demographic differences, c) school size and grade range differences.

Here, my goal is slightly different from the previous analysis in terms of how I characterize resources. Here, the goal is to correct for the aggregate resource inputs to each school, on the assumption that schools (or their operators) might make tradeoffs between teacher compensation, compensation structures and class sizes to achieve greater efficiency in producing student achievement gains. Lacking comprehensive school site spending data in New Jersey, I take a second best (perhaps third or fourth) approach of using the summed certified staffing salaries per pupil as a proxy for total fiscal resource inputs. Otherwise the regressions are identical to those in the previous analysis.

Table 1 shows the regression model output.

Table 1

Slide1

 

Again, we can see that these various factors explain from around 20 to nearly 40% of the variation across schools in growth percentiles.

We also see that aggregate school resources matter. Schools with higher certified staffing spending per pupil are also showing higher growth.

But we can also use these models to compare the relative performance of schools in the models. Specifically, we can evaluate the extent to which a school’s actual growth percentile is higher or lower than would be predicted, given the school’s population, resources and other characteristics. Because there are other unmeasured common pressures on schools in particular locations, including differences in the value of the dollar inputs from Newark to Camden or Atlantic City, I compare schools against others in the same county (rather than city in this case, because so many cities in New Jersey have such small numbers of schools). So, each school is compared against similar spending, similar student, and other similar characteristic schools in their county, but in a statewide model.

Now let’s take a look at performance distributions of charter and district schools for 2013 and for 2012 on math and language arts growth. We know from my previous research note that the average difference in growth between charter and district schools was “0.” But averages are uninteresting and provide little policy guidance. What’s more interesting is evaluating the variation in charter, and for that matter district school growth, corrected for the various factors above.

Figure 2 shows the statewide 2013 performance profile – the relationship between corrected language arts and corrected math growth percentiles. The two are modestly related (.49). Schools in the upper right are high on both and in the lower left are low on both. Charters, like district schools are scattered throughout the distribution.

Figure 2

Slide2

Figure 3 looks pretty much like Figure 2 with charter and district schools scattered. In both cases, it is generally true, via modest correlation (.53), that schools that were high on one assessment, tended to be higher on the other. I cannot be entirely confident whether these patterns reflect true quality differences in production of outcomes, or whether they simply represent outside factors not fully controlled for in the models.

Figure 3

Slide3

Figure 4 looks specifically at schools in the city of Newark in 2013. Again, charter and district schools are scattered, with some district schools performing quite high on both LA and Math. Higher (on both) performing charters, in terms of resource and need adjusted growth, include Discovery, Maria Varisco Rogers and Newark Educators charter, and low performing charters included University Heights and Greater Newark.

Figure 4

Slide4

TEAM academy was average on Math and slightly above average on LA. Robert Treat was average on LA and slightly below average on Math. North Star was slightly above average on both.

Patterns are similar for 2012, with Discovery being the standout, and University Heights being in positive rather than negative position. North Star again showed better than average growth on both tests, but TEAM showed slightly below average growth adjusted for resources, students, enrollment size and grade range.

Figure 5

Slide5

These final two graphs rank charter schools statewide by their performance on growth measures, given their resources, students, enrollment size and grade range. Figure 6 shows the 2013 ranking and Figure 7 shows the 2012 ranking. Both charts are sorted from lowest (average across both tests) to highest growth against expectations.

Figure 6 shows that Freedom Academy, Discovery Charter School and Camden’s promise had the greatest achievement growth given their resources, students, enrollment size and grade range and Union County TEAMS, Sussex County CS for TEC and East Orange CS had the lowest growth against expectations. In 2012, Discovery and Camden’s promise also did very well.

But other more “talked about” charters fall within or closer to the average mix of schools. Specifically, large and long running charter operators in Newark, include TEAM academy, whose performance is consistently around average (slightly above or slightly below). North Star Academy is consistently slightly to modestly above average, while Robert Treat Academy is more consistently below average on the student growth measures adjusted for resources, students, enrollment size and grade range.

Importantly, the distribution of charters around the mean is not different from the distribution of district schools around the statewide mean, as shown in Figures 2 and 3 above, and as estimated in my previous brief.[2]

Figure 6

Slide6

Figure 7

Slide7

 

Conclusions & Implications

Of course, the big question is what to make of all of this, if anything. Much has been debated in recent years about the average test scores and proficiency rates of these schools and of charter schools and district schools in comparison. That debate requires a cautious accounting for a variety of student background characteristics which substantively influence status measures of student performance.

Notably, so too do the state growth percentile measures require substantial adjustment for student characteristics. But these measures should provide some more insights into differences across schools in their achievement, most notably, whether kids under their watch are achieving normatively better or worse achievement growth on math and language arts assessments.

As I’ve opined on numerous occasions, the interesting question is not whether the charter sector on the whole or by location “outperforms” district schools, but rather, what’s going on behind the variation. Using analyses of this type, we should begin exploring in greater depth what’s going on in schools more consistently in the upper right and lower left hand corners of these distributions. Applying these methods and measures, we may find schools we hadn’t previously considered worthy of that closer look.

 

[1] https://njedpolicy.files.wordpress.com/2014/06/bbaker-sgps_and_otherstuff2.pdf

[2] https://njedpolicy.files.wordpress.com/2014/06/bbaker-sgps_and_otherstuff2.pdf

==================

Raw model output: Productivity Output

Stata code for compiling (and rolling up) resource and demographic measures: Step 1-Staffing Files | Step 2-SRC Aggregation | Step 3-School Resource Aggregation

Research Note: On Teacher Effect vs. Other Stuff in New Jersey’s Growth Percentiles

Bruce D. Baker, Rutgers University, Graduate School of Education

June 2, 2014

PDF: BBaker.SGPs_and_OtherStuff

In this research note, I estimate a series of models to evaluate variation in New Jersey’s school median growth percentile measures. These measures of student growth are intended by the New Jersey Department of Education to serve as measures of both school and teacher effectiveness. That is, the effect that teachers and schools have on marginal changes in their median student’s test scores in language arts and math from one year to the next, all else equal. But all else isn’t equal, and that matters greatly!

Variations in student test score growth estimates, generated either by value-added models or growth percentile methods, contain three distinct parts:

  1. “Teacher” effect: Variations in changes in numbers of items answered correctly that may be fairly attributed to specific teaching approaches/ strategies/ pedagogy adopted or implemented by the child’s teacher over the course of the school year;
  2. “Other stuff” effect: Variations in changes in numbers of items answered correctly that may have been influenced by some non-random factor other than the teacher, including classroom peers, after school activities, health factors, available resources (class size, texts, technology, tutoring support), room temperature on testing days, other distractions, etc;
  3. Random noise: Variations in changes in numbers of items answered correctly that are largely random, based on poorly constructed/asked items, child error in responding to questions, etc.

In theory, these first two types of variations are predictable. I often use a version of Figure 1 below when presenting on this topic.

We can pick up variation in growth across classrooms, which is likely partly attributable to the teacher and partly attributable to other stuff unique to that classroom or school. The problem is, since the classroom (or school) is the unit of comparison, we really can’t sort out what share is what?

Figure 1

Slide1

We can try to sort out the variance by adding more background measures to our model, including student individual characteristics, student group characteristics, class sizes, etc., or by constructing more intricate analyses involving teachers who switch settings. But we can never really get to a point where we can be confident that we have correctly parsed that share of variance attributable to the teacher versus that share attributable to other stuff. And the most accurate, intricate analyses can rarely be applied to any significant number of teachers.

Thankfully, to make our lives easier, the New Jersey Department of Education has chosen not to try to parse the extent to which variation in teacher or school median growth percentiles is influenced by other stuff. They rely instead on two completely unfounded, thoroughly refuted claims:

  1. By accounting for prior student performance (measuring “growth” rather than level) they have fully accounted for all student background characteristics (refuted here[1]); and
  2. Thus, any uneven distribution of growth percentiles, for example, lower growth percentiles in higher poverty schools, is a true reflection of the distribution of teacher quality (refuted here[2]).

In previous analyses I have explored predictors of New Jersey growth percentiles at the school level, including the 2012 and 2013 school reports. Among other concerns, I have found that the year over year correlation (across schools) between growth percentiles is only slightly stronger than the correlation between growth percentiles and school poverty.[3] That is, NJ SGPs tend to be about as correlated with other stuff as they are with themselves year over year. One implication of this finding is that even the year-over-year consistency is merely consistently measuring the wrong effect year over year. That is, the effect of poverty.

In the following models, I take advantage of a richer data set in which I have used a) school report card measures, b) school enrollment characteristics and c) detailed statewide staffing files and have combined those data sets into one, multi-year data set which includes outcome measures (SGPs and proficiency rates), enrollment characteristics (low income shares, ELL shares) and resource measures derived from the staffing files.

Following are what I would characterize as exploratory regression models, using 3-years of measures of student populations, resources and school features, as predictors of 2012 and 2013 school median growth percentiles.

               Resource measures include:

  •  Competitiveness of wages: a measure of how much teachers’ actual wages differ from predicted wages for all teachers in the same labor market (metro area) in the same job code, with the same total experience and degree level (estimated via regression model). This measure indicates the wage premium (>1.0) or deficit (<1.0) associated with working in a given school or district. This measure is constant across all same job code teachers across schools within a district. This measure is created using teacher level data from the fall staffing reports from 2010 through 2012.
  • Total certified teaching staff per pupil (staffing intensity): This measure is created by summing the full time certified classroom teaching staff for each school and dividing by the total enrolled pupils. This measure is created using teacher level data from the fall staffing reports from 2010 through 2012.
  • % Novice teachers with only a bachelors’ degree: This measure also focuses on classroom teachers, taking the number with fewer than 3 years of experience and only a bachelors’ degree and dividing by the total number of classroom teachers. This measure is created using teacher level data from the fall staffing reports from 2010 through 2012.

I have pointed out previously that it would be inappropriate to consider a teacher or school to be failing, or successful for that matter, simply because of the children they happen to serve. Estimate bias with respect to student population characteristics is a huge validity concern regarding the intended uses of New Jersey’s growth percentile measures.

The potential influence of resource variations presents a comparable validity concern, though the implications vary by resource measure. If we find, for example that teachers receiving a more competitive wage are showing greater gains, we might assert that the wage differential offered by a given district is leading to a more effective teacher workforce. A logical policy implication would then be to provide resources to achieve wage premiums in schools and districts serving the neediest children, and otherwise lagging most on measures of student growth.

Of course, schools having more resources for use in one way – wages – also may have other advantages. If we find that overall staffing intensity is a significant predictor of student growth, it would be unfair to assert that the growth percentiles reflect teacher quality. That is, if growth in some schools is greater than in others because of more advantageous staffing ratios. Rather than firing the teachers in the schools producing low growth, the more logical policy response would be to provide those schools the additional resources to achieve similarly advantageous staffing ratios.

With these models, I also test assumptions about variations across schools within larger and smaller geographic areas – counties and cities. This geography question is important for a variety of reasons.

New Jersey is an intensely racially and socioeconomically segregated state. Most of that segregation occurs between municipalities far more so than within municipalities. That is, it is far more likely to encounter rich and poor neighboring school districts than rich and poor schools within districts. Yet education policy in New Jersey, like elsewhere, has taken a sharp turn toward reforms which merely reshuffle students and resources among schools (charter and district) within cities, pulling back significantly from efforts to target additional resources to high need settings.

Figure 2 shows that from the early 1990s through about 2005, New Jersey placed significant emphasis on targeting additional resources to higher poverty school districts. Since that time, New Jersey’s school funding progressiveness has backslid dramatically. And these are the very resources needed for districts – especially high need districts – to provide wage differentials to recruit and retain a high quality workforce, coupled with sufficient staffing ratios to meet their students’ needs.

Figure 2

Slide2

Findings

Table 1 shows the estimates from the first set of regression models which identify predictors of cross school and district, within county variation in growth percentiles. The four separate models are of language arts and math growth percentiles (school level) from the 2012 and 2013 school report cards. These models show that:

Student Population Other Stuff

  1. % free lunch is significantly, negatively associated with growth percentiles for both subjects and both years. That is, schools with higher shares of low income children have significantly lower growth percentiles;
  2. When controlling for low income concentrations, schools with higher shares of English language learners have higher growth percentiles on both tests in both years;
  3. Schools with larger shares of children already at or above proficiency tend to show greater gains on both tests in both years;

School Resource Other Stuff

  1. Schools with more competitive teacher salaries (at constant degree and experience) have higher growth percentiles on both tests in both years.
  2. Schools with more full time classroom teachers per pupil have higher growth percentiles on both tests in both years.

Other Other Stuff

  1. Charter schools have neither higher nor lower growth percentiles than otherwise similar schools in the same county.

 

TABLE 1. Predicting within County, Cross School (cross district) Variation in New Jersey SGPs

Slide3

*p<.05, **p<.10

TABLE 2. Predicting within City Cross School (primarily within district) Variation in New Jersey SGPs

Slide4

*p<.05, **p<.10

Table 2 includes a fixed effect for city location. That is, Table 2 runs the same regressions as in Table 1, but compares schools only against others in the same city. In most cases, because of municipal/school district alignment in New Jersey, comparing within the same city means comparing within the same school district. But, using city as the unit of analysis permits comparisons of district schools with charter schools in the same city.

In Table 2 we see that student population characteristics remain the dominant predictor of growth percentile variation. That is, across schools within cities, student population characteristics significantly influence growth percentiles. But the influence of demography on destiny, shall we say (as measured by SGPs), is greater across cities than within them, an entirely unsurprising finding. Resource variations within cities show few significant effects. Notably, our wage index measure does not vary within districts but rather across them and was replaced in these models by a measure of average teacher experience. Again, there was no significant difference in average growth achieved by charters than by other similar schools in the same city.

Preliminary Policy Implications

The following preliminary policy implications may be drawn from the preceding regressions.

Implication 1: Because student population characteristics are significantly associated with SGPs, the SGPS are measuring differences in students served rather than, or at the very least in addition to differences in collective (school) teacher effectiveness. As such, it would simply be wrong to use these measures in any consequential way to characterize either teacher or school performance.

Implication 2: SGPs reveal positive effects of substantive differences in key resources, including staffing intensity and competitive wages. That is, resource availability matters and teachers in settings with access to more resources are collectively achieving greater student growth. SGPs cannot be fairly used to compare school or teacher effectiveness across schools and districts where resources vary.

These findings provide support for a renewed emphasis on progressive distribution of school funding. That is, providing the opportunity for schools and districts serving higher concentrations of low income children and lower current growth, to provide the wage premiums and staffing intensity required to offset these deficits.[4]

Implication 3: The presence of stronger relationships between student characteristics and SGPs across schools and districts within counties, versus across schools within individual cities highlights the reality that between district (between city) segregation of students remains a more substantive equity concerns than within city segregation of students across schools.

As such, policies which seek merely to reshuffle students across charter and district schools within cities and without attention to resources are unlikely to yield any substantive positive effect in the long run. In fact, given the influence of student sorting on the SGPs, sorting students within cities into poorer and less poor clusters will likely exacerbate within city achievement gaps.

Implication 4: The presence of significant resource effects across schools and districts within counties, but lack of resource effects across schools within cities, reveals that between district disparities in resources, coupled with sorting of students and families, remains a significant concern, and more substantive concern than within district inequities. Again, this finding supports a renewed emphasis on targeting additional resources to districts serving the neediest children.

Implication 5: Charter schools do not vary substantively on measures of student growth from other schools in the same county or city when controlling for student characteristics and resources. As such, policies assuming that “chartering” in-and-of-itself (without regard for key resources) can improve outcomes are likely misguided. This is especially true where such policies do little more than reshuffle low and lower income minority students across schools within city boundaries.

 

[1]https://njedpolicy.wordpress.com/2013/05/02/deconstructing-disinformation-on-student-growth-percentiles-teacher-evaluation-in-new-jersey/

[2]http://schoolfinance101.wordpress.com/2014/04/18/the-endogeneity-of-the-equitable-distribution-of-teachers-or-why-do-the-girls-get-all-the-good-teachers/

[3]http://schoolfinance101.wordpress.com/2014/01/31/an-update-on-new-jerseys-sgps-year-2-still-not-valid/

[4]This finding also directly refutes the dubious assertion by NJDOE officials in their 2012 school funding report that the additional targeted funding was not only doing no good, but potentially causing harm and inducing inefficiency. http://schoolfinance101.wordpress.com/2012/12/18/twisted-truths-dubious-policies-comments-on-the-njdoecerf-school-funding-report/

Buyer Beware: One Newark and the Market For Lemons

Mark Weber, PhD student, Rutgers University, Graduate School of Education

PDF of Policy Brief: Weber_OneNewarkLemonsFINAL

             The cost of dishonesty, therefore, lies not only in the amount by which the purchaser is cheated; the cost also must include the loss incurred from driving legitimate business out of existence.

– George A. Akerlof, The Market for “Lemons”: Quality Uncertainty and the Market Mechanism.

In his classic economics paper, Akerlof[1] addresses the problem of “asymmetrical information” in market systems. Using the used car market as an example, Akerlof shows that consumers who do not have good information about the quality of goods often get caught buying “lemons.” This not only hurts the individual consumer; it damages the market as a whole, as honest consumers and producers refuse to participate, concerned that false information keeps consumers from distinguishing a good car from a “lemon.”

The “school choice” movement is predicated on the idea that treating students and their families as “consumers” of education will introduce market forces into America’s school systems and improve the quality of education for all.[2]

But what if those families must make their choices armed only with incomplete or faulty data? How can a market operate successfully when consumers suffer from an asymmetry of information?

This brief looks at one example of asymmetrical information in a school choice system: Newark, NJ, whose schools were recently restructured under a plan entitled One Newark.

Newark’s schools have been under state control for nearly two decades; the elected school board only serves in an advisory capacity, making rapid, large-scale transformations much easier to facilitate. Under State Superintendent Cami Anderson, the district introduced One Newark, a plan that calls for students and their families to select a list of eight schools in order of preference for enrollment in the fall of 2014.

This author, in collaboration with Dr. Bruce D. Baker of Rutgers University and Dr. Joseph Oluwole of Montclair State University, has published several briefs analyzing One Newark’s consequences.[3] Among our findings:

  • The plan affects a disproportionate number of black and low-income students, whose schools are more likely to close, to be turned over to charter management organizations (CMOs), or to be “renewed.”
  • CMOs in Newark have little experience in educating demographically equivalent populations of students to the NPS schools; consequently, there is little evidence they will perform any better.
  • The statistical practices and models NPS has used to justify the classification of schools are fundamentally flawed.

This last point is of particular concern. The One Newark application[4] gives one of three ratings for each school a family may choose: “Great,” “On The Move,” or “Falling Behind.” While the district does offer its own profiles of each school, and the NJ Department of Education does offer individual school profiles, it is likely that the ratings on the application will have the most influence on families’ decisions.

If, however, these ratings suffer from the same defects we found in NPS’s previous attempts to classify schools – the lack of accounting for student characteristics, poor statistical practice, and using flawed or incomplete measures, among other problems – families may have a disadvantage when attempting to make an informed choice.

To ascertain whether the One Newark application ratings make sense, I used a statistical modeling technique embraced by NPS itself: linear regression. Only schools reporting Grade 8 test scores were included. The model here uses four covariates the district acknowledges affect test score outcomes: free lunch eligibility, Limited English Proficiency (LEP) status, special education status, and race.[5] The percentage of each student subpopulation for each school is included in this model, along with a covariate for gender, which has been shown to have an effect on test-based outcomes. This model is quite robust: over three-quarters of the difference in test-based outcomes can be statistically explained by these five student population characteristics.

The outcome used here is the one preferred by NPS: scale scores on the English Language Arts (ELA) section of the NJASK, New Jersey’s yearly statewide test. NPS averages this score across grade levels; however, as we have shown in our previous reports on One Newark, this is poor practice, as scale score means and distributions vary by grade.[6] I explore this problem more fully in the Appendix; for now, however, I accede to NPS and use their preferred measure, however flawed it may be.

When all five covariates are included in this model, they create a prediction of how a school will perform (relative to the other schools in Newark). We can then compare the predicted performance of the school with its actual performance. While not all of the difference can or should be attributed to the effectiveness of the school, this technique does allow us to compare the school’s performance against prediction to how the district rated the school in the One Newark application.

Figure 1

Lemons1
Figure 1 shows the difference from prediction for Newark schools – both NPS and charters – and their rating under One Newark. Schools that are being closed or turned over to CMOs are included for comparison. This graph illustrates several important points:

  • While there are many schools labeled as “Falling Behind” that perform below prediction, there are several schools that perform above prediction. Miller St. and South 17th, in particular, perform especially well given their student population. Under what criteria does NPS find that these schools are “Falling Behind”?
  • Conversely, several schools that perform below prediction are rated as “On The Move” or “Great.”
  • Only one charter school in the One Newark application is rated “Falling Behind” (University Heights, which did not report Grade 8 scores and is, therefore, not included in this analysis). But two charters in the application perform below prediction (Greater Newark and Great Oaks), and all except North Star[7] perform below Miller and South 17th.
  • Two other charters that perform below prediction – Robert Treat and Maria Varisco-Rogers – are not included in the One Newark application; these schools opted not to participate in the universal enrollment process.

Certainly, no school should be judged solely on one (flawed) metric. The point here, however, is that even by NPS’s own questionable standards, the classification of schools under the One Newark rating system appears to be arbitrary and capricious.

To be fair, the One Newark application does state that the district did not use averaged scale scores as its sole measure of a school’s effectiveness (to my knowledge, however, NPS has never publicly released a white paper or other document that outlines its precise methodology for rating schools). The district has also used median Student Growth Percentiles (mSGPs) to create its ratings.

SGPs, as measures of growth, are ostensibly measures that do not penalize schools for having students who start at lower absolute levels but still demonstrate progress. Supposedly, SGPs account for the differences in student population characteristics, which are correlated to test results. Former Education Commissioner Christopher Cerf[8] has stated: “You are looking at the progress students make and that fully takes into account socio-economic status… By focusing on the starting point, it equalizes for things like special education and poverty and so on.”

If this were true, we would expect to see little correlation between a school’s average scale score – its “starting point” – and its mSGP. Figure 2 plots these two measures in one graph.

Figure 2

Lemons2
As this scatterplot shows, there is a moderate but significant correlation between a school’s growth and its “starting point.” The bias shown in a school’s scale score, created by its student characteristics, is then at least partially shown also in its mSGP. In other words: SGPs are influenced by student characteristics, but NPS does not account for that bias when using SGPs to create its ratings.[9]

If a school’s student population, then, affects its mSGP, how do student characteristics affect the One Newark ratings? Figure 3 shows the differences in student populations for all three classifications.

Schools that are “Falling Behind” have significantly larger proportions of black students than schools that are “On The Move” or “Great.” Those “Great” schools also have significantly fewer students in poverty (as measured by free lunch eligibility) than “Falling Behind” and “On The Move” schools. “Great” schools also serve fewer special education students, and a slightly smaller proportion of boys.

Figure 3

Lemons3


Arguably, the One Newark rating is less a measure of a school’s effectiveness than it is a measure of its student population. If a family chooses “Great” schools, they are really choosing schools with fewer black, poor, and special needs students.

There is a serious debate to be had as to whether a “choice” system of education is viable for a city like Newark. If, however, NPS has committed to One Newark, it should view its role as a “consumer advocate,” correcting the asymmetry in information and providing justifiable school ratings, rather than limiting the choices students and their families have.

Unfortunately, it appears that NPS is choosing not to be an impartial arbiter; by forcing the closure of NPS schools that, by at least some measures, outperform charters, the district is actively distorting the market forces it claims will improve education.

Under Akerlof’s theory, then, One Newark may not only lead to more student stuck with lemons: it may actually drive more non-lemons out of the market.

 

Technical Appendix: Problems With Averaging Scale Scores

One-Newark-Enrolls-Paper-Application

In its response to our first report on One Newark, NPS made the case that averaging scale scores across grade levels is a superior methodology to ours, which used Grade 8 proficiency rates. We acknowledged that scale scores are a limited but legitimate measure of test-based student performance; certainly no less limited than proficiency rates, but still arguably as valid and reliable.

In our response to NPS, however, we do argue that while scale scores are acceptable for this sort of analysis, averaging scale scores

across grade levels creates a distortion that renders the scale scores less valid as school performance measures.

The problem with averaging scale scores across grades is that each grade level has a different mean scale score and a different distribution of scores around that mean. Table 1, originally presented in our response, shows the different mean scores for each grade level of Newark’s schools, both charter and NPS. The Grade 8 mean score differs from the Grade 4 mean score by over 16 points.

Table 1– Weighted Mean Scale Scores, NJASK LAL, 2013, Newark Only (Charter & NPS)

Test Obs Mean Std. Dev. Min Max
LAL 8 3301 205.0583 11.06671 183 235.8
LAL 7 3154 193.2245 15.9329 170.5 227.6
LAL 6 3631 192.7007 11.03825 172.9 224.5
LAL 5 3255 189.9525 12.66214 166.1 217.0
LAL 4 3223 188.3744 14.46348 165.6 235.5
LAL 3 3680 194.5205 12.0455 173.9 235.7

Why does this matter? Consider two schools with exactly the same average scale scores in all grades; now imagine that they each scored exactly at the citywide mean in all grades. One school, however, has considerably more 8th graders than 4th graders. That school would have an advantage when compared to the other: its larger proportion of 8th graders would push up its overall average, because the mean score for 8th grade is higher than the mean score for 4th. Weighting the means by the number of students in each grade wouldn’t solve this problem; in fact, it creates the problem, because the “average” student in 8th grade gets a higher score than the “average” student in 4th. More weight is being put on the score that is arbitrarily higher.

This problem is further compounded when running a linear regression. Because the dependent variable, grade-averaged mean ELA scale scores, is distorted by grade enrollment, the independent variables do not have a consistent relationship to the dependent variable from school to school. In effect, the rules change for every player.

A more defensible technique for averaging across grades is to run a linear regression for each grade, then calculate standardized residuals, which allow for comparisons across different mean scores. Those residuals are then averaged, weighted for student enrollment.

Figure 4 uses this methodology. Careful readers will notice that the relative position of many schools has shifted from Figure 1, significantly in some cases. Once again, however, there are “Great” schools that underperform relative to “Falling Behind” schools.

Even under this improved method, the classification of schools under One Newark remains arbitrary and capricious.

Lemons4

 

[1] Akerlof, G.A. (1970). Quarterly Journal of Economics (84) 3, 488-500. https://www.iei.liu.se/nek/730g83/artiklar/1.328833/AkerlofMarketforLemons.pdf

[2] For a classic example, see: Friedman, M. (1980) “What’s Wrong With Our Schools?” http://www.edchoice.org/the-friedmans/the-friedmans-on-school-choice/what-s-wrong-with-our-schools-.aspx

[3] – An Empirical Critique Of “One Newark”: https://njedpolicy.wordpress.com/2014/01/24/new-report-an-empirical-critique-of-one-newark/
– “One Newark’s” Racially Disparate Impact On Teachers:
https://njedpolicy.wordpress.com/2014/03/09/one-newarks-racially-disparate-impact-on-teachers/
– A Response to “Correcting the Facts about the One Newark Plan: A Strategic Approach To 100 Excellent Schools”: https://njedpolicy.wordpress.com/2014/03/24/a-response-to-correcting-the-facts-about-the-one-newark-plan-a-strategic-approach-to-100-excellent-schools/

[4] The paper application used for One Newark is no longer available at NPS’s website. Originally retrieved from: http://onewark.org/wp-content/uploads/2013/12/One-Newark-Enrolls-Paper-Application.pdf

[5] http://onewark.org/wp-content/uploads/2013/12/StrategicApproach.pdf

[6] See: A Response to “Correcting the Facts about the One Newark Plan: A Strategic Approach To 100 Excellent Schools,” p. 8.

[7] North Star, however, does engage in significant patterns of student cohort attrition that likely affect its student population and test scores. See: http://schoolfinance101.wordpress.com/2013/10/25/friday-story-time-deconstructing-the-cycle-of-reformy-awesomeness/

[8]http://www.wnyc.org/articles/new-jersey-news/2013/mar/18/everything-you-need-know-about-students-baked-their-test-scores-new-jersy-education-officials-say/

[9]https://njedpolicy.wordpress.com/2013/05/02/deconstructing-disinformation-on-student-growth-percentiles-teacher-evaluation-in-new-jersey/

A Response to “Correcting the Facts about the One Newark Plan: A Strategic Approach To 100 Excellent Schools”

Full report here: Weber.Baker.OneNewarkResponsewithexecsum

Mark Weber & Bruce Baker

Summary

This brief is a response to the Newark Public Schools rebuttal of our analysis of the district’s schools restructuring plan, One Newark. In this response, we find:

  • The consequences of the One Newark plan are racially disparate, creating a possible legal challenge for both the families of students and staff. NPS, however, has not acknowledged this part of our analysis.
  • NPS uses scale scores from state tests, averaged across grade levels, in their rebuttal. We find these measures to be seriously flawed, and certainly no better than the measures we used in our initial report.
  • Even using these flawed measures, we still find the classifications of schools under One Newark to be arbitrary and capricious when accounting for student population characteristics.
  • Even when using scale scores, we find no evidence that the student population of Newark will do better under schools run by charter management organizations. Further, the patterns of student cohort attrition in some charter schools and other behaviors lead us to question the validity of One Newark’s charter takeover strategy.
  • The statistical models used by NPS in their rebuttal are fundamentally flawed: specifically, the author(s) did not account for collinearity within the NPS model, biasing the results towards NPS’s favored position.

Introduction

On March 11, 2014, the Newark Public Schools (NPS) released a response to our policy brief of January 24, 2014: “An Empirical Critique of One Newark.”[1] Our brief examined the One Newark plan, a proposal by NPS to close, “renew,” or turn over to charter management organizations (CMOs) many of the district’s schools. Our brief reached the following conclusions:

  •  Measures of academic performance are not significant predictors of the classifications assigned to NPS schools by the district, when controlling for student population characteristics.
  • Schools assigned the consequential classifications have substantively and statistically significantly greater shares of low income and black students.
  • Further, facilities utilization is also not a predictor of assigned classifications, though utilization rates are somewhat lower for those schools slated for charter takeover.
  • Proposed charter takeovers cannot be justified on the assumption that charters will yield better outcomes with those same children. This is because the charters in question do not currently serve similar children. Rather they serve less needy children and when adjusting school aggregate performance measures for the children they serve, they achieve no better current outcomes on average than the schools they are slated to take over.
  • Schools slated for charter takeover or closure specifically serve higher shares of black children than do schools facing no consequential classification. Schools classified under “renew” status serve higher shares of low‐income children.

In its response[2], NPS questions both our methodology and our data sources. We are pleased to engage NPS in a thoughtful dialogue about One Newark; however, their rebuttal unfortunately confirms many of our conclusions about the plan, and refuses to even acknowledge many of our critiques.

Rather than answer NPS’s criticisms point-by-point, we take this opportunity to focus on the larger issues NPS raises about our brief, addressing specific arguments within the body of this response. It is our intention here to further the dialogue about One Newark in the hopes that NPS will move toward a position of transparency and engagement with stakeholders, both in and out of Newark.

Conclusions

We are pleased that “An Empirical Critique of One Newark” has generated a response from the Newark Public Schools administration. We have watched over the last few months as the topic of the One Newark plan has generated strong reactions from stakeholders both in and out of Newark. Given the changes that One Newark will bring – changes that even NPS agrees are profound and far-reaching – a measured, careful analysis of the rationale and consequences of these changes is clearly necessary.

Our conclusions are informed by public data using standard statistical methods. We labor to make our results replicable and understandable: we believe it is a testament to our work that NPS was able to respond to “An Empirical Critique” without any questions as to why we reached the conclusions that we did, even if they disagreed with those conclusions.

We believe it is time for NPS to make a similar commitment to transparency in their own formulations of policy. Despite their protestations, we are still no closer to understanding how NPS classified particular schools than we were before. We still do not know NPS’s rationale for why three particular schools are being taken over by two particular CMOs. We still do not know why staff at particular schools face an employment consequence while staff at other schools do not. We don’t know why NPS proposes to divest particular facilities to particular parties.

Backwards-engineering a rationale for One Newark does not contribute to transparency. Using flawed measures like averaged scale scores does not increase stakeholders’ faith in NPS’s ability to justify its plan. Engaging in poor statistical practice does not lead to confidence in NPS’s judgments. And failing to fulfill legal obligations to release data in a timely manner does not encourage a candid exchange of views.

We agree that the educational outcomes of Newark’s students are not acceptable, and that change is needed in the lives of Newark’s deserving children. Whether that change can come solely, or even primarily, through the policies of a state-run school district is an open question. We heartily agree, however, that school policies certainly matter, and Newark should constantly strive to make its schools better, even in the face of seemingly insurmountable problems whose solutions lie outside the purview of the public schools.

But no change can come unless and until an open dialogue about education takes place in front of a well-informed public, where all stakeholders have access to the inner working of the mechanisms that generate policies. If our briefs have compelled NPS to begin to engage in this dialogue, we will consider our time analyzing One Newark to have been well spent.

Follow

Get every new post delivered to your Inbox.

Join 5,834 other followers