A Smarter Charter: Self-Selection Biases in Charter School Studies

Posted on

Chapter 4 of A Smarter Charter summarizes research comparing student outcomes between charter and traditional public schools. The authors concluded that students in most charter schools perform about the same as students in comparable public schools (68). However, since students choose to enroll in charter schools, this comparison is difficult to make due to self-selection biases that make students attending a charter school and traditional public schools different.

The authors discussed a CREDO study that found low-income and ELL students in charter schools outperformed students in traditional district schools, but without a randomized controlled experiment, an alternative explanation could be that families who sought out charter schools were more motivated (70).  Bifulco et al., who studied magnet schools, also listed motivation and parental support as potential confounding variables. Thus, the authors stress that although the findings may look promising for charter schools, it is unclear if charters are directly responsible for gains in student achievement or if the gains are due to other factors, such as family motivation.

An IES study controlled for family motivation by comparing students admitted to charter schools by random lottery with students who applied but were not admitted. The authors explain that although this may eliminate the concern with the CREDO study about family motivation, peer influence is still a potential bias, with lottery winners surrounded by classmates from similarly motivated families, while lottery losers are educated with many peers who did not apply to a choice school, and hence may not be as motivated (72). Again, as in Bifulco et al.’s analyses, we cannot determine if the school is directly responsible for improving student achievement or if another factor is driving the relationship.

The authors also discussed KIPP, a charter organization that emphasizes tough love and boasts demanding expectations (78). This program undoubtedly uses many of the strategies explained by Welner to influence student enrollment, including the “bum steer,” by driving away ELL and special needs students from applying with their tough love mentality. KIPP also makes use of Welner’s “flunk or leave” tactic, and only students that survive the demanding expectations remain by high school, as KIPP does not replace students who leave. Although KIPP students have shown substantial academic gains, when KIPP took over a regular, high-poverty public school, serving a non self-selected population, the program failed, indicating that the academic achievement at KIPP may be due to the high motivation levels of the students, and not the charter program itself.

The authors highlight the self-selection biases that make it difficult to definitively state that charter schools cause gains in student achievement. It is possible that influence of parental motivation and peers may be driving the apparent improvement among charter school students. Additionally, a close look at the KIPP organization indicates that student achievement may in fact be due to the types of students the school enrolls than the actual school itself. Is it the quality of instruction or the students who choose to enroll that make charter schools successful? Can these self-selection biases ever be completely controlled for?