As we have discussed in class, selection bias poses a significant challenge in evaluating the benefits of housing and schooling programs that involve both random assignment into groups of applicants who do and do not get into the desired program as well as differing levels of motivation to apply to the program in the first place. The authors acknowledge the possibility of selection bias in their research and make efforts to account for these effects, but I ultimately thought this was a flaw in the design of their study.
Massey and colleagues make appropriate analyses of the differences between groups, as they compare applicants to the Ethel Lawrence Homes who did and did not get accepted to live there. They note, “both current and former residents and nonresident applicants have self-selected into the group of people seeking to leave their current homes and move into the Ethel Lawrence Homes, thus holding unmeasurable traits such as motivation and gumption more or less constant” (74). The researchers claim their methodology thus produces a study with a high level of internal validity, and I agree with them there. Their comparison groups reminded me of the groups we as a class decided to compare when assessing the benefits of attending a particular school or type of choice school. However, even though the results may reflect real differences that can be causally attributed to residents in the Ethel Lawrence Homes, it is necessary to acknowledge that the lack of randomness in these groups may be a major flaw in generalizing the results.
The managers of the Ethel Lawrence Homes decide who may reside there through a process that is explicitly not random. In Chapter 5, Massey and colleagues write,
Applicant screening is thorough and includes credit checks, criminal background checks, income verification, and a home visit. Of the population seeking affordable housing in New Jersey, then, the residents of Ethel Lawrence are a screened subset of people who are probably less likely than poor people in general to have the proclivity, knowledge, and ability to engage in criminal or delinquent activities” (90).
This language reminded me of the rhetoric of “creaming” that is often brought up with respect to charter school admissions. Most particularly, the ELH admissions process seemed analogous to the “Hooping it Up” and “Send Us Your Best” items from Kevin Welner’s article on the charter school dirty dozen. In the descriptions of these items, Welner argues that charter schools often make the application process so arduous that only the most committed will follow through and that they also place requirements on enrollment that ensure that the already-highest-performing students get in. Similarly, in Mount Laurel, a creaming process seems to be taking place, and the authors acknowledge this at the very end of Chapter 9, but they do not delve into the effects that this has on the community or how it might influence their findings. Their comparison groups are chosen aptly, but they are not random, and to describe the results without mention of this critical fact is a large flaw.