Category: web-essay-2015-draft

BACKGROUND

The Mobility App run by Open Communities Alliance was created with the intent of helping people with government housing subsidies (notably Section 8 vouchers, an initiative that provides rental housing subsidies to low-income households) find homes in thriving neighborhoods with quality schools, low crime rates, and various other necessary amenities. ¹ The App is currently available through the Open Communities Alliance website, but is relatively new and underutilized. A participant begins by inserting an address (perhaps his/her own or perhaps a desired location), clicking “Search,” and is then shown a peg on a map that reveals the location’s “Opportunity Index” (rated on a scale of “very low” (tan) to “very high” (deep orange)) as well as various “Neighborhood Assets” near the location (starting at ½ of a mile away and spanning up to 100 miles) such as public schools, daycare facilities, grocery stores, and places of worship.  Each differently colored section is based off of census tract data.

                                 Figure 1: The Mobility App by Open Communities Alliance.

The “Opportunity Index” is based off of three qualifiers:

1. Economic Quality – such as but not limited to unemployment rate and job diversity
2. Educational Levels — such as but not limited to state math and English attainment exams
3. Neighborhood indicators — such as but not limited to crime rate and percent of people below the poverty line. ²

Yet, while the Mobility App is providing baseline support for housing movement, it is falling short in many regards.

1. There are not enough neighborhood assets to give a robust picture of what living in a specific location would be like.
2. There is not enough overt explanation as to what “Opportunity Index” means.
3. Participants are not fully engaging with the app in a meaningful way.  In a test done with the North End Action Team (NEAT) in Middletown, CT, only 29% of people polled changed how they thought about housing but nearly 65% of these people were looking to move. ³
4. Most notably, there are not direct links to actual available housing for rent, thereby undermining the singular mission that the Mobility App attempts to achieve through “help[ing] people with government housing subsidies interested in an opportunity move find homes within thriving communities.” ⁴

Considering this gap in effectiveness and the primitive resources currently given in the app, this essay calls for a redesign on the Open Communities Alliance Mobility App in order to create more opportunity and effective housing mobility.

HOW PEOPLE CURRENTLY ENGAGE WITH THE MOBILITY APP

On April 15, 2015, members of CSPL 341, the class for which this essay is being written (for more information on the purpose of this class as well as other projects conducted and various resources visit The Cities, Suburbs & Schools website), traveled down to the North End Action Team (NEAT) headquarters in Middletown, Connecticut to interview adults from the community in order to gather data on the public’s reaction to the Mobility App. We used audio and screen recordings to interview the subjects and track their movements and queries whilst interacting with the website.  We began with an interview script that asked participants whether they were thinking about moving or not and what their level of competency was with computers. Out of fourteen participants, 64% said they were thinking about moving and 71% said that they were regular computer users. ⁵
As we began the demonstration, we asked the participants to enter their approximate address (not actual address to protect for anonymity) and then review what the search revealed about their current location. The participants were instructed to explore the app at their leisure, however, this proved more difficult than expected. 57% of the participants could not enter their address without assistance and when exploring, 78% of participants clicked on one or fewer facets. ⁶ After the participants browsed the site, members of CSPL 341 then explained what each element (the opportunity index, the colored location dots, etc.) was designed to highlight in order to teach participants what each aspect of the app meant. Next, we asked participants to enter a predetermined address (100 Main St. Portland, CT) and have them describe to us what they saw. From there we concluded the interview by asking the participants if what they learned through the Mobility App changed how they viewed their neighborhood and if they had any feedback for the creators of the app.  The majority (71%) of participants did not change how they thought about housing mobility. We believe this was a result of the lack of an explicit explanation of each part of the tool. This application has proved not to be as easily accessible or self-explanatory as we once believed as many of the features were lost on the interviewees. However, once the application’s amenities were demonstrated and then explained in intimate detail, all but two users were not only able to navigate the application with ease but also were excited about its benefits. ⁷

WHAT PEOPLE SAY THEY WANT FROM A MOBILITY APP

When thinking about redesigning the Mobility App, there were a few pieces of feedback that came up repeatedly in our interviews at NEAT that coincided with a couple key housing desires expressed in Connecticut Fair Housing Center Study’s “Housing Mobility: What Do Housing Voucher Recipients Want?”  that we feel should be integrated into the redesign. First, there were some amenities that are not currently on our maps that interviewees mentioned were important to them when thinking about housing. This included both unquantifiable amenities such as “room to breathe” and “good neighbors” but also tangible amenities such as public transportation, affordability, and proximity to medical services and shopping centers. ⁸  Public transportation was an especially crucial amenity for both the NEAT participants and participants of the Connecticut Fair Housing. In fact, two members of the NEAT participant pool alluded to the fact they had no transportation at all and the Connecticut Fair Housing members ranked “Good Transportation” fourth behind “Safety”, “Cost,” and “Good Schools” (features which are already included in the current “Opportunity Index”). ⁹ At this point, the only reference we have regarding public transportation is the use of Google Maps. It would be incredibly beneficial to add local bus stops as one of the colored location dots on the map. Similarly, locations of hospitals/medical centers and shopping centers (but not limited to these entities) could be gathered through various data analysis programs. One of the other aspects mentioned was the issue of discrimination from landlords, with the participants in the Connecticut Fair Housing Center study, explaining that a bad landlord can ruin one’s housing search. Fortunately, a website, ratemylandlord.com, which allows tenants to review the landlords of their buildings and allows prospective tenants to read reviews before committing to living in the community. This website could be given as a link in the Mobility App to aid those looking to find housing in a building governed by equity and community.

 However the most important piece of feedback that we received from our interviews, which was seen both through observation and explicit verbal feedback, was that the website was not as accessible as we once believed. As shown in the data earlier, users struggled with the website until the features were explained in detail, which goes against the creators’ hopes for the website: users who stumble upon this app cannot find better housing opportunities if they cannot use the website itself.

Based off of the above, we have compiled a few changes to the design and function of the app in order to make it more accessible and useful.

1. Add in an “Introduction” video explaining how to use the app logistically and also to explain what facets of the methods or tools are, like opportunity level, tract, and neighborhood amenities.
2. Have the participants take a quiz that first assesses their current opportunity level then offers suggestions for higher opportunity level neighborhoods near their work or loved ones.

VIDEO

The inclusion of a video would visually demonstrate to participants (especially those not competent on a computer) how to effectively engage with the Mobility App. Hopefully, the video would make the website comprehendible to the extent that users would be able to navigate the mobility app from any location without the help of a housing counselor. This video feature would pop up automatically once the user has taken a short quiz, which will be explained in detail later. If he or she is not a first time user, the participant will have the option to skip the tutorial. The video tutorial will consist of six components and will be broken down into short video segments for each feature. The first chapter will introduce the application and explain what it is designed to help a participant with, reciting the App’s mission statement. The following chapters will be videos explaining more specific features within the application.

 The first specific aspect explained would be the “Opportunity Index” gradient located on the left side of the screen, as seen below. The narrator will clarify that the gradient ranges from very low, the tan color to very high opportunity level, a dark orange color, and that it was determined based on “economic, education and neighborhood quality.” ¹⁰ The narrator will go on to explain that that means evaluating the caliber of schools in the area, amount of job opportunities, crime rates and more.

                        Figure 2: Explaining the Opportunity Index on The Mobility App

Next the video will explain the color coded amenities below the opportunity index such as public schools, daycare facilities, grocery stores, and the new amenities included such as proximity to hospitals, shopping centers, etc. in addition to pegs with links to homes for rent in the area (marked with red arrows). This will show the user how they can select amenities that are important to them specifically, and how the map will automatically show dots that represent the amenities near an entered address. It will also demonstrate that when a participant clicks on a dot, the title of the resource, the address, the phone numbers associated with this location pop up, as well as a link to getting directions to/from this amenity.

                  Figure 3: The Mobility App with Added Amenities and Available Housing

The next chapter highlighted in the video will be an explanation of how to use the “Get Directions” feature attached to each locale. The “Get Directions” button, the voiceover will explain, leads a participant to a linked website (most likely Google Maps) where the user will be able to see the easiest routes by foot, bike, car or public transportation from the selected destination to a second address. It will give a short demonstration of how to do so by entering a sample address and visually revealing how to use Google Maps.

                              Figure 4: The “Get Directions” Button on The Mobility App

                                           Figure 5: Demonstration of Google Maps

Lastly, the video will describe one of the newer features that would be added with our redesign: the landlord feature. As described in the previous section, many participants mentioned that one of the details they want to know about houses or apartments is the quality of the landlord. Many participants expressed having difficulties with discrimination from unwelcoming landlords that made finding a new housing unit difficult. As a result, we suggest that there be a link be added to the available housing feature so that a participant can read reviews by previous tenants of that space or other spaces that the same landlord manages. One website that could be attached is ratemylandlord.com (see image of website below) . This added feature would address the repeated concerns expressed in the Connecticut survey, and with one of NEAT’s interviewees, and would ensure that the user would be given a full explanation of what living in a neighborhood and potential home would be like.

                                                            Figure 6: Ratemylandlord.com

 

At the end of the tutorial, a participant would click the “next” button, leading him/her to the Mobility App complete with the new amenities mentioned previously. The video tutorial would make the Mobility App more accessible for all users, especially technologically challenged users, and would help the app become more efficient in helping low-income Section Eight voucher recipients find a more upwardly mobile neighborhood and home to live in.

QUIZ

The second major suggestion to enhancing the Mobility App’s mission of “help[ing] people with government housing subsidies interested in an opportunity move find homes within thriving communities” would be to add in a quiz component.  The idea of this quiz is to not only make the app more readily accessible in that it would clearly visualize and explain, step by step, different opportunity levels, but also to individualize a participant’s search and to cater to the specific needs of a potential mover. As is clear from the many responses in the Connecticut Fair Housing Center’s report, “Housing Mobility: What Do Housing Voucher Recipients Want?” people looking to move desire a diverse array of commodities and assets such as level of safety, a “racially mixed” neighborhood, or a locale with schools that have special needs programs, just to name a few. ¹¹ In other words, the inclusion of a quiz would make the app more applicable and would produce results that are actually representative of what a participant would look for in choosing individualized housing.

The first part of the quiz would ask a participant to insert a current or recent address with the intention of then providing information on opportunity in this location.

 

 

                           Figure 7: Opening Screen of Mobility Opportunity Housing Quiz

Then, the Mobility App would locate the inserted location on the map to visually elucidate the opportunity level and various neighborhood amenities nearby (or lack thereof).

 Figure 9: In Depth Visualization of How Location Would Be Shown In Mobility App Quiz

After, this visualization, the quiz would continue by asking a participant to check off various amenities that are important to him/her such as but not limited to transportation, hospitals, job availability, places of worship, grocery stores, shopping centers, etc. In order to generate a full list of significant amenities, further testing of the current tool would be necessary. Further, the quiz would also ask participants to enter a location near their job or a family/friend’s home or a desirable location. The idea behind asking a participant to enter a location near a job or loved one is to reduce the time commuting and help create a more stress-free life. Individual transportation, as mentioned above, is not something readily accessible by many of the NEAT or Connecticut Fair Housing Survey participants and, as a result, high quality public transportation is something they rely on heavily. Next, the app would show the location a participant has just entered on a map with their chosen amenities.

(Note from the instructor: Students were ask to write essays under 2500 words, which would be about here. Evaluators may read as much as they wish, but should not rate the essay beyond this point.)

                 Figure 10: Visualization of Mobility App Opportunity and Housing Quiz

 However, simply asking a participant to insert a desirable address is not enough in supporting movement into more thriving communities. Therefore, based off of the information given in the quiz, the app would then compile a list of neighborhoods or towns that were in a higher opportunity index than the original address entered but that also had all the amenities that the participant listed were important to him/her and were close to the desired address. The app would provide this list in full as to give a participant as much choice as possible. Yet, one of the biggest downfalls with the original design of the app is that it fails to provide actual listings for available housing. Therefore, in this new design, the app would provide external links to rental homes (from Trulia or Zillow for example) as well as to ratemylandlord.com in order to make finding affordable housing as streamlined and pleasurable as possible.

When coming up with the idea of the quiz, one of the ideas discussed was whether or not to include a section that asked for a participant’s income and/or his/her desired rent. However, after consideration, it seemed that although this question initially makes sense in order to ensure that people were not spending beyond their means, considering Section 8 housing vouchers, inquiring about a participant’s income or rent constrictions is counterintuitive. Moreover, the Mobility App aims to close arbitrary socioeconomic stratification. Therefore, we would not want a participant to feel marginalized or forced into certain locations based off of money constrictions that may not even be relevant in light of subsidies.

The point is to keep choice as open and participant-centric as possible.

CONCLUSION

As the Open Communities Alliance Mobility App currently stands, it does not fulfill its mission.  The Mobility App attempts to assist Connecticut residents with housing subsidies such as Section 8 vouchers find homes in new locations with higher opportunity levels than his/her current home. While the Mobility App is a good start to attenuating the issue of socioeconomically and racially stratified housing, it does not provide enough resources to make meaningful change. However, with a few simple additions and edits, the app could be more applicable, pertinent, and easy to use. The inclusion of an Introduction Video would show those new to the app (and potentially not comfortable regularly using computers) how to interact with it and what each facet and aspect of the app means and how they function. This would ensure comprehension. Further, the inclusion of a Quiz would help personalize a participant’s experience with the app to create housing choice catered to individual needs and therefore more useful in terms of suggesting relevant housing. On a more microscopic scale, the list of neighborhood amenities should be expanded to include (but not be limited to) public transportation, location of hospitals, and proximity to shopping centers. Most importantly, as the app stands, it does not assist participants in “find[ing] homes within thriving communities” because it does not offer any links to available real estate to rent in these new and higher opportunity levels.  Therefore, the most critical, and perhaps most basic, change that the redesign should address is the inclusion of a list of or external links to open homes.

If the Open Communities Alliance Mobility App included the aforementioned redesign suggestions, we believe the tool could assist those looking to move find long-term and stress-free housing within his/her budget, desired location, and specific amenities.

There has been much criticism of the lottery, the convoluted application processes and other creaming tactics of charter schools to recruit and maintain a high-achieving, highly motivated and often hyper-segregated student body (e.g., Welner, 2013). Specifically in the Hartford metropolitan region, an investigation of parental perception of choice schools—specifically interdistrict magnet schools—detected a prevalent sentiment that choice schools create community divisions due to the competitive nature of the lottery (Debs, 2015). The present thematic analysis shows that the presentation styles and patterns of verbal communication between charter school and interdistrict magnet school representatives and families at school choice fairs and open houses can also be extremely influential in shaping a particular type of school community. Across the field notes from six different school choice events in the Hartford metropolitan region collected by Wesleyan University students in February 2015, school representatives emphasized building specific types of school communities in explicit and subtle ways.

Since magnet schools by definition unite diverse families from different locations to ensure integrated student bodies, fostering an inclusive and welcoming school community would seem like a necessary selling point. On the other hand, many charter schools, which often prioritize closing the achievement gap, are likely to emphasize a school community that appeals to a particular targeted demographic (e.g., low income minority students). Therefore, the question becomes, are the types of communities emphasized by school representatives at school choice events reflective of the respective school mission and/or vision statements? Rhetoric of school representatives from several interdistrict magnet schools and one charter school from a regional charter network in conjunction with the language of school mission and/or vision statements provides evidence for an array of choice school communities that may subtly or directly seek to attract particular types of students and further or inhibit integration.

Diversity & Community

While school community is often directly or indirectly addressed by choice school representatives in their pitches to prospective families, not all representatives define school community based on the same criteria. The most intuitive type of school community expressed would be based on the student body demographics (i.e., percentage of minority students, students who qualify for free and reduced price meals, students who are English Language Learners (ELL) or have a documented disability). Yet, seldom did the school representatives address this information directly. Observations from the Regional School Choice Office (RSCO) Fair mentioned that school representatives referenced the Sheff v. O’Neill ruling in regard to maintaining a certain racial balance at their interdistrict magnet school (HH, field notes, 2015). Other field notes from the same event reported that during the “entire hour and fifteen minutes there I did not hear the words ‘race,’ ‘Black,’ ‘White,’ ‘Latino,’ or ‘disadvantaged’ (LS, field notes, 2015).” Only the Hartford Magnet Trinity College Academy (HMTCA) was reported to have explicitly mentioned its mission to create a diverse student body through a film shown to open house attendees. The field notes reported the following:

The film talks about plurality and shows shots of children of many different races. The central thesis seems to be that Hartford Magnet Trinity College Academy was the mechanism through which children from diverse racial, geographical, and socioeconomic backgrounds have been permitted to come together and meet people from other milieus. (IF)

Additionally, the mission statements from the magnet schools described in the field notes of open houses do not directly state that their schools seek to attract and educate a diverse student body (i.e., Breakthrough Magnet School, Greater Hartford Academy of the Arts, CREC Academy of Aerospace Engineering Elementary School, HMTCA). These pieces of evidence suggest that, counter to the explicit intention of interdistrict magnet schools to promote racial and socioeconomic diversity, neither school representatives in school choice event settings nor their mission or vision statements were likely to describe their school community using direct language that underscores the diverse students they serve. While the legal language of the Sheff decision may be a standard explanation for school representatives, it may also alienate some parents due to a lack of prior exposure to the topic.

In contrast to a school community composed of a diverse student body, many charters, especially those that follow a “no excuses” model, have been criticized for generating student bodies that are more segregated than traditional district schools in the same jurisdiction (Kahlenberg & Potter, 2014). Data trends from 2011-12 to 2013-14 on racial  integration support this criticism (See Figure 1). 

This data is an update to the Cotto & Feder 2014 Choice Watch Report.

Cultural appeals to low income minority families by school representatives illustrate a strategic way in which charter schools develop racially or ethnically isolated communities. The school representatives at the Achievement First open house appealed to certain demographics through subtle, yet meaningful, gestures and modes of communication. For example, a school representative at the open house in Hartford that began at 5:30 PM provided empanadas and spoke Spanish with some Hispanic families. These efforts were described as a way to “establish the cultural/linguistic bond of identification” (CB, EK, AF field notes, 2015). While these efforts did not directly address community building, they created a welcoming community atmosphere specifically for the Hispanic parents. The Achievement First mission statement’s direct emphasis on “closing the achievement gap” and providing educational equity to historically underserved populations provides a clear impetus for directing promotional tactics at specific demographics who would most benefit from the school’s philosophy (“Our Mission and Vision,” 2015). While racial and socioeconomic integration in education has been repeatedly found to boost minority students’ academic outcomes, Achievement First’s model does not follow this approach to closing the achievement gap, but rather a rigorous academic approach targeted at a concentrated population (Kahlenberg & Potter, 2014).

Students, Staff & Families

Instead of touting the diversity of the student body or making specific appeals to certain demographics, some school representatives characterized their school community using a character appeal by underscoring the collaborative and supportive student-staff culture. The Breakthrough magnet school representative at the school’s open house characterized the school community as having a “family feel” and as a “close knit school community” (NT, field notes, 2015). The field notes went on to report that the school representative “used many buzzwords such as, ‘thriving,’ ‘joyful,’ ‘character,’ ‘safe,’ ‘risks,’ and ‘leadership,’ often using more than one in the same sentence.” Breakthrough magnet school’s emphasis on the character of their students and staff who compose their school community echoes their mission statement’s focus on developing “students as models of outstanding character” (“A Global School for Students of Character,” 2015). What is striking about this school representative’s portrayal of the school community is that the topic of parent involvement in the creation of a collaborative school community was omitted. Conversely, the mission statement dictates that Breakthrough Magnet School seeks to form a partnership between “staff, family and community members.” Although it is unclear whether the topic of parental involvement was intentionally excluded from the school representative’s pitch to families, the omission could indicate a larger problem facing magnet schools: the difficulty of engaging families from disparate communities and backgrounds at the school.

Other magnet schools and the charter school in the field notes sample did, however, directly mention parental involvement as a core element of their school community. At the CREC Academy of Aerospace Engineering magnet school open house, the representatives specifically addressed how a link between the family and the school is established by mentioning that a parent group convenes monthly to plan activities and to fundraise (AG, RU, field notes, 2015). This comment places an emphasis on parental behavioral engagement in the school community, which for many working class parents may be hard to execute. The school representatives at the Achievement First open house emphasized a potentially more accessible way in which parents can contribute to the school community. The field notes from this event reported the following:

The presenters also mentioned several times that they consider parents to be partners in the mission to get the kids to college. They said that there was a lot of communication between parents and teachers, and the teachers send a color-coded report about each student’s behavior to the parents every day. They also had a list of expectations for parents as a slide on the PowerPoint, and this included things like the expectation that the parents get their child ready for school and wearing their uniform each day and that they ensure their child does his or her homework. (CB, EK, AF, field notes, 2015)

 

This observation indicates an attitudinal and philosophical form of parental support and engagement that is a key element of bridging the gap between school and family. Instead of asking that parents regularly volunteer at the school or help fundraise, the school asks that parents be mission-aligned in order for students to be best prepared for the academic and behavioral demands during the school day. Additionally, this commentary prioritizes constant and transparent communication between parents and teachers. This suggests that the Achievement First family-school community operates under a reciprocal, goal-oriented and highly communicative system. As a way of further emphasizing the mission-alignment of all stakeholders in the Achievement First community, related quotes from principals and administrators appear on the webpage containing the mission statement.

Niche Communities

School communities formed by the coalescence of mission-aligned groups is also apparent among interdistrict magnet schools, which often develop niche communities based on specialized interests. In order for magnet schools to attract students from a wide range of socioeconomic and geographic backgrounds, they typically have a special curricular focus or theme that a traditional district school would not offer. The magnet schools Greater Hartford Academy of the Arts and CREC Academy of Aerospace Engineering Elementary School provide quintessential examples of specialized school communities. The mission statement of the Greater Hartford Academy of the Arts portrays a definite dedication to developing visual and performing artists as well as providing an “academically rigorous curriculum” (“An Artistic and Academic Immersion,” 2015). The photograph directly under the mission statement visually displays the state-of-the-art facilities to further highlight the emphasis on the arts.

While there is no explicit mention of fostering an integrated student body in the mission statement, it does state that the arts are to be learned as a “mechanism for social justice” which perhaps connotes a social consciousness of the diverse backgrounds of the students. The field notes from this school’s open house corroborate the prioritization of artistic development and the absence of discourse on diversity or the student demographics. Only in describing a promotional pamphlet did the field notes mention the school’s “desire to be a part of a diverse, multi-cultural environment” (CM, field notes, 2015). Additionally, out of approximately 150 families present at the open house, the field notes observed that only one family was African American and only one was Hispanic. This provides potential evidence that either the curricular theme of the magnet school was particularly appealing to a certain demographic, or the timing or location of open house was not easily accessible or welcoming to minority families.

Similarly, the CREC Academy of Aerospace Engineering Elementary School’s mission statement stresses it’s dedication to teaching rigorous and cutting edge science and technology, yet there is no explicit mention of an aim to establish a diverse student body as is legally mandated by the Sheff ruling (“Academy of Aerospace & Engineering Elementary School,” 2015). In contrast to the field notes from the open house at the Arts Academy magnet school, the field notes from this magnet school’s open house did, in fact, address the issue of integration. Curiously, however, race was discussed though a proxy of geographic locations in the Hartford area, which aligns with the stipulations of Sheff. The school representative was reported stating that, “50% of the students are from Hartford and 50% are from 33 other towns” (AG, RU, field notes, 2015). The families present at the open house were also described as being more diverse and thus representative of the Hartford region. What seems apparent is that while interdistrict magnet schools are designed to establish and maintain integrated school communities, niche community-type magnet schools based on a specific focus are not likely to explicitly express this goal in their mission statements nor in the pitches given by school representatives at open house events.

Community Partnerships

While the magnet schools previously mentioned portray school communities that can seem insular due to their strong focus on a specific theme, other choice schools establish school communities that extend beyond the school and into the local community. The Hartford Magnet Trinity College Academy’s vision statement places a major emphasis on incentivizing community engagement, working to improve the community, and collaborating with community contributors (“Vision Statement,” 2015). The field notes from this magnet school’s open house align with this emphasis on community partnership. While the school representatives did focus on the student-teacher relationships, they also highlighted the school’s main partnership with Trinity College, located across the street (AM; IF, field notes, 2015). Photographs on the vision statement’s webpage illustrate students’ engagement with the college. Yet, when the school representative discussed extracurricular activities, the representative mainly focused on the arts and failed to explicitly mention any type of community engagement (IF, field notes, 2015).

The mission statement and the presentation by school representatives at Achievement First also underscore a school community that extends into the larger community. However, a dichotomy between a collaborative spirit and a competitive tone emerged regarding this topic. The Achievement First mission statement highlights its goal of sharing best practices with district schools and other charter schools in order to bring about greater education reform and to close the achievement gap (“Our Mission and Vision,” 2015). In contrast to criticism of some charters for their push to replace district schools (Kahlenberg & Potter, 2014), this message emphasizes collaboration. The field notes from the Achievement Frist open house indicate that the school representatives also mentioned other district schools in the Hartford region and in the state of Connecticut. A data-driven approach was taken in which schools were compared by test scores and presented as bar charts on a PowerPoint (CB, EK, AF, field notes, 2015). This message has a competitive charge and does not acknowledge any sort of collaboration between Achievement First schools and non-Achievement First Schools. Thus, school community in this case can be conceptualized as the “in-group,” or schools and educators aligned with the Achievement First mission, juxtaposed against the “out-group,” or those who do not share practices or a common philosophy with the charter network.

(Note from the instructor: Students were ask to write essays no more than 2500 words, which would be about here. Evaluators may read as much as they wish, but should not rate the essay beyond this point.)

While an array of school communities are delineated by school mission statements and largely supported by communication between school representatives and families, little evidence points to the communities formed on the classroom level that may reflect the larger school community or represent internal community divisions. The most prominent way in which either of these possibilities could be apparent is through the presence or absence of tracking, ELL programs and mainstreaming opportunities for students with disabilities. None of the school mission statements mentioned meeting the needs of ELL or special needs students. Seldom did school representatives mention such programs and when they did, they were reported as being said in passing. Thus, while a school as a whole may serve a diverse student body, a closer examination of internal programs would elucidate whether students receive a diverse classroom experience. Additionally, discourse on school community building was largely absent from the RSCO fair notes where there seemed to be greater emphasis on logistics rather than on school culture and community. School community building is possibly better addressed at events that are less hectic and focused on one school. 

References

A Global School for Students of Character. (2015, May 2). Retrieved from http://www.breakthroughmagnetschool.org

Academy of Aerospace & Engineering Elementary School. (2015, May 2). Retrieved from http://www.crecschools.org/our-schools/academy-of-aerospace-engineering-elementary-school/

An Artistic and Academic Immersion. (2015, May 2). Retrieved from http://www.crecschools.org/our-schools/greater-hartford-academy-of-the-arts/

Cotto, R. & Feder, K. (2014). Choice Watch: Diversity and Access in Connecticut’s School Choice Programs. Connecticut Voices for Children. New Haven, CT.

Debs, M. (2015). Untouchable Carrots: Marketing School Choice and Realities in Hartford’s Inter-district Magnet Program. Unpublished Manuscript, Yale University, New Haven, CT.

AG, RU; AM; CB, EK, AF; CB; IF; NT. Field Notes. (2015, February).

Kahlenberg, R. D., & Potter, H. (2014). A Smarter Charter: Finding What Works for Charter Schools and Public Education. New York: Teachers College Press.

Our Mission and Vision. (2015, May 2). Retrieved from http://www.achievementfirst.org/our-approach/achievement-gap-and-mission/

Vision Statement. (2015, May 2). Retrieved from http://hmtca.hartfordschools.org/vision.html

Welner, K. (2013). The Dirty Dozen: How Charter Schools Influence Student Enrollment. Teachers College Record. Retrieved from http://www.tcrecord.org.

School choice—a resounding success! Or is it?
Across the nation the popular rhetoric used to describe school choice is glowing. Describing Connecticut’s choice system, newspaper headlines proclaim, “[Choice programs are] a major contributor to closing the achievement gap” ¹ and “Students [in school choice programs] are improving each year!” ². The much talked about full-length documentary, Waiting for Superman, holds charter schools and parent choice up as the last hope for our urban students to succeed. But in reality, many of these assertions are made based on a faulty comparison ³. The current rhetoric used in the public sphere about choice schools and student performance is not accounting for the fallacy of selection bias.

Measuring the achievement impact of choice schools compared to traditional public schools on students is very difficult. The only true comparison would be to take advantage of a parallel universe in which one could compare students who attended a choice school in one universe with the very same students who simultaneously attended a public school. If this technique was possible, many researchers would be out of the job.

This article points out the flaws in many evaluations of choice schools, and highlights several ways to mitigate and improve school choice analysis. Additionally, using a robust data set, I provide original analysis that mitigates some these issues and situate the findings in a broader context.

Selection bias—the problem that plagues all school choice studies
To investigate the effect that school choice has on student outcomes, researchers leverage statistical tools to try to make the most accurate comparison. The issue we are most concerned about when trying to make this comparison is selection bias. Selection bias occurs when the population of students you are looking at is not random but is self selected. The thing about self selection, is that it may be due to some factor that actually matters in the whole equation. In the school choice debate, we worry about selection bias when the families who chose to apply and attend a charter school are even slightly different that the families who just end up keeping their kids in traditional public schools. The problem arises when we try to compare these two groups. It may be that the difference we observe in test scores is really due to the dissimilarity in the family characteristics rather than in the effectiveness of choice or traditional public schools. Herein lies the challenge: How do we make a true comparison of student outcomes between choice schools and traditional public schools?

Virtual twin method—one way to minimize the impact of selection bias
The CREDO team at Stanford University came up with a method called “virtual twin” to try to make better comparisons. Essentially, they use measurable student characteristics and previous achievement to match students in charter schools with students who attend public school in their same school district. For example, CREDO compares two students with similar prior test scores both coming from low income and high parental education families, but one student now attends a charter school and the other attends a traditional public school. They do this with many pairs of students or “twins” to curb selection bias and make a better comparison between the two school types. Using this methodology in 2009, the CREDO team found that only 17% of charter schools outperformed traditional public schools, while 46% did worse, and 37% had no statistical difference. ⁴ They repeated this study on a slightly larger sample of students in 2013 and found that charter schools on average performed slightly better than in the 2009 study ⁵, but that at the end of the day, an average charter school is just average.

The virtual twin methodology is not perfect, because not all factors can be matched. There still may be some unobservable differences between students who attend charter schools compared with their public school peers. For example, a family that takes the time and effort to apply to a charter school, might be more involved in their student’s education than a family that just sends their student to the neighborhood school, and that might be why we see choice school students performing better than the traditional public school students.  In other words, the result may be driven by the unobservable characteristics of the students who attend charter schools, rather than the actual effect of the charter school themselves.

Randomization—another way to address the problem of selection bias
Using another method to mitigate the issue of selection bias, some researchers take advantage of the randomization inherent in a charter school lottery. When charter schools receive more applications that spots available they are required to hold a randomized “lottery” to determine which students receive a spot. In a large study of charter schools, Gleason et al, (2010), compared the achievement of students who won charter lotteries and attended charter schools to students who lost charter lotteries and attended traditional public schools. Since the lotteries are random, we assume on average, there is no difference between the people who won and lost the lottery ⁶.

In terms of methodology, randomized trials are the closest one can get to a perfect comparison. The methodology helps mitigate the selection issue present in the CREDO study, since the student population they are comparing, the winners and losers, both have the unobservable characteristics that lead to a family applying to a charter school. The Gleason study found, on average, that there was no statistically significant impact of charter schools on student achievement. Similar to the CREDO studies, Gleason reports positive outcomes for students with low-SES backgrounds. But even this study with randomized design has it’s limitations. For example, only schools that receive more applications than spots use a lottery, therefore the charter schools analyzed in this study were charter schools that received lots of applications, potentially meaning they were on average better charter schools.

Big Data Analysis—a third method to account for selection bias
I set out to find a different method to add to the current understanding of the effect school choice has on student outcomes, taking into account the main issues involved in investigating student outcomes, including selection bias and the unobserved factors that come with it. Increasingly, researchers are collecting data about students over time, in what are referred to as longitudinal studies. These studies often involve capturing data about large numbers of students via surveys, resulting in large data sets. I decided to use one such data set, from the High School Longitudinal Study of 2009.  By using a variety of variables focusing on student achievement, family background, and school characteristics from the High School Longitudinal Study of 2009 (HSLS:09) I wanted to see if I could shed light on the school choice debate.

The High School Longitudinal Study of 2009 was conducted by the National Center for Education Statistics. The data set is comprised of nearly 24,000 9th graders selected randomly from 944 schools. Students, parents, teachers, administrators, and counselors are all surveyed to collect a wide variety of data on both the students and their learning environment. Called multi-level surveying, this data set, in concert with students test scores provided a rich data set for analysis.For an extended explanation of these data, click here.

One of the main issues with using survey data is that it is impossible to account for every potential factor that determines student achievement. In order to isolate the true effect of participating in a school choice program, it’s necessary to hold constant every other potential difference between students. This is obviously an impossible task, especially considering the many unobserved and unmeasurable factors that are present, such as differences in student motivation or innate ability. However, there are analytic and statistical strategies that enable you to control for these differences, that allow you to better isolate the true relationship between school choice and student achievement. I used a variety of these student, parent, teacher, and school controls to try to measure the underlying components that affect a student’s test score.

My Assumptions
I set out, assuming that five factors are most important in determining student achievement.They are: 1) whether or not a student attends a choice school, 2) students’ demographic characteristics, 3) students’ motivation, 4) a student’s parental characteristics, and 5) a students teacher characteristics. If we had the data and could measure all of these underlying factors, we could make a convincing case for the accuracy of our estimates to truly measure the effect of choice on student achievement.

Unfortunately, many of these underlying constructs are unobservable, not measured, or have layers of complexity. To mitigate this issue, I used factors I could measure that get at the underlying construct and are highly correlated with these unmeasured factors. For example, when looking at student motivation, I controlled for whether students think getting good grades is important and whether students think they will graduate from college. The hope is that high student motivation, an unobservable characteristic, will overlap sufficiently with students who think getting good grades is important and expect to graduate from college to serve as a proxy. For results to be reliable, these relationships need to be highly correlated but not necessarily perfectly correlated. This is because when working with a very large number of students, as I was, one can begin to see that on average these factors will account for motivation. The rest of our proxies are displayed in table 2. Click on each underlying construct for a deeper examination of the variables used to measure them. Click here for a full breakdown of the model used in estimation.

Click to read more on how each construct is represented: true achievement, school choice, family background, student motivation, parent characteristics, or teacher characteristics.

The Tricky Bit—How to Account for Selection Bias
Now for the important question, in the context of these data and techniques, how did I compare students in choice schools to students in traditional public school knowing that that difference in decision might be because of some unobservable characteristic obscuring the true comparison between choice students and traditional public school students?

My hypotheses going in to this study is that when first looking at choice schools on student achievement I would see a positive effect because of selection bias; I expected that the students in choice schools would be systematically different from those in traditional public school due to parental factors that affected their selection of a choice program. However, I expected that after explicitly controlling for parental characteristics, and making a much more valid comparison between students in both types of schools, the initial positive result will not persist. My hypothesis is consistent with past studies that support the idea that parents who send their students to choice schools are on average more involved in their students education thus effecting achievement (Gleason et. al, 2010; Ballou et al., 2007; Betts et al., 2006, CREDO, 2009).

To control for this confounding factor, I used a variety of controls to account for a wide variety of parent involvement. I considered whether or not a parent attends any meeting at the school, a parent teacher organization meeting, or a parent teacher conference. I took into account whether a parent volunteers at their child’s school or helps fundraise for the school. I also considered parents’ expectations of how far they think their student will get in school, and whether or not they help their student with their homework. My assumption was that together all of these variables account for and overlap sufficiently with the unobservable characteristics that choice school families have that would affect student achievement. Although these factors do not directly account for the underlying construct I argue that these characteristics would signal and proxy for the unobserved ones.

The strength of this approach is that it addresses the issue that comes in to play with the Virtual Twin methodology—selection bias, and it gets around some of the main issues of randomization including only looking at over-subscribed schools. The weakness of the method I used is needing to rely on my proxy strengths without being able to actually tell if they sufficiently account for selection bias. There are some who say that data analysis is more  art than science. A statistical model is an argument and it is important to question each assumption, while at the same time stepping back to look at what the whole thing has an ability to tell. I argue that the above variables account for enough of the underlying factors of student achievement for our results to be unbiased.

Click here for the descriptive statistics of all the variables used in estimation.

My Findings
Using data from the High School Longitudinal study of 2009 (HSLS 09) and the above methodology, I indeed found that when initially looking at the relationship of participation in a school choice program and student learning, there exists a positive effect for students of low socioeconomic status. This result explains some of the promise and glamour that the idea of school choice receives. However, after using more robust methods and explicitly controlling for the difference in students and families that chose to attend choice programs, the once promising result, disappears.

To arrive at this conclusion I first compared the achievement of students who went to choice schools to that of students who went to traditional public schools while accounting for their race, socioeconomic status and intrinsic motivation. I found that attending a choice school had a positive impact on students from low socioeconomic background. Results based on simple comparisons like this are constantly held in the media as evidence of the positive impact of school choice. To account for the issue of selection bias and the potentially unobserved parent characteristics as the possible reason choice students appear to perform better in my first comparison, I next also accounted for the parent-related variables. As discussed above, these variables are used to account for the potential selection bias introduced because of the differences between the populations at choice schools compared to traditional public schools. I found that after accounting for selection bias, on average, students in choice school perform no better than students in traditional public schools. This result confirms my hypothesis and corroborates other literature indicating that after accounting for selection bias, on the whole choice schools do not outperform traditional public schools. Lastly, when accounting for teacher quality, the results remain the same. Click here to see the full table of regression results.

In summary, looking at the simple relationship between choice schools and student achievement, I found a positive effect of choice schools, consistent with popular claims made in the headlines. However, when accounting for the observed and unobservable differences in data, these once promising results do not persist.

The Limitations
As previously discussed, there are several limitations to this study. First, without random assignment there is no way to be sure that we fully accounted for selection bias. I can make an argument, and I hope that I have, that my methodology accounts for selection bias, but we will never know for sure. Second, beyond selection bias, we don’t know if there are other factors that affect achievement that we are not accounting for that are systematically different between students in choice schools and students in traditional public schools. Researchers call this omitted variable bias, and it is always an issue when working with survey data in particular.  One indicator that this study may sufficiently account for both selection and omitted variable bias, is that its results are consistent with randomized studies on schools choice that also find no relationship between choice and student outcomes ^{7 8 9}.

Additionally, it is worth noting that this study looks at choice schools on average. This does not mean that no choice schools are outperforming traditional public schools. Rather, it means that as a whole the choice school reform movement is not outperforming the status quo of traditional public schools. Further, this paper also does not distinguish between types of school choice. Because of data limitations charter schools, magnet schools, and voucher programs were clumped together.

Click for more technical limitations and solutions to these limitations including, missing values, attrition, and other data issues.

The Implications
With school choice becoming increasingly popular among reforms it is crucial to investigate its actual effect on students. Although there is a large body of existing research, it is important to keep looking for pieces in the solution to bring better educational opportunities to students as policies shift and school systems progress. A single assessment of the choice system alone will not provide enough evidence on it’s own, but using an abundance of data and a range of techniques, we can continue to fill in more and more of the picture.

Next time you read about a school choice success, don’t accept the result outright. Make sure you consider the comparison they are making, and ask yourself: Do you believe these two groups are equivalent? Has the study sufficiently accounted for the unobservable differences between students in choice schools and students in traditional public school?