Pomona College

Using Qualitative Evidence to Support A Holistic Model of Student Advising

Submitted by:
Dr. Cecelia Conrad, Dean of the College
Dr. Miriam Feldblum, Dean of Students and Professor of Politics

October 28, 2009

Previous research has found evidence that high quality advising is positively related to student success.  (Tinto 2004, Metzner 1989, NSSE 2005, Kuh et al, 2006). NSSE data (2005) show that students who rate their advising as good or excellent are more satisfied with their overall college experience, and gain more from college in most areas. Yet, seniors in the classes of 2006 and 2008 at Pomona College and its peer group report that they were less than satisfied with their pre-major advising experience and, at Pomona (for which we have data), black students report greater dissatisfaction than do others.   For example, on a scale of 1= very dissatisfied and 4=very satisfied, Pomona respondents rated “academic advising before declaring major” 2.70 and “ academic advising in your major” 3.27.  Black Pomona respondents rated “academic advising before declaring major” 2.55 and “academic advising in your major” 2.96.

Given the level of dissatisfaction among black students and the less than stellar ratings given to pre-major advising by all students, we have begun to look for ways to improve our advising model.   An example of advising that appears to work well on our campus is the mentoring that happens through the POSSE program. Unlike traditional academic advising, POSSE mentoring pays attention to personal and social issues that often influence a student’s academic performance. We tracked the GPA trajectories of POSSE students compared to “control” groups of Questbridge, Latino, Black and lower income students and found that POSSE students showed greater resilience.  This evidence, coupled with the data from the focus groups of POSSE students and interviews with mentors, suggested a positive impact of holistic mentoring on the academic success of nontraditional students. This finding is corroborated in published research.  In a study of nineteen high-achieving black students at a predominantly white, medium sized research university in the Northeast, Douglas Guiffrida (2005) found that students described faculty as “student-centered” if they provided comprehensive advising – “student-centered faculty took a much more holistic approach to their career advising that went beyond simply giving students advice regarding course selections. Instead, they invested time patiently listening to students to understand their professional fears, dreams, and goals.”

We introduced the concept of holistic advising to all first-year faculty advisors at the Fall 2009 workshop.  The ensuing discussion revealed faculty discomfort because of a perceived crossing of boundaries and because they felt ill-prepared to respond to what they might hear.  Nevertheless, in a fall survey of first year students, 46% reported that their advisor asked questions about their lives outside of the classroom. We lack baseline comparison data, but suspect that is an increase as compared to earlier years.

As we ask faculty to inquire about the personal and social life of the student at the college, we need to provide them with more information about the student experience, particularly the experiences of students who may be marginalized at the college – first-generation, low income students and students of color.  We need to develop a sustained and effective method for communicating to faculty the experiences of these students at the college.

For many years, Pomona College has conducted student surveys that included open-ended questions on student satisfaction and the experience of diversity on campus. However, this qualitative evidence has not been systematically analyzed and organized to communicate to faculty.  Quantitative data, like the overall satisfaction index, is regularly posted on the college web page, but numerical data cannot reveal the depth and complexity of experiences in a way that would help faculty engage students more holistically in the advising process.

This research project will mine the open-ended questions on three surveys conducted over the last five years to create communications to faculty regarding student experiences with respect to diversity over this time period and to develop additional holistic advising guides for faculty utilizing this information.  The effectiveness of these communications and resources, their impact on faculty advising, and their (indirect) impact on student satisfaction with advising will then be assessed.

We hypothesize that the provision of qualitative evidence and additional training will encourage faculty to adopt a holistic advising framework.  A faculty member who has some knowledge of the prior experiences of historically marginalized students and the tools to guide his/her inquiry will feel more comfortable asking questions and providing advice on personal and social issues.  We also hypothesize that the adoption of a more holistic advising framework will lead to greater satisfaction with advising, especially among black students.

The first step will be an analysis of qualitative data, answers to open-ended questions, from two surveys of Pomona students:  a Senior Exit Survey in 2005 that included open-ended questions about campus climate and a survey conducted by the student government in Spring 2009. Because these surveys were not designed in concert and asked different questions, we will need to develop specific questions about student experience before we begin this analysis   Step two will be to translate this data into specific findings and narratives to be communicated to academic advisors.  We plan to complete this phase by late Spring 2010.   Also, in Spring 2010, we will collect additional baseline data through separate focus groups with first and second year students and faculty advisors.  In August 2010,  we will hold a training workshop for first year advisors to communicate this information and to discuss its relationship to the holistic advising framework.  The survey of first year students distributed this fall will be repeated in Fall 2010 for both first and second year students. In addition, we will conduct a second round of focus groups. An assessment of the impact on student satisfaction will await the Senior Exit surveys of 2013 and 2014.

The results of this research will affect the future design of pre-major advising training and will allow us to assess our ability to scale up the Posse model to a larger group of students.

BUDGET with justification – $6500

Personnel – $5000

Graduate student  ($4000) and undergraduate research assistants ($1000) will assist one-person Institutional Research office with analysis of open-ended survey questions and to prepare report, a labor intensive process.

Other Costs –  $1500

We will need to provide incentives to attract student participants in the proposed focus groups. ($1000) and there will be costs associated with production of materials for faculty workshop to disseminate results.  ($500)

Background of Investigators

The principal investigators are Cecilia Conrad, Dean of the College and Stedman Sumner Professor of Economics and Miriam Feldblum, Dean of Students and Professor of Politics.  The research team includes Jennifer Rachford, Director of Institutional Research; Marcelle Holmes, Associate Dean of Students and Assistant Professor of Psychology; and Lynn Thomas, Professor of Anthropology. Professor Conrad’s research focuses on the effects of race and gender on economic status.  She is the former Director of the American Economic Association’s Pipeline program and in that capacity designed a mentorship and networking program for minority graduate students in economics. She has active research on the climate for African American women faculty and delivered presentations on this topic at Ohio State University and University of California-Irvine.   Dean Feldblum has written on ethnic politics in France.  Before coming to Pomona, Dean Feldblum was the senior director for academic planning and support at Caltech. At Cal-tech, she was the PI on a grant from the The Gordon and Betty Moore Foundation, “Achieving Diversity in Science and Engineering.” Jennifer Rachford has conducted studied Black and Latino student performance in gateway courses and of interactional diversity at Pomona.  Marcelle Holmes is a psychologist who specializes in student and African American mental health. Lynn Thomas’ fields of expertise in anthropology include meaning in language and cross-cultural communication. He also teaches a course in methods.

Pomona College is committed to sharing the outcomes of this project with the larger CHAS community.




Guiffrida, Douglas, 2004. “Othermothering as a Framework for Understanding African American Students’ Definitions of Student-Centered Faculty,” The Journal of Higher Education, Vol. 76, No. 6 (Nov. – Dec., 2005), pp. 701-723.

Kuh, George D., Jillian Kinzie, Jennifer A. Buckley, Brian K. Bridges, and John Hayek, 2006.  “What Matters to Student Success: A Review of the Literature”, Commissioned Report for the National Symposium on Postsecondary Student Success: Spearheading a Dialog on Student Success, National Postsecondary Education Cooperative, July 2006.

Metzner, Barbara, 1989.  “Perceived Quality of Academic Advising: The Effect on Freshman Attrition,” American Educational Research Journal, Vol. 26, #3 (Autumn 1989), pp 422-442.

National Survey of Student Engagement (NSSE). (2005). Student Engagement: Exploring Different Dimensions of Student Engagement. Bloomington, IN: Indiana University Center for Postsecondary Research.

Tinto, Vincent, 2004. “Student Retention and Graduation: Facing the Truth, Living with the Consequences.” Occasional Paper #1, The Pell Institute for the Study of Opportunity in Higher Education, July 2004.