Traditional Ratings for a Non-Traditional Audience: Improving Relevance for the MajoritySandra Woodley | Senior Fellow, AASCU
Today, 74 percent of higher education students are non-traditional. They are adults, they are pursuing higher education part-time, they are working and they have dependents. However, the systems that rank and measure the performance of colleges and universities speak largely to the priorities and experience of the minority traditional student population. In this interview, Sandra Woodley shares her thoughts on the dangers in the discrepancy between available consumer information and today’s consumers, and reflects on how rankings could be improved to better speak to the needs of today’s students.
The EvoLLLution (Evo): What are a few of the benefits of the standard rankings published by groups like the US World and News?
Sandra Woodley (SW): It’s important to have some way to compare universities for students to make really good choices when they’re trying to determine where they want to get their degree. I think there are quite a few challenges to the traditional way of ranking institutions and there are efforts underway to improve them.
Evo: What are a few aspects of the work institutions do that these rankings miss, especially for institutions that serve non-traditional students?
SW: When you rank universities based on, basically, status—when you’re looking at how many students that you exclude from a low acceptance rate, for example, as well as tracking the standard graduation rate—it misses a lot of students, especially non-traditional students.
Non-traditional students like me end up being well served by institutions who focus on meeting the needs and expectations of that student population. You may have flagship institutions like the University of Texas at Austin, for example, which serves a large number of traditional students, is rewarded by the rankings for their selectivity, brings in students with higher ACT scores and enjoys a high graduation rate. But institutions like the University of Texas at Tyler serve a lot of transfer students and adult students who are getting back into the work force. Only 19 percent of their recent graduating class actually even showed up in the graduation rate and so it’s really important. After all the bachelor’s graduation rate only accounts for full-time, first-time students who graduate in six years.
One of the best articles written about this came from Malcolm Gladwell. He wrote an article about this for the New Yorker some years ago and he described it like the ranking for automobiles. He said you have a certain set of categories and weights on those categories for what you want in a car. So categories like affordability and good gas mileage would be important for a van, for example, when the buyer is looking for something to help them cart kids to soccer games and so on. But if you’re in the market for a luxury sports car, gas mileage is not the first thing on your list. His point was that, when it comes to ranking colleges, students need to look at the experience and the factors that best fit their situation. There’s no one-size-fits-all and, when you have rankings that look at the same categories and the same ratings of the categories, it misses a lot for students like me.
Evo: What impact can rankings have on institutional reputation and student demand?
SW: If students believe that some of these really great colleges are inferior because of the rankings, they’re going to miss out on a really great fit. There are wonderful colleges that serve different roles and scopes and missions that are not well captured in the rankings, so it’s really important to have a better way to categorize and look at institutions rather than the traditional way.
Evo: What happens to institutions that move away from serving unique groups and instead start pursuing rankings?
SW: Well the quickest way to improve your rankings is to only take students who are going to be successful. Taking that all the way out to a logical conclusion, it would drive institutions towards only taking students that have higher ACT scores. Now, we won’t be successful in the U.S. if we can’t increase the educational attainment of all populations of students including students like me who didn’t make the best choices in high school. We have to have an opportunity for a large number of students to be served and if you only chase the rankings then there will be a lot of students who will not have access to higher education.
Evo: What needs to change in order to minimize the number of institutions that choose the rankings pathway?
SW: Benchmarking is really important. When you’re looking at for example the UT Austins and the LSUs of the world, only a certain set of criteria is really important because it fits their mission. Comparing flagship institutions and looking at the data they give you a truer picture of how those institutions rank within their unique category.
But I think you have to find a way to statistically benchmark institutions that have different missions so that your comparisons are fair. In the University of Louisiana System, we use statistical analysis to find national peers for each of our institutions and then compare across a number of criteria. This gives a much better sense of institutional performance and provides a much better comparison.
Evo: What are the factors that should rank an institution higher when that institution’s focus is on creating accessibility pathways and serving the non-traditional audience?
SW: You can look at the success rate of students who come in needing remedial education, or bring lower ACT scores, or transfer from community college. You could take a look at how those students are served within the institutions. For example, you could ask, “What percentage of Pell Grant students go on to be successful at that institution?” There are a number of ways to look at success beyond graduation rates.
Graduation rate is really inferior metric for most of the colleges and universities in the United States because it captures the results of such a small population of students. Looking at degree production, retention rates and success rates of those non-traditional populations provide a better way to look at success for most of the institutions.
Evo: Is there anything you’d like to add about the danger of chasing rankings especially for institutions that have that alternate mission focus for serving non-traditional audiences?
SW: Beyond the rankings, overall statewide policy needs to change. We have boards and coordinating boards and management boards who tend to gravitate towards those more traditional metrics when it comes to defining and understanding the performance of public institutions and I think we have to do a better job at the state level of articulating the value of different kinds of institutions to serve different populations. This is a really important policy improvement that could create much more access for students that badly need it.
Author Perspective: Administrator