The following interview is with Sandra Woodley, president of the University of Louisiana System. Woodley, who was herself a non-traditional student, has been one of the leading voices in the discussion around ensuring that institutional performance metrics take into account the work and achievements of non-traditional learners. In this interview, Woodley expands on that topic, shares her thoughts on how institutional performance would be best measured and discusses some of the dangers of going about performance tracking and performance-based funding the wrong way.
1. What is wrong with the metrics currently used to assess institutional success?
When you’re looking at measuring success of students — particularly in those institutions who serve large numbers of non-traditional students — benchmarking becomes very important.
When you look at the most common metric — first-time, full-time freshmen graduation rate — many of our colleges that do herculean efforts to help non-traditional students, many of them have a very low cohort of those first-time students. In other words, these are institutions who are serving students and that doesn’t get picked up in the metric at all.
One of the strategies I’ve employed in my recent history — particularly in Arizona, Texas and now here — is one of benchmarking using statistical peers. … We looked at our institutions and we’ve developed this statistical model that groups other national universities with them, based on role, scope and mission. I want to compare an institution to those who have similar kinds of students. … When I look at any measure that I need to use, then I’m comparing that particular institution to others who face similar challenges.
In this way we can look at even our current metrics that give us some problems, but when we benchmark it appropriately, it makes it more of a fair fight.
2. Looking at President Obama’s proposed college ranking system, how would non-traditional students be affected if those metrics were put into place?
I’ve had some conversations with members of the Department of Education on this very topic, and I think I’ll go back to my earlier statement: sometimes it’s not about the metric; it’s about how you benchmark the expectations of performance. …
I think there’s a way to use this methodology to benchmark performance so that the baby doesn’t get thrown out with [the] bathwater. It remains to be seen how the President’s proposal will work out for non-traditional students. But I think one way to minimize the problems is to use some kind of benchmarking that will allow for us to make better comparisons.
3. At a larger scale, how are institutions that mainly serve non-traditional students impacted by performance-based funding models?
Well, I’ve done a lot of work over the years in looking at performance-based funding; in fact, my dissertation was on this very topic, performance-funding and performance budgeting models.
It is really important, when we do look at performance at all of the institutions, to group them appropriately. That kind of methodology can cure a lot of ills. In Texas, for example, one of the proposals we’re working through now on proposal funding revolves around the work we’ve done recently on these statistical peers. “How can I compare performance at each one of my institutions, relative to their own best-in-class?” This gives us a way to allow the institution to compete on the correct playing field and to get credit for the hard work they do.
When you look at institutions who serve large numbers of non-traditional students, sometimes that’s a much tougher job than an institution who does very good work, but is much more selective and attracts students who are going to graduate in four or six years anyway.
It’s all good, we want more of all of it, but I think it’s really important for us to make sure we don’t have perverse disincentives for many of our colleges and universities to take those students who may be struggling and find a way to get them to completion.
The other point I would make is, in these performance systems, it’s really important to focus on not just the traditional metrics but to look for a broader mosaic of metrics that looks at every single aspect. For example, if I’m looking at an institution and I want to look at their students’ success, one of the good ways to do that is to look at how many degrees they produce given how many students they have in the pipeline. This metric is a through-put metric that does look at their ability to move students through to graduation and is less of a penalty than looking at a small cohort. … We have to look at ways to count graduates that aren’t normally counted in some of these more traditional methods and to use benchmarking to make the comparison fairer.
4. What other performance metrics should be put into place to assess institutional success while taking non-traditional students into account?
I like ‘time to degree’ too. It’s a different metric than just looking at your cohort metric, which is your traditional graduation rate, or your through-put metric, which is, “How many degrees do you produce given how many students you have in the pipeline?”
The other one is to look over time at, “Are we shrinking the amount of time it takes generally for our student body to get a degree?” It is true many students are like me and they took a long time to get a degree, and some of that is avoidable and some of that is not avoidable. But I think it’s fair to say that all of our institutions, even those that serve non-traditional students, are trying to find a way for students to navigate much more quickly … and also be able to lower the cost. The quicker you can get out, the lower the cost in the end.
The other point is these are not your traditional graduation rates — the IPEDS [Integrated Postsecondary Education Data System] definition that only looks at those who started in your cohort and who graduated at your institution. Within state systems who have better unit record data, we are able to look at a composite graduation rate, for example, that allows you to look at [successful transfer students and recognize] … the institutions that help them on the front end and the institutions that were able to help them on the back end.
This is especially crucial when you’re looking at transfer between community colleges and four-year institutions. I’m not unique in the sense that I swirled; I did some of my credits at the community college, I moved, I had more than one four-year institution that I went to. Sometimes students take classes at more than one place at any given time.
Our data systems and our willingness to dig a lot deeper than the easy statistics to get would be time well spent because we want to be able to reward our community colleges, for example, for helping our students to get remediated and, then, even if they only stay a semester or two, if they move on somewhere else, anywhere, and they go on to get a degree, then both of those institutions participated in the success of that student. In our normal metrics, they both do not get credit for that.
5. In the benchmarking scenario, what steps could or would need to be taken in order to ensure institutions are still trying to push toward improvements when they are being compared against their peers as opposed to being compared against institutions serving different audiences?
The goal of any accountability system, whether it’s performance funding or accountability systems, should be, “How do I hold the institutions accountable for that part of their performance that they actually do have the ability to change?”
That’s the trick; trying to make sure you don’t punish an institution for doing what you really want them to do just because of the metrics, and not giving them more credit than they deserve because of the manner in which, maybe, you picked your statistical peers.
It’s really important in our university systems to be honest brokers, if you will, about looking at performance and being able to show the bad with the good and understand that we do need to be able to improve in every single way that we can. We should be able to not be afraid of showing our failures or the things that are less successful than we would like them to be. If we do that and we’re honest about it, and if we have a multifaceted approach, then we can gain additional credibility with the main goal being actually to improve our performance.
6. Is there anything you’d like to add about the importance of making sure that any implemented performance metrics take into account non-traditional students?
Well, I think the President’s proposal on the college ranking system brings up a really important point. All of these states and the federal government and state systems who are interested in developing these systems are well-meaning. They are interested in students and they want students to be able to progress. They’re interested in pushing performance for a good policy reason.
It’s also true that we have to be very careful about unintended consequences; even with the best of intentions, it’s very easy to do this badly and not mean for it to be done badly. … If we can continue the dialogue and look at these thoughts and recommendations that will allow us to minimize the unintended consequences that could come from even a well-meaning approach, I think we would all be better served.
You Might Also Like
