Where Completion Goes Awry: The Metrics for Success Mark Mounting Problems with Quality
Our nation’s future does indeed depend on developing all Americans’ talents to the fullest extent possible, but there is mounting evidence that college has become a “revolving door” for entirely too many students, at high cost to their hopes for the future and to society’s need for a well-educated citizenry.
The intense national commitment to increased college attainment needs to be matched by an equally intense focus on quality or, more specifically, on the kind and level of learning that degree attainment is supposed to represent. Completion ought to mean that students have demonstrated — cumulatively — their acquisition of the knowledge and skills they will need for the complex and fast-changing challenges of work, citizenship and contemporary life.
Unfortunately, the completion agenda is steaming ahead without setting either goals or markers for educational quality. As the authors in this issue of Liberal Education make patently clear, when we create incentive systems for enhanced degree production, with no questions asked about the sufficiency of learning, the door is literally wide open to choices that deplete educational quality.
As national studies on “high-impact practices” make clear, higher education has already invented a raft of learning-intensive programs and pedagogies that can simultaneously lift completion rates and student achievement of key skills such as writing, research and analytical reasoning. There are well-tested models for deploying technology in support of high-order student inquiry and achievement. The question is whether we are willing to use the knowledge we already have to create incentives for high-quality learning as well as incentives for increased degree production.
It should not be an either/or choice; we can do both.
In what follows, I offer my own observations on where the completion and productivity agendas have gone awry, and on how we can actually achieve those greater quality expectations.
The Credit Hour as a Proxy for Quality
The term “quality” is scarcely even mentioned in most of the abundant policy materials that frame the nation’s current efforts on completion and productivity. Even when “quality” is invoked, it is never defined. Policy statements at all levels solemnly promise that costs can be cut and graduation rates raised “with no compromise of quality,” of course. But without meaningful references points or incentives for the demonstrated achievement of quality, warnings against its erosion are empty at best. The real message seems to be “more degrees, more cheaply and quickly” with no questions asked about what the degree represents.
Without meaningful markers for quality, the real marker for progress on completion and productivity has become that hardy but hoary invention of the early 20th century: the course credit hour. Established in the heyday of the industrial assembly line, and given a significant boost forward when Carnegie Foundation for the Advancement of Teaching promoted it, the credit hour was designed to combine students’ time spent in class plus the time spent outside of class learning course-aligned material.
Freshened up and sometimes disaggregated (by student subgroup and/or income), the credit hour is now the marker of choice for reporting progress on persistence, retention to the next level, transfer of courses from one institution to another and, of course, degree attainment. It is our de facto substitute for quality. But as a proxy, it both protects and disguises subprime performance.
Employers are pleading with colleges and universities to build higher levels of American capability. Yet, credit hour production tells us only about “efficient through-put” and, when used in productivity analysis, the comparative costs of different degrees. Credit hours tell us absolutely nothing about what students are even doing in a course, much less about their levels of achievement. Does the course require extensive writing? Research? Projects? Team problem solving? Applied learning? Or just coasting and exam-time cramming? The federal government has recently blown the whistle on courses that meet too briefly for the credit hours awarded. But frankly, this is a small part of a much larger problem.
The credit hour is equitably awarded for high-impact learning and low-impact activity alike. It is our coin of exchange. When it comes to actual quality, however, the credit hour effectively hides rather than reveals. And yet, incentives for increased credit production — at lower cost — have become the strategy of choice for making America once again the “first in the world.”
Mounting Evidence on the Quality Shortfall
This association has insisted for a decade that the real key to economic opportunity and advancement does not depend on whether the student possesses a credential but rather on whether students actually leave college with that rich portfolio of learning that employers seek and society urgently needs: broad knowledge, strong intellectual and practical skills, grounded commitments to personal and social responsibility and demonstrated capacity to deal with complex challenges.
The facts are stark. As ACT annually reports, only one in four students who enroll in college is well-prepared to be there. Most need help to recover from an inadequate foundation, especially students who have been out of school for a while and need to strengthen basic skills that they may not have used in years.
While we would like to believe that college helps short-changed learners get up to speed, there is overwhelming evidence that many college seniors graduate still lack essential facilities that they need for work, for citizenship and for lives of continuous learning. These graduates — “successful” in their acquisition of the right number of course credits — possess credentials, but they have not achieved the kind of learning that a degree should represent.
Observers and leaders across the higher education space have been highly critical of postsecondary institutions and success in recent years. Colleges are “underachieving,” says former Harvard President Derek Bok. Students are making small gains or even declining on several key measures of learning outcomes, say the scholars doing a longitudinal study of student achievement through the Wabash Center of Inquiry in the Liberal Arts. Graduates lack global knowledge and are weak on key competencies such as writing and critical thinking, say employers surveyed by Hart Research Associates for the Association of American Colleges and Universities (AAC&U).
The “preponderance of the evidence,” as AAC&U’s own director of assessment puts it, shows us that many students do indeed complete college without actually achieving the high-level capacities and complex knowledge that a liberal — and liberating — education ought to provide.
The completion and productivity juggernaut pays no attention to any of this. At best, it kindly assumes that if students gather the right number of course credits, “success” has been achieved. At worst, productivity enthusiasts are looking for ways to accelerate “degree production” by focusing on narrow, technical training credentials tied to specific “labor market signals and needs,” while paring back or eliminating the broader education and higher-order intellectual skills that have long been the key to our world leadership in postsecondary education. This may help us increase through-put but it perversely defeats the entire project of helping Americans achieve higher levels of knowledge and capability.
What Can Be Done to Provide 21st-Century Quality Markers?
While there is good reason to sound the alarm bells about both the existing quality shortfall and the policies likely to worsen it, there also is reason for hope. Just a year ago, the Lumina Foundation released (in beta form for widespread experimentation and testing) the Degree Qualifications Profile (DQP), a 21st-century quality framework intended to provide the quality markers higher education so urgently needs.
The current content of the DQP reflects the views of the AAC&U community about the essential aims and learning outcomes — or, in DQP terms, competencies — needed for success in the economy and to contribute to a flourishing democracy. The DQP emphasizes the importance of broad as well as specialized learning for degree earners at the associate’s, bachelor’s and master’s levels. It calls for students to develop intellectual skills that faculty value and employers seek. It calls for students to integrate their learning across disciplines and to apply that learning to complex, unscripted questions. It argues that the actual test of a 21st-century education is whether students can apply their learning to new settings and complex problems. This is liberal education in contemporary form, albeit now a “rose by another name.”
Many worry, almost reflexively, that a framework of this sort will be constraining. But the framing of this document is intended to apply to a wide array of academic content and curricula. At the heart of the vision of the DQP is an insistence that there are multiple ways to develop and certify their achievement once we know what students are supposed to accomplish.
But how is DQP achievement actually certified? What I like best about the DQP is that it builds directly from the evidence-based research on the value of high-effort, high-engagement practices in fostering student achievement and supporting student persistence in college. The DQP places students’ work at the very center of the assessment equation. Projects, research, writing, performances and portfolios — course-based and field-based — are the centerpiece of DQP assessment.
How would we tie these DQP innovations back to the completion agenda? I believe the answer is to create incentives both for students’ timely completion and for the display of evidence of students’ demonstrated achievement in relation to Degree Qualifications competencies. We already know that students’ engagement in high-effort, high-impact work helps keep them in college. If we can link demonstrated DQP achievement — anchored in exactly the kind of student work that supports both persistence and learning — to the new completion and productivity incentives, we will produce the completion agenda “do-over” higher education urgently needs.
This piece is adapted from an article originally published by Schneider in the Winter 2012 Association of American Colleges and Universities publication, “Liberal Education.”
Author Perspective: Association