Can Competency-Based Assessment Move Us from Credit Hours to Competencies?

It seems simple enough: Just change from credit hours to competencies. It seemed simple to Sally Johnstone, Peter Ewell and Karen Paulson when they wrote Student Learning as Academic Currency for the American Council on Education (ACE) in 2002 and updated it in 2010. It seemed obvious to Amy Laitinen in the landmark Cracking the Credit Hour in 2012. It seemed likely (someday) to the Carnegie Foundation in The Carnegie Unit: A Century Old Standard in a Changing Education Landscape (whose advisory panel included Johnstone, Ewell and Laitinen) in 2015.

But more than 16 years after Johnstone et al. felt that academia “may be on the verge of an evolutionary change,” the change has yet to happen. Why? Because it’s not a simple shift.

The credit hour, also known as the Carnegie Unit, was established by the Carnegie Foundation over a century ago to indicate student readiness for college. Each unit indicated an amount of instructional time in which students were exposed to subject matter. It has never measured what students learned—it wasn’t intended to. At the time, this simplicity that made it the building block of modern American education. Today it undergirds everything from daily school schedules to faculty workloads, federal financial aid eligibility to graduation requirements. It is “a common currency that makes possible innumerable exchanges and interconnections among institutions” (Silva et al., 2015, p. 5).

Former Carnegie Foundation President Lee Shulman summed up the challenge thusly: “There is nothing simple about measuring the quality of learning. The reason for the robustness of the Carnegie Unit is not that it’s the best measure, just that it’s much more difficult than folks think to replace it.” Despite the difficulties, the US Department of Education proposed a reconsideration of the credit hour in Winter 2018. Clearly, the time to talk about changing from the credit hour to competencies has (again) arrived.

The idea of using a measure of learning instead of a time-based proxy is not novel. In 1984, the Involvement in Learning report called for colleges and universities to “establish and maintain high standards of student and institutional performance” (National Institute of Education, 1984, p. 3).

Twenty years later, in 2005, then-Secretary of Education Margaret Spellings created the Commission on the Future of Higher Education, which reiterated the call for institutions to measure student achievement and report “meaningful student learning outcomes” (US Department of Education, 2006, p. 4). The criteria for such outcomes were articulated by the Council of Regional Accrediting Commissions, which released a set of guidelines for institutions around specifying what students should learn by:

1. Setting clear learning goals, which speak to both content and level of attainment
2. Collecting evidence of goal attainment using appropriate assessment tools
3. Applying collective judgment as to the meaning and utility of the evidence
4. Using this evidence to effect improvements in its programs

Nevertheless, it is increasingly evident that “the flexibility of the credit hour concept has reached its limit” (Johnstone et al., 2010, p. 4).  Even Carnegie concluded that “relying on the Carnegie Unit to measure the time that students spend in courses as a proxy for student progress has resulted in insufficient attention to what students are actually learning, or not learning” (Silva et al., 2015, p. 14).

As a new regulatory window opens, we can agree that change is needed. Nevertheless, we need to ask ourselves: Are we in a better place to consider changing academic currency than we were in the past?  Has the work of groups like AAC&U, NILOA and the Competency-Based Education Network (C-BEN) made a conversion more likely?

Indeed, movements like AAC&U’s Liberal Education and America’s Promise and Valid Assessment of Learning in Undergraduate Education, in combination with other thought-leading work such as Lumina’s Degree Qualifications Profile, have broadened and deepened the postsecondary industry’s ability to describe and measure proficiency in academic performance. In particular, the work of the National Institute for Learning Outcomes Assessment (NILOA) at the IUPUI Assessment Institute and the Association for the Assessment of Learning in Higher Education continues to build understanding and drive the ability to measure the unmeasurable nuances of postsecondary education.

The CBEN Quality Framework for Competency-Based Education Programs and the attendant Quality Framework User’s Guide have increased the field’s understanding and ability to manage competency-based education (CBE) effectively and with a high degree of quality assurance. The Experimental Sites Initiative (ESI), launched by the US Department of Education in 2014, has also helped to explore and address (if not completely solve) issues of financial aid and credit hour accounting for direct assessment CBE programs. Working with the Southern Association of Colleges and Schools Commission on Colleges (SACS-COC), a group of states established accreditation guidelines for CBE programs that did not rely on direct assessment.

What is evident from all this thought and work is that two essential developments are needed: The decoupling of administrative uses from academic uses of the credit hour; and a shift from measuring seat-time to assessing competency.

While the Carnegie Commission identified areas where the Carnegie Unit was impeding innovation, they also pointed out that “suddenly eliminating the Carnegie Unit would make it very difficult for institutions and students, educators, and administrators to function efficiently … Whatever challenges the Carnegie Unit may pose, in its absence there would be no common language to organize the work of schooling and … no one understands this challenge better than the school and college administrators who rely on the Carnegie Unit to manage institutional finances and student records” (Silva et al., 2015, p. 30).

That said, institutions are finding creative ways to overcome this challenge. In a growing number of cases, institutions are keeping credit hours to manage administrative (and financial aid) processes, then mapping or aligning those hours to assessments of academic competencies. Western Governors University, Capella University and Southern New Hampshire University have all instituted competency models while managing their administrative processes, complying with federal requirements and generating traditional transcripts. These sorts of dual-reporting systems are also emerging in the K-12 sector as well. So, this decoupling of the administrative and academic uses of the Carnegie Unit can now be done and “does not tear asunder the current model of academic accounting with respect to either credentialing or cost” (Johnstone et al., 2010 p. 13).

Pivot from Measuring Seat-Time to Assessing Competency

The most adventurous model of CBE is direct assessment, where students work through projects using a variety of learning resources and activities, and take assessments at the end of each unit to determine if they have mastered the competencies needed to pass the unit. As Laitinen noted in Cracking the Credit Hour, “if crafted well, direct assessment could open space for high-performing, innovative institutions and accreditors to create a better model for how we measure and pay for learning” (p. 21).

What the direct assessment model demonstrates very clearly is that the measurement of learning is deeply dependent upon the quality and validity of the assessments. Moving from clock hours to an assessment of learning will require that we confront an awkward truth in teaching: Since only the faculty member who taught the course knows what was taught, only the faculty member can devise the assessment to validate what the student learned. When the credit hour was time-based, this variability was acceptable. But when financial aid will be paid based on competencies achieved instead of time spent learning, the assessment requires increased evidence of causality.

As Carnegie notes, achieving this trusted proof of learning will “require standards that clearly define rigorous expectations and serve as the basis for equally demanding assessments that reveal students’ actual learning” (Silva et al., 2015, p. 14).  One tool will be rubrics, which are an increasingly popular way to increase rigor and consistency across assessments. They are essential to objectifying and building trust in CBE assessments.

The central concern we need to confront is our (entirely justified) fear of standardized testing. Part of this comes from confusing standards with standardized.  While every postsecondary education program and course has a consistent set of learning outcomes (standards) a student must achieve to graduate, every course and instructional model need not be mandated (standardized) to be exactly the same. “Standards,” as Rob Able, CEO of IMS Global, has said, “are the key to agility.”  Having consensual, fixed learning outcomes and valid assessments that accurately measure and assure mastery, and map back to those intended outcomes, opens the door to much more flexible and transparent faculty innovation than our current models support.

Transparency is different from common—and standardized—as exemplified by the Transparency in Learning and Teaching in Higher Education (TILT Higher Ed) program that helps faculty design and implement a transparent teaching framework. It promotes a student’s conscious understanding of how they learn and helps faculty to gather, share and promptly benefit from current data about students’ learning.

Where Do We Go From Here?

The idea that moving from time-based to competency-based measures of assessment will provide a more transparent and accurate record of what a student has learned has been on the table for more than twenty years. We don’t need to change everything—we need to change the right things. I contend that our proven ability to separate the administrative from the academic uses of credit hours will enable us to finally move our academic measuring stick from the convenient to the competent.

As the Carnegie Commission stated: “As these new systems prove themselves, the reliance on the Carnegie Unit as a proxy for student learning may begin to diminish. In this context, the Carnegie Unit today is more of a bridge to the future than a barrier from the past” (Silva, et al., 2015, p. 33).

– – – –

References

“Negotiated Rulemaking Committee; Public Hearings.” Federal Register (July 31, 2018). https://www.federalregister.gov/documents/2018/07/31/2018-15929/negotiated-rulemaking-committee-public-hearings

Johnstone, S., Ewell, P., & Paulson, K., (2010). American Council on Education, Center for Policy Research and Strategy, One Dupont Circle NW Washington, DC. https://www.acenet.edu/news-room/Documents/Student-Learning-As-Academic-Currency.pdf

Laitinen, A., (2012). Cracking the Credit Hour. New America Foundation, Washington, D.C.. https://static.newamerica.org/attachments/2334-cracking-the-credit-hour/Cracking_the_Credit_Hour_Sept5_0.ab0048b12824428cba568ca359017ba9.pdf

National Archives and Records Administration, “Department of Education: Program Integrity Issues,” Federal Register 75 (209) (October 29, 2010): http://www. libraries.iub.edu/index.php?pageId=2558.

National Institute of Education. (1984). Involvement in Learning: Realizing the Potential of American Higher Education. Washington, D.C..

Regional Accreditation and Student Learning: A Guide for Institutions and Evaluators, (2004). Council of Regional Accreditation Commissions

SCRIP http://www.uncpress.org/book/9781469641928/proceedings-of-the-unc-cbe-summit-2017/

Silva, E., White, T., & Toch, T., (2015). The Carnegie Unit: A Century Old Standard in a Changing Education Landscape. Carnegie Foundation for the Advancement of Teaching, Washington, D.C. https://www.carnegiefoundation.org/resources/publications/carnegie-unit/

US Department of Education. (2006). A Test of Leadership: Charting the Future of U.S. Higher Education. Washington, D.C.