This is the conclusion of a two-part series. In the first article, Duellberg, Hansen and Swendsen explained the rationale for their study into whether graduation rates are appropriate as an institutional success metric. In this article, the authors highlight the most significant problems with using this measure.
We attempted to analyze discussion board interactions through multiple roles. We began with sets of random discussion board pulls and assigned scores for the highest cognitive skill level demonstrated based upon predetermined criteria. We compared the mean cognitive skill level scores across three graduation rate categories based on their reported IPEDS graduation rates. We had expected to find a strong correlation between lower-level cognitive skills and higher graduation rates, but this was not the case.
The discussion board posts with the highest cognitive skills displayed belonged to the schools with the highest graduation rates. Although we did find a significant relationship between the level of cognitive skills displayed on discussion boards and graduation rates, the relationship was not robust. We then attempted to interpret this data from the standpoint of three different institutional roles: administrator, instructor and student. We felt this would go some way toward minimizing the power differential in such interpretations and give it a broader interpretive scope.
We also want to mention the following issues with reported graduation rates:
1. Graduation rates do not account for all students
Graduate rates, by themselves, do not always measure success, as there are an increasing number of college attendees who “drop in” for a class or two in order to fill a void. Their intent was never to graduate, obtain a certificate, etc., but to gain information or a skill. Currently, there is no mechanism to account for this type of student.
2. High cognitive skills do not always translate into high graduation rates
Although there might be evidence of higher cognitive domain skills displayed, we can’t assume this translates into a more educated individual. Learning assimilated to a high cognitive skill level, but within a narrow band of specialization, may not optimally equip the student for either professional or personal success. Just as a biological species that eats only one type of leaf may not thrive under a diverse range of conditions, so the overly-specialized student may not be suited to thrive.
3. Push for degree completion leaves out certificate seekers
On the other hand, there is economic pressure to specialize, and highly-sought specialized skills can be demonstrated even with a certificate, rather than a more expensive degree. Yet the rules governing tuition assistance funding for military students restricts funding to degree seekers, causing many individuals to try to appear to be on a degree track, when their real motivation is only to obtain a certificate along the way. This has implications for interpreting graduation rates.
4. Lack of tuition assistance is damaging
Of course, another obvious reason for low graduation rates is based in economics. If no tuition assistance or Pell Grant funding is available, students may drop out because they are economically unable to continue. This is likely common and has less to do with institutional effectiveness and more to do with general problems in the economy and with the way education is funded in the United States. Even the Post-9/11 GI Bill, which provides financial support for education to military veterans, is only designed for 36 months (three years), although most students require more than 36 months to graduate.
5. Accurate assessment can be very difficult
One challenge is that assessment tends to be pass/fail for a given skill, whereas in practice, an adult student may partly demonstrate the skill, but not show competency in every single part of it. For example, let’s say the learning outcome is to correctly use MLA as a citation style. There are many facets to these entries, and too many to stand as individual outcomes. Therefore, the outcome being assessed is really a kind of “basket” of outcomes, in that it incorporates many individual skills such as correct capitalization protocols, correct use of a hanging indent format, correct use of italics and quotes where necessary, correct alphabetizing on the page, correct choice of the particular resource template for the entry from the MLA Handbook, etc. Very few students fulfill every single fine point of this outcome. So, at what point is the outcome achieved?
6. There is a very fine line between work and academic skills
Adult students are frequently working professionals. How can work experience be differentiated from “purely” academic skills, and to what extent should outcomes for adult learners involve work-relevant skills?
Though it is difficult to summarize all of these points at this preliminary stage of our role-related research, we are sure engaged students are motivated students, and engaged faculty are likewise motivated faculty. We believe there may be systemic principles in the distribution of roles and their relationship that contribute to overall graduate rates, and that role-related research has a place in institutional research.
– – – –
References
Selingo, Jeff. “The rise and fall of the graduation rate.” The Chronicle of Higher Education. March 2, 2012. Accessed at http://eab.state.wi.us/resources/performancestandards/ChronicleCompletion1.pdf
Glenn, David. “6-year graduation Rates: a 6-minute primer.” The Chronicle of Higher Education, December 6, 2010. Accessed at http://chronicle.com/blogs/measuring/6-year-graduation-rates-a-6-minute-primer/27573
You Might Also Like
