Understanding Health Metrics in Higher Education

What began as the program health metrics at South Central College (SCC) evolved into the health metrics over a yearlong process that began in fall 2018 as an idea and a list of possible measures and operationalizations. Over the course of a year, these measures were refined in partnership with two faculty taskforces. The result is the Health Metrics that SCC leverages today.

Developing health metrics at South Central College correlated with redefining program review. As the faculty and administration worked to redesign the program review process, a four-year cycle that culminates with program faculty and their dean presenting their findings to Cabinet, the conversation began focusing on what additional data were available and what else should be included as aspects of program review.

Once this conversation began, faculty and administration believed that standing up a cross-functional team in the form of a taskforce would be the best institutional approach to address these questions. The first taskforce focused on creating a shared understanding of the available data, building consensus relative to the operationalization of the various measurements and developing confidence in data collection and entry methods. As an outcome of the first taskforce, more than 40 possible measures and various weighting schemas were developed. Developing these possible measures and weighting scenarios led to requesting a second taskforce charged with shaping the total number of measures included in the dashboard, finalizing how and which measures would be weighted and determining how the health metrics would be used.

Both taskforces were chaired by the Vice President of Academic and Student Affairs, the Vice President of Research and Institutional Effectiveness/Dean of the Faribault Campus, and the faculty union president. The second taskforce met weekly during summer 2019 and presented the final version of the health metrics to faculty and administration during the 2019 fall in-service. The key to this taskforce’s success rested in developing an understanding of how the health metrics would be used. The concern expressed most was: “What will this mean for a program/faculty if they need monitoring or immediate attention?” In other words, building trust that the administration wasn’t going to close programs or terminate faculty based solely on the health metric score was the key element to a successful implementation.

To assuage this concern, the administration relied on its track record and its belief in and reliance on shared governance. Additionally, the administration highlighted how action was denoted relative to program, developmental subject or transfer subject score. Based on its score, a program, developmental subject or transfer subject either needed no attention, monitoring or attention. The administration also stressed that attention could take many forms, ranging from increased marketing to additional resources. Lastly, the administration stressed that health metrics were part of program review and that program review was a continuous improvement process.

The health metrics dashboard visualizes 25 measures for programs, 16 measures for developmental subjects and 15 measures for transfer subjects. Measures like achievement gap and professional development are visualized and weighted in all three categories. However, measures like momentum, defined as the rate at which students complete attempted credits, are only visualized and weighted for programs. Of the 29 measures visualized on the dashboard, twelve are weighted in the program score. Of the 16 measures illustrated for developmental subjects, nine are weighted. And of the 15 measures highlighted for transfer subjects, eight are weighted for the overall score.

The measures aggregated comprise four broad categories. Student success comprises 30 points of the program and developmental subject total possible score and 20 points of the transfer subject total score. Institutional core competencies—SCC’s nomenclature for program learning outcomes—program assessment and enrollment are each worth 30. Professional development accounts for the final 10 points. Currently, the maximum score for programs and developmental subjects is 85 points and 75 points for transfer subjects, as the 15 points that will be awarded for program assessment are not currently included in the database.

A program or developmental subject is considered needing no attention if its overall score is greater than or equal to 63 (75% of the maximum score). If the program or development subject overall score is between 42 and 62 (50% to 74% of the maximum score), it is considered in need of monitoring. If the overall score is below 42 (less than 50% of the maximum score), the program or developmental is considered in need of attention.

For transfer subjects, the current maximum score is 75 points. For these subjects, if the score is greater than or equal to 56 (75% of the maximum score), the subject is considered doing well. If the score is between 37 and 55 points (50% to 74% of the maximum score), the subject is considered in need of monitoring. If the overall score for a transfer subject is 36 points or lower (less than 50% of the maximum score), then the subject is considered in need of attention.

Takeaways

1. Engage faculty from the beginning. 
   1. Faculty know programs best and will be invaluable in helping to develop meaningful metrics.
2. Start early. 
   1. The implementation of South Central College’s Health took almost a year to complete. 
3. Allocate resources.
   1. Provide release time or stipends to faculty. 
4. Define goals.
   1. Honesty and transparency are the best policy. 
5. Follow through.
   1. Health metrics are a two way street. Some programs lose and others gain be true to both commitments.