The EvoLLLution (Evo): What were some of the driving forces that led you to look at best practices for digital learning?
Lou Pugliese (LP): There were two main forces driving our study.
The first was that no one had done a comprehensive “deep dive” for the digital learning marketplace. All other studies had a more targeted focus on topics like online efficacy or descriptive statistics. We felt that there was a need for an overarching analysis that looked at the digital learning environment at multiple institutions over the long term to determine proven best practices.
To accomplish this, we triangulated three dimensions which determine whether an institution will be successful in scaling up their online environment: economic, organizational and operational. From there, we pulled a minimum five years’ worth of data from each of the six institutions we studied to get a longitudinal growth pattern for their online programming.
The second driving force behind our work was that our industry had no clear explanation for why some online community college programs succeed while others fail. The Gates Foundation, after having invested a significant amount of money in accelerated programs and thought leadership in this field, couldn’t point to why there’s such disparate success rates between institutions who’ve managed to scale and those who haven’t. When you look at the institutions we feature in the study, you realize that the community college market is very heterogeneous: Some have mainstream online practices, while others who started in e-learning with Blackboard back in 1998 are still struggling.
Evo: How do you define “scale” for an online provider?
LP: A scale is defined by a percentage of online offerings as compared to their face-to-face and on-campus counterparts.
Organizations often have their own internal definition of scale as well, which our study takes into account. It’s not always as straightforward as an organization saying, “let’s scale one program” – the institutions we chose have been very thoughtful about organizing scale in their own operations and have seen significant growth in their online programs as a result.
Evo: During the course of your research, was your team able to confirm or reject any commonly held beliefs about digital and online education?
LP: “Reject” is probably the wrong term. Rather, our study allowed us to confirm two commonly held beliefs about online and digital education.
Firstly, in order for a program to be successful, it’s critical that institutions empower cross functional teams. Each institution we studied showed consistency on this point: they had all figured out how to effect cross-functional change to meet their goals for online growth. The way they effected that change differed between institutions. Sometimes it involved physical reshaping, and often it involved administrative change. That’s unusual: postsecondary institutions are notoriously slow to change, and it can be extremely difficult to reshape cultural expectations and structures.
One other thing that we were able to confirm was the belief that institutions need to do more to create a pipeline to prepare inbound, pre-matriculation students for online study. That applies not only to mainstream students in their freshman year, but also to later-stage students who’ve had some academic successes and some failures. Our study showed that successful institutions had figured out ways to create a “prepared pipeline” approach to meet the needs of those later-stage students—that is, shaping programs in their early college years to position them for success later on.
Evo: One thing that’s come up a few times over the course of our conversation is the idea of a dedicated cross-functional unit that can lead the charge on creating an institution-wide rethink on online education. How common is it to give jurisdiction for developing new online programming to standalone divisions within the university?
LP: If an institution is able to get program-level or provost-level agreement on centralization, whether physical or virtual, it gains the ability to make decisions about the quality of instructional design. For example, at Arizona State University it wasn’t until we formed a centralized group representing five separate governing boards within the university that we were able to effect true change, operating as an integrated but independent unit. That’s a hard thing to do, but I believe that of all success drivers, that’s the most important one.
In this study, we set up a research lab to look at the efficacy of online programs. We studied 7.6 million course observations from 2011 to now, to get a strong sense of the indicators of predicted student success for how these courses are designed and delivered. Our lab sits right next to an instructional design team, which allowed us to connect our research to a series of random control trials without changing the instructional design. This enabled us to look at predictive success factors so that we could dig deeper into why instructional design works in some cases but doesn’t in others.
We found that having some autonomy and degree of control over purchasing, decision making, and administration is a key condition for success.
Evo: In your report, you identified seven promising practices as key to success in developing online programs. Of those, which strike you as the most important?
LP: The first practice I want to mention is the need for centralizing and consolidating data to facilitate continuous improvement. The data environment for online programs is very challenging. My team had to become forensic scientists because relevant data existed in so many disparate places across the institutions we studied. For me, the lesson learned is that each individual institution needs to develop its own organized method for evaluating the success of its online programming.
At ASU, we are building out that structure by creating a learning analytic centre that brings together disparate data from multiple sources, including LMS, Peoplesoft, and financial aid. We believe that this learning analytic centre will allow us to collect and use data more effectively. That’s a difficult prospect for many institutions, because you can’t go off the shelf and buy a data system that will solve every single problem. You have to piece together a system that will result in the right solution for the right university. If institutions don’t start doing this now, they’re not going to be able to improve online programming moving forward.
Evo: Why is it so important for senior institutional leaders to invest in consciously growing their digital learning presence?
LP: There are a number of reasons why such a focus makes sense for senior administrators.
First, such an analytical focus on online growth will result in organizational growth as a whole. By increasing enrollment numbers through online platforms, institutions can build their student base without having to physically grow their campus.
Second, scaled implementation in online programs are cheaper to run on a student-per-credit-hour basis. By effectively employing data regarding online program growth, institutions can hit the “sweet spot” in the margins on delivering quality education to new audiences at a profit.
Third, studies show that online instruction provides a blended learning environment that yields greater student success. Our findings show that students working in both mediums—online and on campus—reap the benefits of both models. They can study much of the classroom content in advance, freeing them to focus on more abstract learning when they enter the classroom. The homogeneity of those two modalities works.
Finally, the demographics enrolling in higher education are changing. Today, 16- to 23-year-olds make up approximately 15 percent of the entire learner population. The amount of diversity that we’re trying to deal with today is radically different from what it was ten or fifteen years ago. We believe that an online modality serves this diverse student population as well as, and often better than, an on-campus modality. In fact, in some areas students actually perform better in an online environment, and I believe that’s down to the demographics.
Evo: Looking at the next five to fifteen years of online education growth in the postsecondary sector, what impact are you hoping to see your research have on the practice of developing and scaling a digital online learning presence?
LP: That’s a question of design and history. We’re currently at a junction point in terms of online education: We’re transitioning from technology tools that were built in the early 2000s and which shape our current online environment, to more advanced technologies that can leverage adaptive learning in new and exciting ways. Today’s technology allows us to shape the learning experience today in ways we couldn’t imagine 15 years ago, and that’s promising because we have a much greater ability to tailor a learning experience to the unique needs of an individual student. That’s a good spot to be in, after having studied this field for 20 years.
Evo: Is there anything else you would like to add about Making Digital Learning Work, and the impact you hope to see it have on higher education?
LP: Administrators shouldn’t look at this as a silver bullet. There are sections of this report that apply to certain functional areas, such as finance or enrollment management or functional design. Institutions have to look at the report accordingly, and apply it accordingly.
This report is intended to start a conversation between institutions who are seeing success and those who want to interpret our results and apply them within their own institutional context. Administrators should look at this study and ask, “What’s relevant in this report to my institution?” It’s a method to inspire experimentation, and encourage institutions to try these different kinds of tactics and strategies. That’s what I’m hoping this will accomplish.
This interview has been edited for length and clarity.
You Might Also Like
