The Business Needs Drive Training EvaluationScott Dade | Program Manager, United States Department fo Defense
Determining when a training solution is the right solution to a performance problem is at the very heart of what we do as learning professionals. Being able to clearly articulate the link between business needs, performance gaps and the impact of the learning solution is where we prove our value as professionals to our business partners. But how do we do that?
It Starts with the Business Need
Our process starts with the initial request for training, when a perceived deficiency in performance is identified. By analyzing performance deficiencies around training development, we identified performance gaps of training professionals in the bank related to developing training solutions to solve specific business goals.
Dana Gains and James Robinson developed their well-known Performance Relationship Map (PRM) as a tool for doing up-front analysis. We used the PRM to clearly identify the business need, desired state and performance gaps within a large U.S. bank to develop a blended learning solution later called the Instructional Design Foundations course. With this approach, the desired state of performance was first identified. Then the current state of performance was identified and the difference between these two states, known as the performance gap is identified.
The cause of these gaps was first thought to be solely due to knowledge, skill or ability deficits when, in fact, there was a number of contributing factors to the performance deficiencies. Since a training solution is best used to support increased capabilities in knowledge, skill and ability, it had to be determined that there truly was an actual gap in required knowledge, skill or ability of a sufficient number of associates to justify the development of the course. This process and the resulting information helped us obtain senior leader support for the training solution and helped to have clear and measurable outcomes to measure our success against.
Linking Needs, Solutions and Evaluation
Once the business need is properly identified, objectives are developed to be observable, measurable and linked to identify business needs through performance objectives. In our case the need was identified as a lack of knowledge and skills around performance needs assessment, instructional design and training evaluation. Participants’ preferences or learning styles were also considered when developing learning objectives and helped us tailor the e-learning approach to fit associates learning needs that meant anytime, any-place learning from their own desks. This also helped facilitate the transfer of the learning to the actual workplace.
We used electronic pre- and post-assessments built around the module objectives for both the associate and the associate’s manager. An electronic reaction survey was also used to capture the associate’s reaction to the course immediately upon finishing the learning modules. To determine behavior change on the job and business impact we structured the post assessments to report use of the knowledge and skills on the job 90 days after completing the training and asked both the associates and their managers about the business impacts they had achieved in their workplace that could be directly attributed to the training.
The graphic above illustrates the link between the needs analysis we completed using the PRM and training evaluation plan we implemented using pre and post assessments.
As shown on the left, the value proposition of the business problem begins the process of determining the business need, which in our case was that trainers throughout the bank had varying degrees of knowledge and skill in using performance consulting, instructional design and training evaluation. This helped identify the behaviors required to achieve the desired state. The desired state was that trainers should be better able to link training solutions to business needs and identify the behaviors that needed to be developed to address the business need. Training professionals throughout the bank were not consistently using industry best practices to develop solutions that would enable their business partners to increase efficiencies, decrease time to market and reduce costs.
To change behavior, new knowledge and skills needed to be learned and shared across the bank. The Modules within the course were built around useable chunks of information and case studies to drive home knowledge and understanding. To maximize their learning, the preferred learning styles of the associates were addressed by making content bank relevant and accessible from their desks, eliminating time away from work.
This also proved to be very popular with both the associates and their managers, which were captured during the post training surveys and follow-up interviews. The link between training objectives and business needs must be considered at the beginning, instead of at the end of a program, when it becomes an after-thought. If we don’t link the solutions to business needs, we risk becoming irrelevant and expendable in the eyes of our business partners. The up-front analysis determines whether a training solution is required, what type of performance solution is best and the level of evaluation suited to the solution and the business needs.
Implications for Practitioners
As training professionals, we must maintain a business focus, analyze performance gaps to be sure that training is the best solution and ensure that any training solution is linked to performance analysis and identified business needs. In less than 6 months, participations in the self-paced voluntary training course exceed our 12-month participation goal and continue to have great success and support in the bank. The end of course reaction survey consistently provides greater than a 4.0 using a 1-5 Lickert scale in all major categories. The amount of learning between pre-assessment and post assessment scores resulted in an average gain of 30% or more. The pre and post assessments have shown associates are using their new knowledge and skills on the job immediately or within a week of completing the course. The business impact has averaged 20% or higher after being isolated and reduced by confidence factors from associates’ and their managers’ responses.
The business impact shown to have the greatest positive impacts was quality, efficiency, reduced re-work, and associate satisfaction. The ability to capture this data has created a “chain of impact,” as Jack Phillips refers to it, which has enabled us to calculate a 115% Return-on-Investment (ROI). The ROI was achieved after collecting the fully loaded costs and the benefits of the training solution and inserting the numbers in the ROI formula: ROI = Net Program Benefits/Program Costs X 100 = 115 %.
The evaluation results we achieved were linked to the work we did up front in the performance needs analysis which drove the performance based objectives. These objectives were broken down into learning objectives from which the content was derived. This process was then reversed to determine the level and types of evaluation we used to capture reaction/satisfaction learning, application/behavior change and all the way through business impact and ROI. These concepts are not new, but it is even more critical today to show our value as more and more training jobs become candidates for outsourcing.
We need to be able craft performance improvement solutions that solve business needs in a cost-efficient manner. If we don’t, someone else will.
Author Perspective: Employer