By Dr. Shirley J. Caruso, Ed.D.
This article places emphasis on measuring the value of training, or training evaluation, in terms of its contribution to business goals. It introduces Impact of Learning (IOL) as a simple, three-step process that organizations can follow to evaluate employee development programs. IOL equips chief learning officers (CLOs) with a mechanism for providing evidence of a training program’s success by demonstrating the value of learning rather than measuring its return on investment (ROI), or the financial benefits of learning.
Training Evaluation is a Systematic Process
According to Phillips (1997), “Evaluation is a systematic process to determine the worth, value, or meaning of an activity or process.” (p. 36). Crucial to the evaluation strategy is the determination of what to evaluate. Exactly what should be evaluated depends on the type of training program, the organization, and the purpose of the evaluation.
One thing is certain; adult learners are a credible source from which evaluation of training stems. Their reactions to training are solicited, they are tested, and they are even observed to determine the merit of a given training program. IOL is a three-step process that organizations can follow to evaluate employee development programs. Insight, Individual, and Impact comprise the three basic steps to the IOL process. Naughton (2008) suggests an insight map be created to visualize what needs to happen and how and when success will be measured. The second part of the process is to listen to statements of the learners as they provide feedback on the alignment of the training with the goals of the organization. Thirdly, an impact statement is made as soon as learning outcomes can be seen in the organization. The impact statement should include at least two cases that demonstrate the impact of learning and refer back to the original insight map.
IOL promises connection to business objectives while producing quicker and more persuasive evaluation results that lie within the credibility of the learners themselves. This connection is important to determine whether or not the learning intervention is strategically aligned with organizational goals before training is implemented.
Kirkpatrick and Phillips Models of Evaluation
When taking a comprehensive look at the Kirkpatrick model and the Phillips model for purposes of comparison to the IOL model, the Kirkpatrick and Phillips models mimic a check and balance system. If the learners did not learn (Level 2), reactions to the training compiled at Level 1 (Reaction) reveal the barriers responsible for the lack of learning. If the learners did not apply the knowledge and skills (Level 3), the knowledge and skills may not have transferred to begin with (Level 2). The difficulty and related cost of conducting an evaluation based on the models of Kirkpatrick and Phillips increases as evaluation at each level increases.
The Difficulty of Measuring ROI
Phillips (1997) and Naughton (2008) are in agreement that most organizations view the process of measuring ROI as requiring too many resources. It is perceived as difficult, time-consuming, labor intensive, and expensive. Nevertheless, if the worth of the CLO and the training function are to be demonstrated, some process must be utilized to establish the value of learning at the corporate level. Evaluation determines the worth, value, or meaning of a training program, and whether or not and to what degree evaluation is performed determines the worth, value, or meaning of the CLO. CLOs would be jeopardizing their value to an organization if they are not equipped to provide the heads of an organization with evidence of a training program’s success. The IOL model factors out time and cost as barriers to measuring ROI , and the CLO is left with a means of justifying his/her existence while simultaneously determining the merit of the training function.
Summing It All Up
The IOL model offers CLOs a win-win situation. Organizations are provided with the data they need, and CLOs spend less time and resources providing the data, all while the CLO has justified his/her existence. Additionally, placing value on the learners creates a climate of self-esteem, and the more the learners are involved in the training program, the more likely its implementation is a success.
References
Naughton, J. (2008). IOL: Determining the impact of learning. Retrieved July 29, 2010 from http://www.clomedia.com/includes/printcontent.php?aid=2343
Phillips, J. J. (1997). Handbook of training evaluation and measurement methods (3rd ed.). Houston: Gulf Publishing Company.
Evaluation is the last step in the ADDIE model; however, evaluation should be an outgoing process throughout the entire training process. Sometimes, the organizations consider evaluation process to be time consuming, expensive, and even unnecessary. However, the Impact of Learning (IOL) model provides the organizations with the essential information, and the Chief Learning Officer does not have to spend too much time on collecting the data. IOL model emphasizes the value of the learner and produces a quicker and less expensive evaluation.
In my opinion, ROI is the most overlooked metric by organizations and ultimately proves to be fatal for many organizations. Training evaluation by way of the IOL model ensures the proper feedback is given and changes can be implemented to ensure learning is not only effective but productive ensuring a return on the investment which translates to a profitable operation where everyone wins including employees, shareholders and clients/customers.
Many companies do not invest enough money into ROI which in the long run will hurt their organization. Many times old work habits may creep back into the system and if the training isn’t evaulated 3 months after it has been implimented the company may not realize that employees’ are still working ineffectively. Ultimately, I feel learners’ want to be evaluated so they feel they are contributing to their organization and that their work matters.
The evaluation is the necessary component of the training process. This is how to measure whether or not the training was sucessful. The learners feedback is essential to the IOL process as it connects to the organization goals.
I do believe that adult learners need feed back at what they are doing wrong or right to improve their skills. I do agree that that is a way of achieving the goals of a HRD to improve their learning. Evaluation determines the worth of the time put into the training, The values that the employees take from the training program.
I agree when this article states that the IOL in the end is a win a win for the CLO. Many companies would be successful if they chose to follow this process. Without evaluation no company would be successful because they would not know where to begin to identify an issue or how to address it.
Evaluation is one of the most important steps. It allows the organization to learn whether or not the people being trained are gaining the knowledge, skills, and attitudes that are being implemented in the training session. IOL gives a quick evaluation of the learner and how that learner was impacted by the training session.
Evaluation plays a major role when you are in the training process and you will be able to evaluate if this process was successful. having the learners feedback is great information for an organization and thier set goals.
If a company spends money and time in the search of a better trained work force, there better be a way to measure results. This is the only way to justify a CLO. I imagine that a good CLO has to be a very good educational designer, and an even better needs Analyzer.
In this article I agree with the steps of evaluation. The most important step of the evaluation I think is one of the levels in Kirkpatrick’s model which is reaction. The way a learner reacts to a training says a lot. If they are engaged then they find the training effective. I don’t really agree with if the learner can do the task on the job the the training was effective, because there could be something going on in the work place that can keep the learner from performing.
If a company is investing money and time to train its work force, there better be a way to measure how effective the training has been. A CLO has to be an excellent change agent and knowledge facilitator to make sure that the training has worked.
I agree that evaluations are necesssary for getting feedback of how the training was. Getting the learners feedback gives an idea of what one is lacking in or what areas they need to improve.
I agree that evaluation is a very important part in determining the IOL. The adult learner is different and needs to give feedback when they are going through this process. If they feel that they are learning, it will used again not just during training.
For the training process, evaluation is necessary. It shows if the training was successful or not. The learners’ feedback is essential and connects to the organization’s goals and mission.
Evaluation is key to success. Training is costly and it’s important to evaluate the value of training by getting the learner involved. They know first-hand what’s working and what’s not and their involvement will only prove beneficial in the long run.
Training evaluation, simply put, will determine the level of worth of the training. Whether or not the learning officers had an impact on the learners is a big part of how effective and worthy the training was. If change did not occur as a result of the training, then its implementation was pointless to begin with; however if the learners were involved and improvement in performance and actual learning occured, it is a clear indicator that there was a return on the investment (the program).
The Impact of Learning (IOL) process seems like a good alternative to calculating the Return on Investment (ROI). By using the IOL it saves time, resources, and money, and relies. Since the IOL process relies on input from the employees, implementations are more likely to be effective. The IOL process is a good way for the CLO to show that he is doing his job without having to through the lengthy process of calculating the ROI.
The Return on investment, or ROI is a very costly procedure, but the IOLis definately a much less costly and more beneficial process for companies. Following up with evaluation is crucial. It is necessary to perform evaluation in order to seeif learning had been transferred and if training was sucessful.