Training Effectiveness Measurement

Training Effectiveness Measurement

Each year globally many millions of dollars are spent on training. It would be reasonable to establish just how effective this training has been if only to help justify the budget. The aim is to determine the effectiveness of the training by scientific means on the level of performance of the individual. The methodology has to quantify the relationship between their corporate objectives and the training and the training aims and objectives.

The evaluation approach can be divided in to two main schools; the narrow focused approach being attributed to Donald Kirkpatrick and the broader model advocated by P. Warr’s known as CIRO model. The CIRO model looked at context evaluation, input evaluation, reaction evaluation and the outcome evaluation.

Donald Kirkpatrick’s “Four Steps to Measure Training Effectiveness”.

Stage 1: Reaction

The first stage is about the reaction of the learners to the training. Kirkpatrick likes to call this step a measure of customer satisfaction, which is because this step involves gathering feedback forms the trainees about they felt about the training programme. The standard course feedback sheets are an example of the Kirkpatrick stage 1 evaluation. Most organization do not do any more than this type of measurement and analysis.

 

The evaluation description and characteristics

• Did the learners like and enjoy the learning?
• Did they consider the training is relevant?
• Level of participation.
• Level of effort required to make the most of the learning.

·         

Tools and Methods

• Typical happy sheets
• Post-training surveys or questionnaires.
• Online evaluation or grading by learners.
• Subsequent verbal or written reports given by delegates to managers back at their job.

·         

 Stage 2: Learning.

 

By learning, Kirkpatrick means measuring the level of change in the competency of the participants in terms of positive attitude change, improved knowledge or increase in skill as a result of attending the program. As Kirkpatrick says – “Some trainer say that no learning has taken place unless change in behavior occurs. Learning has taken place when one or more of the following occurs: Attitude is changed. Knowledge is increased. Skill is improved. One or more of these changes must take place if a change in behavior has to occur.

The evaluation description and characteristics

• Did the trainee learn what had intended to be taught?
• What was the level of advancement or change in the trainees after the training?

Tools and Methods

·         Typical assessments or tests before and after the training.

·         Interview or observation can be used before and after.

·         Hard copy, electronic, online or interview style assessment are all possible.

Stage 3: Behavior

Behavior means the measurable change in an individual as a result of their attendance on the training course. Behavior evaluation is the level of applied learning back on job implementation. In L&D activities some tend to bypass the first and second level and get straight to measuring the changes in behavior. This is a critical mistake. Obviously the conclusion drawn that the training was ineffective. This may not be accurate, because even the training might be flawless and does well with the first and second stage, it may not guarantee a change in behavior.

Kirkpatrick listed four conditions are necessary to bring change in behavior.

1. The person must have the desire to change.
2. The person must know what to do and how to do.
3. The person must work in the right environment.
4. The person must be rewarded for the changing.

If no change in behavior is observed after training, the reaction and learning should be re-evaluated to check whether the training delivery or content was effective, if it was then may be the environment is not conducive to change.

The evaluation description and characteristics

·         Did the learners put their learning into effect when back on the job?

·         Were the relevant skills and knowledge used?

·         Was the change in behavior and skill sustained?

·         Is the trainee aware of their change in behavior, knowledge and skill level?

Tools and Methods

•          360® feedback is the useful method
•        Tool can be designed around relevant performance areas and specific key performance indicators.
•      Self assessment can be very useful, using carefully designed criteria and measurement.

Stage 4: Result

Kirkpatrick is now concerned with the training to determine if it has actually been translated into tangible benefits to the organization. The various metrics it includes could be increase in production, decline in cost, reduction in employee turnover, higher profits. It is on these results that the scope and future of the training department.

“It is difficult if not impossible to measure the final results for programme on such topics as leadership skills, decision making, time management, motivation or managing change. We can state and evaluate desired behaviors, but the final results have to be measured in terms of improved morale or improved quality of the work life will automatically result in tangible results” in the words of the master himself.

The evaluation description and characteristics

Measures would typically be business or organizational key performance indicators, such as volumes, values, staff turnover, growth retention, accreditation's etc….

Tools and Methods

·         Annual appraisals and ongoing agreement of key business objectives are integral to measuring business results derived from training.
 

Conclusion

Implementing Kirkpatrick’s model requires engagement and sponsorship of the executive management. Very often the feedback arrived from the first and second stage is positive, but the third and last stages leave a lot of desired.

References

• Evaluating Training Programs by Don Kirkpatrick
• Other L&D forums