I took this article from frwded email in MHRO and originally produced by CLO Magazine. It's a great stuff to read and useful for those who are in Learning and Development...
Have a reading pleasure!
Don Kirkpatrick: The Father of the Four Levelsby Daniel Margolis Don Kirkpatrick wrote the book on evaluation in learning and development, literally. Developed in the 1950s, the Kirkpatrick Model, often referred to as the Four Levels of Evaluation, is the standard for training evaluation and is still being taught and read today.
Kirkpatrick himself is now an elder statesman of corporate training. Asked his age, he said "I'm 80," before pausing and adding "five." But the octogenarian is still active in the field. This year, he taught two sessions at the American Society for Training & Development (ASTD)'s annual conference, as he has every year since 1960. "I've been in training all my life," Kirkpatrick said. "My feeling is that training is absolutely necessary for companies to keep up to date, to meet competition. As far as the curriculum is concerned, it must be practical.
Many programs have been just enjoyable; the teacher is out to get good ratings on the reaction sheets." That type of complacency is just what Kirkpatrick's four levels set out to address.
The levels measure:
1. Reaction: what students thought of the training.
2. Learning: the resulting increase in knowledge or capability.
3. Behavior: the extent of behavior and capability improvement and implementation.
4. Results: the effect on the business resulting from change in students' performance. "It used to be that trainers would feel, 'If we get a good reaction and teach people the skills and knowledge they need, that's all we can do.
We have no control over them when they go back to their job,'" Kirkpatrick said. "My philosophy is, yes, you have no control over them, but you must have influence on them, because unless that training gets used on the job, it's really worthless." Kirkpatrick believes that conception is the biggest challenge for learning and development in the corporate arena, and one that organizations across the board still grapple with in evaluation. "Many companies still are doing just Levels 1 and 2, but the trend is certainly for people to get up to Levels 3 and 4, because otherwise the training is really not worth it," he said. "Top management, we call it the jury, is not going to approve the budget unless you can prove that when people go back to the job they're using what they learn, and that's going to accomplish the results that they look for.
" A Life in Learning Begins Kirkpatrick began his college education in the wake of World War II. He received a bachelor's degree in business followed by an MBA from the University of Wisconsin in 1948 and '49, respectively.
Upon receiving his master's degree, Kirkpatrick was given a choice of paths.
"A professor said to me, 'Don, we have two opportunities. One is for you to go to a training program at a company and move up the ladder and become a manager of some kind,'" Kirkpatrick said. "'Two is to go with the Management Institute at the University of Wisconsin to teach and train supervisors and managers to do their job better.' I chose the second one because I had a desire to teach, and this was not only a teaching job, but it was a practical job.
I wanted to teach supervisors and managers to be more effective. I felt in a university you teach one course, and who knows what happens to [the students] - how does that apply to their whole life?" Kirkpatrick immediately began teaching seminars at the school's Management Institute, then decided to get his Ph.D. "I was one of the fortunate ones who picked a dissertation that would pay off in the future; [it] was on evaluating training programs," he said. "I felt as long as I was teaching, why not evaluate it?" Kirkpatrick remained there until 1960, when he took a job as a training manager at the International Minerals & Chemical Corp. in Chicago. In 1963, he became personnel manager at manufacturing and engineering company Bendix Corp. in South Bend, Ind., before returning to the University of Wisconsin at Milwaukee.
He remained there for two decades, eventually becoming professor emeritus, before retiring. In the meantime, Kirkpatrick's status in learning and development grew via his published work. In 1959, he received a call from Robert Craig, then editor of ASTD's journal. Craig knew Kirkpatrick had done research on evaluation and asked if he'd contribute an article. Kirkpatrick shot back that he'd write four articles. "So I wrote one on reaction; one on learning; one on behavior; and one on results; and it took off like wildfire," he said. "I never called it the four levels, I never called it the Kirkpatrick Model, all I did was teach it in certain places where I had the opportunity. "[As] soon as that article came out, I was on national conferences and invited into many companies to do the program. A guy at Ford Motor Co. said to me, 'Don, what you've done is taken that elusive term 'evaluation,' which we all talk about in training, but we were all talking in a different language, and now you've given us a language of four simple practical terms that everybody can understand,'" This new notoriety gave Kirkpatrick's career as a business consultant so much momentum that his published work faded from view. "I didn't even write my first book until 1993 - 33 years later," he said. "Someone said to me 'Don, people can't find your articles anymore.
They keep quoting from [them] and people talk about the levels, but where can they find from the horse's mouth what you really came up with?
'" That book, Evaluating Training Programs: The Four Levels, is now in its third edition and has been translated into multiple languages. More books have followed, some co-authored by Kirkpatrick's son Jim, vice president of global training and consulting for SMR USA. A New Piece of Equipment Kirkpatrick is still a big believer in classroom-based, instructor-led training. "As a matter of fact, I'm so old-fashioned I still use the overhead projector instead of PowerPoint, which ASTD and others don't like too well, but [that's] the way I teach best," he said. "I tell people, 'Here's a new piece of equipment. You call it the overhead projector,' and they get a kick out of that." He said e-learning, while convenient, is limited by the learners' willingness to teach themselves: a fatal flaw.
"There are limitations in what you can do on the computer, and unless people are really motivated, it's not going to work very well, and you don't get the interaction either," he said. "That's one of the problems with e-learning, is to try to get metrics. When you do it on the computer, you've got a lot more problems in trying to get people to reply to surveys because they're so busy. It's much easier to do it when you've got instructor-led programs."
According to Kirkpatrick, the state of measurement when he started in learning and development was limited to measuring reaction of students and what they had learned. "To some extent they were using pre-tests and post-tests to try to measure learning, but there was very little of Level 3 and Level 4 - going out on the job and working with managers to get them on board so they can help you evaluate programs and also help you implement them," he said. Kirkpatrick relayed a story from early in his career when he asked a group of supervisors he was training what their boss was likely to say to them when they returned to work.
Kirkpatrick was told: "Well, he's probably going to say, 'Boy, the work piled up while you were gone. Hope you had a good time; now, let's get to work.'" "And I thought to myself, 'What a mistake,' because many of those people were sent to programs because the personnel department had signed them up," he said. "And their manager didn't even know the content; the supervisor was forced to let them go and was not too happy." Thereafter, Kirkpatrick began advocating for management to speak with people before they're sent to training, encouraging them to enjoy the experience but letting them know when they returned they would be expected to report what they learned and apply it on the job. "And then we've got to measure that," he said. "We've got to determine to what extent have they applied it on the job, and we do it by going out or doing surveys, asking them to what extent they've applied [training] and getting their managers to say to what extent they've applied it. Unless those people change their behavior, the program really is of no benefit to the company." Kirkpatrick believes that senior management should take a similar approach and get out in front of the development of training. He said any training program should begin by asking executives what they expect to accomplish with the learning intervention, what success will look like and what results they hope to achieve. "We try to start with the people who are going to judge it, and then back up," Kirkpatrick said. "We don't just sit in our office and have curriculum design people design the content of the program." The Next Generation Asked what young learning and development professionals - people coming into the industry this year - should think about in terms of their job role and their approach to measurement, Kirkpatrick said they should learn the four levels. As fundamental as they are, some companies still struggle in their application. In particular, end users often think they can skip levels, but this will blur measurement or render it useless.
"I had a lady from Microsoft one time call me and say, 'Don, we've measured Level 1 and Level 2, is it OK to go to Level 4 and measure results?'" Kirkpatrick said. "I said, 'No, it is not OK because you don't know where the results came from.'" "We call it the chain of evidence. You get evidence on Level 1 that they liked the program - they thought it was practical; Level 2 that they learned the knowledge, skills and attitudes; Level 3 that they changed their behavior; and Level 4 that you're going to get the results from it." Kirkpatrick acknowledged that when learning and development professionals have mastered the four levels, they may choose to depart from them or further the methodology, citing research published by Jack J. Phillips in 1996 titled "How Much Is the Training Worth?" Phillips suggested a fifth level: ROI. Here, the results of training are converted to monetary values and compared with the cost of the program itself, thus quantifying the true contribution of training. "Now if you want to get to Jack Phillips' ROI, that's up to you, but that's a very complicated process, and it isn't that easy to do," Kirkpatrick said. "Anybody in training, you better start with the four levels, and then you can take off from there any way you want to." What should young learning and development professionals think about in terms of their job role and their approach to measurement? Kirkpatrick said they better learn the four levels before choosing to further the methodology.
[About the Author: Daniel Margolis is a managing editor for Chief Learning Officer magazine.]
No comments:
Post a Comment