I was attending a fantastic training event last week, the Bronze level of Kirkpatrick four levels evaluation certificate program. I’m going to try to explain the model taught and (of course) used in class with an example a little bit more close to heart than “evaluate the result of a training program”.
Let’s start from the back (at Level 4) – What do we want?
So, I and my family want to be able to pick our own apples in two years’ time and we don’t know a thing about planting or caring for trees. Therefor we’re starting up a tree planting project with homegrown apple pie as our first prize. This is our Level 4 Stakeholder Expectation. Now, why do we start with the fourth and last level and not the first one? Because it actually makes more sense this way! Be patient and continue reading…
Picking and eating apples from our own tree is our stakeholder’s end result but if we’d take a look at the project as a whole its Desired Result would be to have a fruits to pick as well as a healthy apple tree. How do we evaluate this? The first part is easy: if we are able to pick fruits and eat them, that’s pretty strong evidence. The second part might require several methods. Our Leading Indicators.
With our end-result in mind, the next thing at Level 4 is to ask: What are the short-term observations and measurements (Leading Indicators) that suggest we’re doing the right thing to improve the probability of us eating our own apples? These are the suggested things we must see to be certain that the tree will bear fruit within our time frame:
- Growth a certain amount of centimeters every 3 months
- Blossoming each summer
- Healthy leafs without leaf lice or other pest
All in all, it’s down to observing if the tree grows at it should. If it grows in the presumed amount each month and quarter – it probably is healthy enough to bear fruit.
Down to Level 3 – What do we have to do?
With these Leading Indicators in mind we move on to the Critical Behaviors of Level 3 instead. These are the “doing the right thing to improve probability of success” as described earlier, i.e. what do we have to DO to make sure things happen so that the tree bears fruit? For each of the things we need to do we also need ways of monitoring them as well as supporting them to happen with reinforcement, encouragement and rewards (Drivers). Let’s list these in a table as shown below.
| Critical Behavior | Monitoring | Drivers (Reinforce, Encourage and reward) |
| Pick out an old enough tree so that it can bear fruit within our time frame |
|
|
| Fertilize when needed to assure that the tree gets the nutrients it need to grow |
|
|
| Water the tree when required and some looking after if it rains to much |
|
|
| Apply anti-pestilence in correct amount and time |
|
|
| Perform correct branch cutting after each season |
|
|
The learning Level 2 – What do we have to know?
When we know what desired result we want and what critical behaviors are required to reach that result, it’s time to look at: What do we have to know to be able to perform those critical behaviors? Also, who, in the family, must increase their knowledge to do it right? Let’s make another table.
| Training Audience | Learning Objectives | Evaluation methods |
| The kids |
|
|
| The Dad | Know about:
|
|
Skills:
|
| |
| Attitude |
| |
Confidence:
|
| |
Commitment:
|
|
Level 1 – How do we do it to make it a fun experience for the family?
We’ve got our end-result and its indicators, our critical behavior and its drivers, and now we know what we have to learn for us to being able to do all of the new things. What’s left to figure out is what most people start with: In what way should we do this so that it’s as enjoyable as possible? Because, you know, if it all feels boring and dull, the risk is pretty big everyone just doesn’t follow through.
There are some things that should be looked at when considering what way to deliver the apple tree project. The family must feel:
- Satisfied during the project
- Engaged by the project
- The content of the project is relevant to their need
- It’s focused on them (the learners)
- Evaluation is appropriate to the project outcome (not too heavy and often)
- There is variation in the project
- The project has a clear alignment to the goal
Since this apple tree project is what it is we’ll of course not use any classic Level 1 evaluation methods like surveys. That’s too much… We’ll appoint the rest of the family as dedicated observers and actually instruct them to, now and then, ask me questions like “How’s it going?”, “Do you need any help?” and “Is there any more information or tools you need to take care of the tree?”. This ought to take care of the Level 1 evaluation which leaves us with the “How do we do it?”.
- Let it be student driven and involving
- Get a good classic gardening book with all the basics
- Get a specific “Apple tree caring for dummies” book with checklists and growth chart job aids
- A first meeting with the nursery-garden expert (when buying the tree)
- Tools for fertilizing, watering, branch-cutting etc.
Summary
Instead of starting with the beginning in mind, i.e. “Let’s go get us a tree!”, we started this apple tree project with the end-result in mind and looked at what things would make us get there. These four levels of Kirkpatrick are not that hard to understand and implement so there really isn’t a reason for you to not do that in your organization. I suggest that you go visit Kirkpatrick Partners and register for their newsletter and their separate resource page since there’s a wealth of information that’s available for free.
Thanks Jim for a great couple of days in London and also to the crew at DPG!
PS: If you enjoyed this post, find more at ennoble.se ! DS
Comments