Evaluating Learnining

I wanted to jot down my thoughts on something I've decided to do for a one-off "pilot" training course I'm running.  The course by it's very nature is purposely designed to have an influence on a particular aspect of business performance for one of our venues.

Do we honestly always evaluate?

The first thing to say is if I'm honest, I'm guilty from time to time of not building in evaluation methods at every level of Kirkpatricks Evaluation Model for every single piece of training I do.  I'm sure we can all put our hands up to that one.  But because this particular piece of training is so obviously linked to performance, it's really important that I pick my game up for this one.  

Evaluating behaviour changes (level 3) and results changes (level 4)

So I've put some things in place for the managers to observe the changes in behaviour (Level 3 of Kirkpatrick's Model) over the weeks that follow the training.  I've also done some benchmarking before and after (Level 4 of Kirkpatrick's Model) on the Key Performance Indicators (KPI's) that this training is likely to have an effect on.  These two alone are going to demonstrate return on expectations (ROE) in a positive light (fingers crossed).

Evaluating reactions (level 1) and learning (level 2)

So, turning my attention to the reaction (Level 1) I've again wanted to do some benchmarking here, mainly around confidence.  Before the training starts I want to ask the question, on a scale of 1-10 how confident are you with..... and then after the training repeat the same question to demonstrate (hopefully) some upturn in levels of confidence.

I begun designing an evaluation form.  Knowing that I wanted the confidence question answering right at the beginning started to do some thinking around how best to do this.  So I designed a form that asks the confidence question, along with some other Level 1 questions - how has the training helped with..., what else do you think this training should include.... etc.  I've also put in some knowledge and learning (Level 2) questions in there too such as give an example of....., describe the benefits of..... etc

Give the form out at the end, or perhaps at the beginning?

But, here's what's especially different in my approach this time.  Instead of waiting till the end of the training (where people are wanting to just get out the door) before giving them this form, I'm going to give it to them at the beginning of the session.  What this will do is not only allow me to benchmark the confidence question, but also to get them thinking about the evaluation of the training throughout the training, rather than racing to complete it at the end.  The Level 2 learning questions, some might say, MUST be done at the end.  But as we know, the process of writing down responses to questions is a great way of enforcing learning anyway and maybe should be done throughout rather than a test at the end.  We're talking here a subject where that this test of learning isn't a pass or fail thing and I appreciate there are some topics that this approach may not suit.

So the learning for me here is, at the beginning of our training sessions, if we reveal from the very start what feedback we're going to be asking for at the end, and encourage people to think about these areas throughout, would that make it easier for us to get a better level of reaction level evaluation than those rushed tick in the box responses we usually get.

I shall let you know...!

You need to be a member of DPG Community to add comments!

Join DPG Community

Replies

  • Hi Lucy,

    So some feedback for you:

    Level 1 Reaction - The idea of giving out the evaluation form at the beginning worked really well.   Particularly as I was able to take a "rate your confidence in...." benchmark at the beginning and end of the training - great for demonstrating results at this level

    Level 2 - Learning- On the form were also some knowledge type questions that were great again for people to jot their thoughts down as they went through the course again great for demonstrating results

    Level 3 - Behaviour - In the training we described what we were looking to see in relation to behaviour change and how it will be monitored and supported.  This gave the group a very clear expectation in terms of what they were being asked to do and how they will be measured and supported.  

    Level 4 - Results - I took your advise and we gave them a clear view of what we were looking for in terms of results.  This went VERY well and actually a couple of them commented (one verbally and a couple on the feedback forms) how much they appreciated this.  One said, "It's so much easier when we know what we have to achieve.  Even when we're not doing well, at least we know it's time to pick our game up.  It was suprising how quickly they all started talking about what they were going to do to achieve it!

    So sharing with your delegates in training how training will be measured at all levels seems to be good practice in this case.

    I'll let you know what the real results tell us!

    • Hi Ady. Thanks for this post and it is great to see you sharing your approach and learning too. I have an idea for next time and I hope you are ok with me sharing. You can build an activity into the day where you ask:

      The shift from the bottom to somewhere near the top won't happen overnight. So, I want to explore with you what you think the journey will be to the top of the leader board. Specially,

      What will be the things you will be doing tomorrow to start the move?
      What support do you have and/or need around you to make it really easy to do those things?
      What will be the clues you will find that tell you movement is happening and in the right direction?

      I find it gets them to do some future thinking and tells you:

      What needs to be in place to help them succeed
      What you can look for, really quickly after the event to check that things are happening
      (Indirectly) what possible pitfalls may be waiting and you can take action to overcome them.

      All great things for establishing and creating value.

      Hope this helps

      Phil
      • Hi Phil,

        Love that idea.  I think anything that can create the action needed following a learning event is essential. Really useful for proving value of the training.

        Love that idea Phil and I'll definately be using that.  I have a huge bee in my bonet at the moment about embedding learning, particularly making management development learning stick so anymore on establishing and creating value much appreciated. 

        Thanks again Phil

        Ady

         

  • Hi Lucy,

    Thanks so much for your feedback.  Yes I shall take this on board, put this into practice and let you know how it goes.

    I can't ever remember sharing my Level 4 aspirations with any group and I totally agree that this would be a good thing to do.It's being open, upfront and transparent isn't it.

    When I scoped out this particular piece of training, it was done over a conference call with myself and three key stakeholders. I remember it well, because even this I approached in a slightly more structured way than I ever have. I used the GROW model to work with the group on the Learning Needs Analysis.  My fist question (goal related of course) was:

    "Where is the performance of this particular team in relation to other teams?"

    And that led to further discussion around where this team was on the leader-board. The answer was that this team were in the bottom third of the leader-board in the particular KPI we were discussing. My response was "So if this training moves them up to the top of the leader-board is that something you'd be happy with?" You can imagine the resounding YES as a response!

    We then explored how realistic it was to do this and what the shift in performance needed to be.

    Anyway, long story short, our bunch (the whole organisation) is very competitive.  They've always responded well to league tables and leader boards so your theory on sharing this with them, should really go down a storm! So perhaps the way I'll share this with them is:

    My goal for you is to move you away from being at the bottom of the leader board and closer to the top.  I'd love to see that and this training will help you to do that. Who's up for it!

    I can imagine their faces and excitement. No doubt some of them will ask "So if we do that, what do we get in return?" given that they're quite used to being given incentives to up their game. But I have another plan for that and that's a whole different topic of discussion that you can find here: Beyond the Financial Incentives

    Thanks again Lucy.  I really value your input and I will let you know how this goes.

    Thanks loads

    Have a good weekend

    Ady

  • HI Ady,

    That sounds great - and I've got no doubt that this approach will get you the results you're after!

    You could even go one step further - by sharing your Level 4 desired outcomes with your learners at the start of the session, or even in the days before the workshop, you're enabling them to really buy into the larger organisational goals that they are a part of.

    It's about them learning (level 2), yes, and answering questions throughout the workshop rather than at the end will really help to reinforce knowledge acquisition. But sharing your level 4 can really positively affect your Level 3 (behaviour change), as the learners are more aware that their role in the bigger picture is not only to learn, but to use this training event to modify their behaviour to ultimately aid a shared organisational goal.

    There's definite motivation in your actions being a part of something bigger, and putting the onus on the learner - making them responsible for ensuring their own learning and behaviour change-  can really help them to commit and see the importance of this, simultaneously improving their own learning experience.

    While we're on the subject of evaluation, here are a couple of blogs around Kirkpatrick which you may find useful:

    http://community.dpgplc.co.uk/profiles/blogs/evaluation-between-the...

    http://community.dpgplc.co.uk/profiles/blogs/collaborate-to-evaluate

This reply was deleted.

CIPD Branch Events

Did you know your local CIPD branch will put on relevant events that are free to CIPD members.

Take a look for your local branch here and what events are happening. Remember attending these events are great CPD evidence.

CIPD Branch Event Search

Members

Click here to see a full list of members including our Facilitators.

Did you know that if you go to the list of members, the Members Online button will show you who is online right now? Why not say hello?

What's Happening?

Melanie Guttridge and Maria Kenneth joined DPG Community
Monday
David Woodhouse and Jacqueline Ruth Dhanoya joined DPG Community
Mar 19
ZLYI posted a discussion
Mar 16
Ruth Miller and Simone Howarth joined DPG Community
Mar 13
alan cubero replied to NoahCarcia's discussion Besteonlinecasinoschweiz
Mar 13
alan cubero replied to Oscar Poppy's discussion Play wolf gold slot
Mar 13
Patty Black commented on Mike Collins's blog post Unlocking Potential Through Learning
Mar 7
Samuel Rocher is now a member of DPG Community
Mar 7
More…