The L&D Forum

Training Evaluation

Hi All

Your help would be greatly appreciated!

I'm currently working on a project to implement a new Training Evaluation Process in our organisation. Currently, our evaluation process is very rigid and does not give me the quality of data that I need to report to the business on return on investment and personal development.

I would like our new process to include a Pre-Course Evaluation, Happy Sheet, Post-Course Evaluation and a Measuring Success review.

In order to implement a quality system, I would like to see what evaluation processes your organisation currently adopts, our for you to share your thoughts on what a best practice training evaluation process should include.

Look forward to hearing from you...

Kind regards

Claire

You need to be a member of DPG Community to add comments!

Join DPG Community

Email me when people reply –

Replies

  • Dear Claire, 

    It really sounds to me like you are on the right track in terms of a blended approach to training evaluation. I encourage people to determine what information:

    1. Is useful to them, as the training professional, to ascertain if the program is of good quality, and learning has occurred. 

    2. Creates credibility for the training in the eyes of stakeholders, which is usually surrounding if on-the-job performance improves, and important company metrics are positively impacted

    You mentioned that you evaluation process is rigid. Here are some ways that might help you to work around it: 

    - Focus on formative: think about what information you might be able to gather during training. You may have more control over the instructional design than the evaluation tools

    - View every evaluation instance as a possibility to gather information at any of the Kirkpatrick levels that have data available. For example, don't view your post-program evaluation as a "Level 1". View it as an opportunity to gather information about Level 1 Reaction, Level 2 Learning, and predictive information related to Level 3 Behavior and Level 4 Results. 

    - Find ways to gather both qualitative and quantitative data. Your "measuring success" review might be your chance to actually speak with some training graduates. If not, don't overlook the possibility to simply call, email or text training graduates and ask them if they are using what they learned, and/or seeing any results. Their stories, combined with numeric data, create a strong story. 

    I hope these ideas help! DPG offers the Kirkpatrick certification program, which is a deep dive into the ideas I have mentioned here, and much more. 

    Best regards, 

    Wendy Kirkpatrick

  • Hi Zoe, Mike

    Sorry to necro post but I'm building an approach to measuring the impact of our leadership and management training in BT, and I'm looking for examples outside BT where the impact has been measured successfully.

    So far I've identified 3 levels of measures in my approach:

    Level 1 - strategic measures being tracked by the top table

    Level 2 - business measures, being tracked by the lines of business that indicate management and leadership capability

    Level 3 - operational measures, being tracked by L&D that indicate the uptake and the success of the learning interventions we're using

    I was wondering if you'd come across any models other companies have used that have been successful that could provide a different perspective on how this can be done?

    Thanks

    Adam

    • I had to look up necro post Adam lol. This is why I love the community approach so much, no post is ever dead and can help with current thinking and continue to develop ideas.

      Personally I think the approach you've highlighted above is great, what's key is that the 3 levels link together so if you take the strategic measures / business objectives - how do they translate in to business measures and likewise translate in to operational measures. 

      The New World Kirkpatrick Model describes a shift from Return on Investment (RoI) to that of Return on Expectations (RoE)...sometimes not everything can be measured by a KPI, so by asking the questions what do expect to see / hear change as a result of this can be just as powerful.

      Identifying 'leading indicators' at each stage of the approach will really help you understand what is going to change and the types of thing you'd expect to see or hear change as a result.

      Have you also discussed and identified what you'd like to see change, what success looks like against the strategic measures and how you'll know when it's happened? Great questions to use to help shape the right approach which may or may not be training in isolation or training as part of a wider change programme etc 

      Let me know what you think

      Mike

  • I think the new process you are looking to adopt is a positive step forward.  

    I really like the pre-course evaluation.    You need this to understand the starting point and can sit alongside any needs analysis.    We have this built in with our systems and we keep it to a few questions around understanding and confidence.    Then you can build on these answers after any training and understand the individual’s new situation.   Confidence or lack off influences behaviour. 

    Mikes a very valid point about the happy sheets.  If they are good then they can give you a great insight on the activity and how people are feeling at the end of any sessions (usually positive and raring to go) .    People tend to take more action when feeling positive about something therefore knowing if they enjoyed it is important BUT only if you then can judge effectiveness and impact.  

    Are you looking for more practical guidance?   

  • Hi Claire, I'm sure you'll receive some great ideas & suggestions. I recommend starting with your key stakeholders & asking them what are the indicators they'd like to see on their 'dashboard'. I'd keep asking questions to define what good looks like in the business, remembering to gather both quantitative & qualitative measures from them. I'd put together a high level proposal on what to measure & when to ensure the business gets its Return on Investment (ROI) & Return on Expectations (ROE) & present this to stakeholders for their agreement. Once agreed I would then start implementing the plan & regularly reviewing it for effectiveness.
    Sadly, many organisations still don't measure the impact of their L&D interventions & that means much time & money is wasted in areas thought to be important but are actually not adding any value!
    Hope this is useful, you'll also enjoy your ELD elessons on this topic!
    Zoe
    Hope this helps.
  • Hi Claire,

    Great post and one that is a hot topic for any L&D professional challenged with demonstrating the value of what they do....

    I'd start a step before the evaluation even starts and ask what the business / stakeholders are actually looking for in terms of return on investment (can this actually be done accurately) and personal development around training?

    In other words are you delivering the right training in the first place - if you can't honestly review what you provide against links to business objectives or how it aligns and what the measurement of success will be then no amount of evaluation will provide you with the right quality of data. 

    I'd flip the process completely, but some considerations and thoughts below:

    - Measuring success review - what does success look like, what changes are you hoping to see, productivity, behaviour; what are the expectations around change and how will these be measured, my whom and by when - if this isn't clear and aligned to clear business metrics and KPI's or leading indicators then you shouldn't be doing it in the first place.

    - Post course evaluation - is this learner focused or business / performance focused or both? Does it involve learners rating the relevance / quality of the training in relation to their role? How has the training improved what they do personally or towards a task / activity i.e are they performing better. Does this include observations, who does this? Were line managers involved in the training, are they coaching and supporting to ensure learning transfer takes place. Would learners recommend the training to other people? How are you going to gather and display the feedback and how long will this process take?

    - Happy Sheets - don't call them happy sheets, one of L&D's biggest challenges is moving mindset away from the training room environment and focusing on performance not training. Happy sheets in my opinion put the focus on the trainer/training rather the the output of what they are training. If you want to gather feedback on the training session then make it absolutely relevant to the output - as a business leader I am not interesting in whether people enjoyed the training I'm more interested in whether it was effective and it has had a positive impact on their behaviour or performance. If L&D focus is on the event itself and looking inward as to whether or not 'we' did a good job and got great feedback on our delivery style or way we ran the session then we're never going to be able to demonstrate a return on investment. Some great thoughts here put far more eloquently than me

    - Pre-Course Evaluation - not sure what purpose this serves? Is this to ensure the right people are on the course? Is it a needs analysis piece - how does it link with measuring success review as the two need to be explicitly linked and joined together.

    Probably more thoughts and opinions here than a process to follow but hope some of this resonates. I'd be keen to get your thoughts on the above :)

    Mike 

This reply was deleted.

CIPD Branch Events

Did you know your local CIPD branch will put on relevant events that are free to CIPD members.

Take a look for your local branch here and what events are happening. Remember attending these events are great CPD evidence.

CIPD Branch Event Search

What's Happening?

Omer Sap updated their profile photo
35 minutes ago
Rachel.Higham@hotmail.co.uk updated their profile photo
10 hours ago
David Kirk and Donna Brauns are now connected
10 hours ago
Adam Goodsearles posted a discussion
11 hours ago
Sarah Loates posted a discussion
13 hours ago
More…