evaluation - Blogs - DPG Community2024-03-29T13:03:59Zhttps://community.dpgplc.co.uk/blog/feed/tag/evaluationThe role of data and analytics in evaluationhttps://community.dpgplc.co.uk/blog/the-role-of-data-and-analytics-in-evaluation2019-06-20T10:44:39.000Z2019-06-20T10:44:39.000ZAdy Howeshttps://community.dpgplc.co.uk/members/AdyHowes<div><img src="https://storage.ning.com/topology/rest/1.0/file/get/3036011792?profile=RESIZE_400x&width=400"></div><div><p><strong>What is the sort of data we should be gathering and what questions should we ask?</strong></p>
<p>Last week, I saw a session at <em>The Festival of Work</em> about <strong>Learning Data and Analytics</strong> at capacity.</p>
<p>No more space, not one seat, packed, full, people turned away (I myself had to watch from the side lines from the standing room only section).</p>
<p>Data and analytics, it seems is a hot topic right now and as ever you can find this topic explored already in some detail here on the DPG Community. There are articles <a href="https://community.dpgplc.co.uk/blog/i-m-not-a-data-scientist-but">here</a> and <a href="https://community.dpgplc.co.uk/blog/are-l-d-slow-to-change">here</a> that are well worth reading that link to data in L&D.</p>
<p>So why now?</p>
<p>I think the focus on data has always been there but one of the issues is accessibility and the type of data & information we as an industry have focussed on. I also think data is being looked upon to understand relationships and the way in which we draw conclusions and make links to how and why things are as they are.</p>
<p>There is a seismic shift across all industries to use the abundance of data & insight now available. Using this data to help and better understand customer behaviour, customer lifecycles, problems we’re looking to solve, product / service satisfaction, more effective product development, credibility, buying habits and trends and positive and developmental feedback (reviews). We live in a digital world and this has changed our relationship with data, but the principle of data analysis and product development can apply to all aspects of L&D. For me data helps us ask fundamentally better questions or helps us know the right questions to ask.</p>
<p>If L&D were a start-up business, developing a suite of products, this is also exactly the type of data we should be looking to gather or questions to ask in terms of:</p>
<ul><li>Clearly identifying the problem / challenge / pain faced by potential customer</li>
<li>Credibility – consultative conversations with customer to ensure deep understanding and right approach is used with valid recommendations provided that meet that need.</li>
<li>Customer lifecycles – where and at what point is support / training / solutions required to address any problems</li>
<li>Customer behaviour – how do customers interact with a product to address problem, where do they access it or buy it and crucially how does the product change / modify behaviour in a positive way and ‘solve’ problem.</li>
<li>Product satisfaction (feedback on learning solutions / performance support tools)</li>
<li>Product development / continuous improvement based on feedback</li>
</ul><p>Maybe one of the previous challenges for L&D is the ‘product’ has been predominantly a face to face classroom session or a face to face workshop and we just haven’t thought about what we do and how we do it in these terms before. The questions and approach above can be applied to any L&D product but I just don’t think we think as commercially as we should when it comes to demonstrating value and measuring success. As a cost centre L&D need to be able to demonstrate the value they provide, and more is being demanded of us than ever before.</p>
<p>To respond to this demand, more and more L&D teams are working using agile principles and methodology. Rapidly prototyping and developing minimal viable products quickly and far more collaboratively with end users to ensure whatever product is developed meets the need of the end user. Pilots, experiments and beta products are all things you will find in other industries, but we don’t tend to embrace these approaches as much as we perhaps should. One of the things I’ve been thinking about is how we link data & insight to asking and answering the questions above to traditional L&D evaluation.</p>
<p>Evaluation in L&D needs to evolve. I’m not suggesting that the likes of the <a href="https://www.dpgplc.co.uk/courses-qualifications/learning-and-development/kirkpatrick-four-levels-evaluation-certification-programme/">Kirkpatrick Evaluation Model</a> (Link) are obsolete. As a <a href="https://www.dpgplc.co.uk/courses-qualifications/learning-and-development/kirkpatrick-four-levels-evaluation-certification-programme/">Kirkpatrick</a> Partner we are big believers in the KP model. The framework that the KP model sets out helps still provides a structure to follow that can be adapted to each of the elements listed above. It’s a move from an old training mindset to that of a data driven and evidence-based department who focus less on individual training events to that of overall engagement and customer or employee lifecycles and product / performance improvement.</p>
<p>Rather than taking the 4 levels as a linear model and moving through them 1 by 1, analysing the key elements of each and applying them in a way that meets the need of the customer. For example:</p>
<p><strong>Results (level 4)</strong></p>
<p>Start with the end in mind, analyse challenges and pain points – what does success look like and how will it be measured. Consultative approach working collaboratively with customer.</p>
<p><strong>Learning (level 2)</strong></p>
<p>Make a recommendation and begin prototyping with the customer to build an MVP. Get it in front of people to test and use. Pilot, pilot, pilot.</p>
<p><strong>Reaction (Level 1)</strong></p>
<p>Get effective user feedback, is it effective, easy to use, easy to understand, what works what doesn’t, how can it be improved, collaborate and demand feedback.</p>
<p><strong>Behaviour/Transfer (Level 3)</strong></p>
<p>Does it have the desired impact, does it change behaviour, provide new skills, can this be observed, how does this link to the desired success and measures identified.</p>
<p>If yes – release and get back to & track ‘Results’</p>
<p>If no – back to Learning and better product development</p>
<p>Repeat.</p>
<p>OK, so I appreciate this is a crude example but the point I’m making is that we have always looked at evaluation as something that we do to other people or something we apply to the end of the training process as opposed to something that we apply to our process and continually analyse the review the feedback and data we have.</p>
<p>To do this in a much more agile way and to work much closer with our customers to provide better products and service. Working with pilot groups and working in sprints can speed up the development process and make things so much quicker. Today, we don’t need to wait 3-6 months to analyse a change in behaviour and we also don’t have to rely on outdated modes of gathering feedback. Apps, pulse surveys and polls can get us closer to the data we need much quicker if we’re prepared to experiment and explore.</p>
<p>In this age of data, we need to look at our own processes and approaches to stay relevant and keep up with the demands of our customers. Not a complete overhaul of everything we do but looking at what we do and how we do it through other lenses and borrowing ideas and concepts from other industries to make us more resilient and responsive.</p>
<p>Perhaps learning analytics while the latest buzz word is just another way for L&D to evolve its’ practices and that’s why there is such a focus on it. For me though it isn’t about disregarding everything that we’ve done before but using that we can see and experience to make better and more informed decisions. Yes, there is an element of shiny new technology, analytics dashboards and ‘evidence based’ approaches but this needs to blend together with ‘evaluation’ of sorts and we can learn a lot from new forms of data but also our existing data and use it in a more considered and effective way.</p>
<p><em>What do you think?</em></p>
<p><em>How are you using data & evidence-based practice to improve your product development?</em></p>
<p><em>Is the way you do evaluation changing?</em></p>
<p>If you are interested in finding more about how to use a data and evidence based practice to your Learning and Development activities, check out our <a href="https://www.dpgplc.co.uk/courses-qualifications/learning-and-development/kirkpatrick-four-levels-evaluation-certification-programme/">Kirkpatrick Four Levels Certificate</a></p>
<p> </p>
<p> </p></div>How to evaluate learninghttps://community.dpgplc.co.uk/blog/how-to-evaluate-learning2018-11-27T12:22:53.000Z2018-11-27T12:22:53.000ZAdy Howeshttps://community.dpgplc.co.uk/members/AdyHowes<div><img src="https://storage.ning.com/topology/rest/1.0/file/get/139392503?profile=RESIZE_400x&width=400"></div><div><p>'Around the world, training and development is in a state of crisis'. That's <a href="https://www.td.org/newsletters/atd-links/why-evaluate-learning" target="_blank" rel="noopener">according to the Association for Talent and Development</a>. They go on to say, 'Training professionals who only think abot training events and not what happens before and particularly after them will become extinct'.</p>
<p>It's a scary picture when you look at it like that, but you can't deny what's happening here. As the article rightly points out, training budgets are among the first to be cut when economic times get tough. Is L&D demonstrating enough of a positive impact on the performance of the orgnanisation?</p>
<p>If there's ever been a time to demonstrate the return that learning iniatives bring now has to be it. Part of the problem is that many L&D functions don't consider the desired results of learning interventions from the outset and there are often now well alligned to the desired performance requiremets of the organisation. Failing to create stong links between learning and performance could bring about situations where L&D is placed in the nice-to-have category.</p>
<p>Here at DPG, we're the UK's sole partner for the most recognised model of evaluation, Kirkpatrick. You can <a href="https://www.dpgplc.co.uk/courses-qualifications/learning-and-development/kirkpatrick-four-levels-evaluation-certification-programme/" target="_blank" rel="noopener">find out more details about that on our website</a>.</p>
<p>But is the Kirkpatrick model of evalution still relevant and how can it work in today's modern organisations?</p>
<p>We got chance to catch up with Phil Willcox, Kirkpatrick practitioner and facilitator of the <a href="https://www.dpgplc.co.uk/courses-qualifications/learning-and-development/kirkpatrick-four-levels-evaluation-certification-programme/" target="_blank" rel="noopener">DPG Kirkpatrick Programmes</a>. Check out this recording, under ten minutes long, of why he thinks it is....</p>
<p> </p>
<p><iframe src="https://w.soundcloud.com/player/?url=https%3A//api.soundcloud.com/tracks/536192190&color=%23ff5500&auto_play=false&hide_related=false&show_comments=true&show_user=true&show_reposts=false&show_teaser=true&visual=true" width="100%" height="300" frameborder="no" scrolling="no"></iframe></p>
</div>The Best Questions to Understand Learning Needshttps://community.dpgplc.co.uk/blog/the-best-questions-to-understand-learning-needs2018-08-31T12:18:19.000Z2018-08-31T12:18:19.000ZAdy Howeshttps://community.dpgplc.co.uk/members/AdyHowes<div><img src="https://storage.ning.com/topology/rest/1.0/file/get/85243328?profile=RESIZE_400x&width=400"></div><div><p>What questions do you ask to understand the learning needs of individuals and teams in your organisation? This often comes up in our <a href="https://www.dpgplc.co.uk/courses-qualifications/learning-and-development/kirkpatrick-four-levels-evaluation-certification-programme/">Learning and Development programmes</a>.</p>
<p>Training, learning, development call it what you will is essential for businesses to grow. So why wouldn’t the learning professional want to ask the right questions to make sure solutions hit the mark?</p>
<p>Whether that solution ends up as being a course, a programme, a resource, a piece of online learning, offline learning, coaching, mentoring or something entirely different matters not. What does matter is that the solution is fit for purpose and delivers the performance improvements that are needed by the organisation.</p>
<p>It’s like starting with the end in mind and figuring out what good looks like right from the offset.</p>
<p>I hopped onto the weekly #LnDInsight chat, a weekly chat on Twitter every Friday from 8am to 9am hosted by LnDConnect. That was exactly the topic of discussion this morning....</p>
<p><a href="https://storage.ning.com/topology/rest/1.0/file/get/85236294?profile=original" target="_blank" rel="noopener"><img class="align-full" src="https://storage.ning.com/topology/rest/1.0/file/get/85236294?profile=original" width="593" /></a></p>
<p>I’ve taken time here to list here some of the questions that came up as part of that Twitter discussion. I’d highly recommend a read through to choose the ones you're most likely to use next time. Let me know in the comments. I'm interested.</p>
<p>In the meantime, if you want to get even better at these approaches, figure out your next steps with our <a href="https://www.dpgplc.co.uk/courses-qualifications/learning-and-development/kirkpatrick-four-levels-evaluation-certification-programme/">Learning and Development programmes</a>.</p>
<p> </p>
<p><a class="twitter-moment" href="https://twitter.com/i/moments/1035481096077684737?ref_src=twsrc%5Etfw">The Best Questions to Understand Learning Needs</a></p>
<script charset="utf-8" src="https://platform.twitter.com/widgets.js"></script>
</div>#TrainingCrimes: Episode 3 - Evaluationhttps://community.dpgplc.co.uk/blog/trainingcrimes-episode-3-evaluation2018-07-11T14:10:00.000Z2018-07-11T14:10:00.000ZAdy Howeshttps://community.dpgplc.co.uk/members/AdyHowes<div><img src="https://storage.ning.com/topology/rest/1.0/file/get/42332915?profile=RESIZE_400x&width=400"></div><div><p></p>
<div class="video-container"><iframe width="1280" height="720" src="https://www.youtube.com/embed/DPMlB68NzUE?rel=0" frameborder="0" allow="autoplay; encrypted-media" allowfullscreen="allowfullscreen"></iframe></div>
<p></p>
<p> </p>
<p><strong>Training Crimes: Evaluation</strong></p>
<p>In the <a href="https://community.dpgplc.co.uk/blog/trainingcrimes-episode-1-training-needs-analysis" target="_blank" rel="noopener">first #TrainingCrimes episode</a> we created in association with <a href="https://www.myfirsttrainers.com/">My First Trainers</a> we looked at Training Needs Analysis while the <a href="https://community.dpgplc.co.uk/blog/trainingcrimes-episode-2-design-and-delivery" target="_blank" rel="noopener">second episode focussed on design and delivery</a>.</p>
<p>In this concluding episode, we consider the topic of evaluation. DPG are the UK partner for the <a href="https://www.dpgplc.co.uk/courses-qualifications/learning-and-development/kirkpatrick-four-levels-evaluation-certification-programme/" target="_blank" rel="noopener">Kirkpatrick Programme </a>so this is a topic which is close to our hearts.</p>
<p> </p>
<p><strong>No Alignment to the Business</strong></p>
<p>This is one sure-fire way of making sure a training initiative falls flat on its face. You’d assume that aligning a solution to the business should be done at the beginning of the process and you’d be right in making that assumption.  Starting with the end in mind from the offset is a good thing to do. Consider what the initiative is setting out to achieve and what measures could be used. It’s those that will help you demonstrate the value of the work that’s been done and the progress that’s been made to stakeholders.</p>
<p> </p>
<p><strong>No measurement of the impact of training</strong></p>
<p>Time and money is precious. How do you know the investment has been worthwhile? How do you know that it’s been a success? Our <a href="https://www.dpgplc.co.uk/courses-qualifications/learning-and-development/kirkpatrick-four-levels-evaluation-certification-programme/" target="_blank" rel="noopener">Kirkpatrick Programme</a> here at DPG is designed to help people measure the true impacts of the work done by L&D on many levels. You can find out more about that here.</p>
<p> </p>
<p><strong>Bribing the delegates</strong></p>
<p>Feedback is crucial. If you truly care about making approaches to learning the best they can be, honest feedback is essential. It’s not about getting the best scores on the happy sheet! Take time to gather and assess the feedback you get from those making use of the learning on offer. These are your customers who will give you clues on how to make what you do even better for the future.</p>
<p> </p>
<p>To get better at evaluating learning initiatives, you can find out more about our <a href="https://www.dpgplc.co.uk/courses-qualifications/learning-and-development/kirkpatrick-four-levels-evaluation-certification-programme/" target="_blank" rel="noopener">Kirkpatrick programme here</a>.</p>
<p>What do you think?  Have you come across these crimes before? What other crimes happen when it comes to evaluation can you think of? How can we avoid them?</p>
<p>I’m interested to hear in the comments below….</p>
</div>The most likely used model to demonstrate the value of learning is...https://community.dpgplc.co.uk/blog/kirkpatrick2018-07-06T08:32:00.000Z2018-07-06T08:32:00.000ZAdy Howeshttps://community.dpgplc.co.uk/members/AdyHowes<div><img src="https://storage.ning.com/topology/rest/1.0/file/get/49941880?profile=RESIZE_400x&width=400"></div><div><p>With many different types of evaluation models available to use, its sometimes quite hard to decipher which one is best for organisations or project’s. As L&D professionals, the importance of demonstrating the business value of learning & development initiatives must be known. According to the CIPD, “By evaluating learning and development programmes, employers are better able to ensure that these initiatives are aligned with their business objectives and overall learning and development strategy.”</p>
<p>A survey of our DPG Community members revealed that <a href="https://www.dpgplc.co.uk/courses-qualifications/learning-and-development/kirkpatrick-four-levels-evaluation-certification-programme/" target="_blank" rel="noopener">Kirkpatrick</a>is the most commonly used method.</p>
<p>It’s over 50 years since Donald Kirkpatrick published his training evaluation model. It quickly became adopted within L&D and became the most widely used evaluation model in modern times. The <a href="https://www.dpgplc.co.uk/courses-qualifications/learning-and-development/kirkpatrick-four-levels-evaluation-certification-programme/" target="_blank" rel="noopener">Kirkpatrick Evaluation Model</a> has recently been adapted to The New World Kirkpatrick Model, to meet the needs of 21st century Learning and Development teams. The New World Kirkpatrick Model encourages learning professionals to measure and track success on four levels. Rather than start with the learner and ‘happy sheets’ the model flips itself to ensure that the business objectives and measurements are agreed with key stakeholders and leading indicators are identified to gauge success.</p>
<p>According to Lesley Blower, Learning Resource Manager at utility giant Npower who completed a <a href="https://www.dpgplc.co.uk/courses-qualifications/learning-and-development/kirkpatrick-four-levels-evaluation-certification-programme/" target="_blank" rel="noopener">Kirkpatrick programme</a> with us, <strong>"….the light bulb moment was the realisation that you start at level 4, identify your desired results and leading indicators to measure if you’re on track, and design your training to reflect that outcome."</strong></p>
<p>The levels in the model are:</p>
<ul>
<li><strong>Level 4 - Results</strong> – The degree to which targeted outcomes occur as a result of the training and the support and accountability package</li>
<li><strong>Level 3 - Behaviour</strong> – The degree to which participants apply what they learned during training when they are back on the job</li>
<li><strong>Level 2 - Learning</strong> – The degree to which participants acquire the intended knowledge, skills, attitude, confidence and commitment based on their participation in the training</li>
<li><strong>Level 1 - Reactio</strong>n – The degree to which participants find the training favourable, engaging and relevant to their jobs</li>
</ul>
<p>Find out more about joining thousands of other L&D professionals who have gained the internationally recognised Kirkpatrick certification by clicking here.</p>
<p>How do you evaluate the learning initiatives for your organisation? How do you demonstrate value?</p>
</div>The #LoveCPD Spotlight Series September editionhttps://community.dpgplc.co.uk/blog/the-lovecpd-spotlight-series-september-edition2017-10-06T08:35:49.000Z2017-10-06T08:35:49.000ZLucy Boltonhttps://community.dpgplc.co.uk/members/LucyBolton<div><img src="https://storage.ning.com/topology/rest/1.0/file/get/2217382?profile=RESIZE_400x&width=400"></div><div><p>This months spotlight focuses on:</p>
<ul>
<li>
<p>What evaluation model are you most likely use to demonstrate the value of your learning interventions?</p>
</li>
<li>
<p>Social media in recruitment</p>
</li>
<li>Personality tools for leaders</li>
</ul>
<ul>
<li>Listening to music while you work</li>
</ul>
<p></p>
<p>To view this months spotlight online <a href="http://docs.dpgplc.co.uk/Spotlight/September/September_flipbook/" target="_blank">click here.</a></p>
<p>To download a copy of this months spotlight click here: <a href="http://docs.dpgplc.co.uk/Spotlight/September/September_Spotlight.pdf" target="_blank">Download</a></p>
<p></p>
<p>To help us out with next months spotlight issue, click into each of the zones and enter your vote into each of the polls on the right hand side!</p>
<p>If you have any ideas on what next months topics could be, share them in the comments below!!</p>
</div>Develop your evaluation skills!https://community.dpgplc.co.uk/blog/develop-your-evaluation-skills2017-06-26T16:08:34.000Z2017-06-26T16:08:34.000ZDebora Figueiredohttps://community.dpgplc.co.uk/members/DeboraFigueiredo<div><img src="https://storage.ning.com/topology/rest/1.0/file/get/2217331?profile=RESIZE_400x&width=400"></div><div><p>Hi everyone,</p>
<p>Here's a great opportunity for you to network with other L&D professionals and line managers and gain the internationally-recognised Kirkpatrick Evaluation Certification.</p>
<p>In this 2-day open programme, you will:</p>
<ul>
<li>learn how to evaluate the effectiveness of training in your business </li>
<li>apply your knowledge back in your organisation to demonstrate the effectiveness of your learning interventions</li>
<li>understand how to effectively bridge the gap between learning and organisational development</li>
</ul>
<p>The Kirkpatrick Model teaches you the four levels of training evaluation starting with the desired outcome: <strong>Results</strong>, <strong>Behaviour</strong>, <strong>Learning</strong> and <strong>Reaction</strong>.</p>
<p>Hope to see you in London on the 11th & 12th September.</p>
</div>17 Visualisation Tools to Make Your Data Beautifulhttps://community.dpgplc.co.uk/blog/17-visualisation-tools-to-make-your-data-beautiful2015-04-20T14:22:23.000Z2015-04-20T14:22:23.000ZMike Collinshttps://community.dpgplc.co.uk/members/MikeCollins<div><img src="https://storage.ning.com/topology/rest/1.0/file/get/2216469?profile=RESIZE_400x&width=400"></div><div><p><span>Data is key for Learning and HR professionals. Interpreting data to identify trends and patterns providing important business intelligence to make key decisions is a key requirement for any modern professional. The way in which we then display data also needs to be visual, interesting and relevant. It also needs to be easy to understand so when we came across these 17 tools to help bring your data to life and to make them beautiful we had to share it. </span></p>
<p><span>**Curated from the <a href="https://econsultancy.com" target="_blank">Econsultancy</a> website** </span></p>
<p><em>You don’t need to be a seasoned data scientist or have a degree in graphic design in order to create incredible data visualisations.</em></p>
<p><em>It has become a lot simpler to mine your data and interpret your insights in an engaging, attractive, and most importantly easy to understand way.</em></p>
<p><em>This is a list of various different free and premium tools and platforms available which will allow you to communicate data in a variety of different formats, from interactive charts, to infographics, to maps, to word clouds.</em></p>
<p><em>Not all will be suitable for your particular data set or your audience, but its certainly worth experimenting with a few to seek out the most relevant. </em></p>
<p><em>Hopefully you’ll not only learn some new skills and find some new creative tools, but also learn something about your own data.</em></p>
<p>We'd love to hear if you use any of these tools to bring your data to life and make it beautiful.....even in any assignment you might do.</p>
<p>Get going data lovers...</p>
<p><a href="https://econsultancy.com/blog/66131-17-visualisation-tools-to-make-your-data-beautiful/" target="_blank">17 Visualisation Tools to Make Your Data Beautiful</a></p>
<p></p>
<p></p>
</div>Thoughts from LDP in Manchester today....Evaluation!https://community.dpgplc.co.uk/blog/thoughts-from-ldp-in-manchester-today-evaluation2015-04-14T12:41:26.000Z2015-04-14T12:41:26.000ZKrystyna Gaddhttps://community.dpgplc.co.uk/members/KrystynaGadd<div><img src="https://storage.ning.com/topology/rest/1.0/file/get/2216453?profile=RESIZE_400x&width=400"></div><div><p><a rel="nofollow" href="https://howtoacceleratelearning.files.wordpress.com/2015/04/elephantbum.png" target="_blank"><img src="https://howtoacceleratelearning.files.wordpress.com/2015/04/elephantbum.png?w=300&width=256" class="align-right" width="256" /></a></p>
<p>Today I am with a fab group of trainers in Manchester, looking at evaluation. I asked the group what they thought about evaluation having had time to consider all 4 levels of Kirkpatricks model and so here is an insight shared by one of the participants Adam Gilroy:</p>
<p>"As it is at the back end, we rarely give evaluation the thought it requires and in future I definitely will give it more thought".</p>
<p>My immediate thought turned to the use of the phrase "the back end" and herein lies the problem. Why are we only considering evaluation at the back-end and not at the front-end?</p>
<p>If we consider evaluation at the front end with needs analysis, we can see what is already in place that we can measure, to help in the evaluation process. Also in this way we can engage stakeholders at the start to support learners in:</p>
<ul>
<li>Pre-work</li>
<li>Change in behaviours</li>
<li>Post-learning follow up and coaching</li>
<li>Measurement of the impact of the learning</li>
</ul>
<p></p>
<p>So when you think of evaluation in the future, will it be at the back .... or front-end?</p>
<p></p>
</div>Maximising the Impact of Short-term Coachinghttps://community.dpgplc.co.uk/blog/maximising-the-impact-of-short-term-coaching2014-10-20T11:57:40.000Z2014-10-20T11:57:40.000ZAngela Armstronghttps://community.dpgplc.co.uk/members/AngelaArmstrong195<div><img src="https://storage.ning.com/topology/rest/1.0/file/get/2216309?profile=RESIZE_400x&width=400"></div><div><p>This article was recently published in Coaching World. I thought you might be interested.</p><p><strong>Maximizing the Impact of Short-Term Coaching</strong></p><p>Sometimes organizations have a point-need for coaching rather than embarking on a comprehensive long-term coaching strategy. With such an intense focus on short-term demonstrable impact it takes an experienced and confident coach to step up, step in, get the results and step out. These six tips, gleaned from five years’ experience coaching managers and leaders across a range of industries, will help to ensure that the coaching impact is achieved in the short-term and sustained long-term.</p><p><strong>1. Clearly define the engagement’s context and outcome.</strong></p><p> Having absolute clarity on the context and outcome of the coaching intervention is critical to success in any coaching engagement, but especially so for short-term coaching. Notice I said outcome, not outcomes: When addressing a particular goal or concern, doing one thing well at once focuses everyone’s efforts in the same direction.</p><p><strong>2. Ask yourself whether you can deliver value.</strong></p><p>There can be a temptation for coaches to take on every engagement that presents itself; it is important that you also take care of yourself in order to better serve others. The high accountability for delivering results through others coupled with the objectivity of the outcome adds a background tension to delivery; some coaches flourish under the pressure and actively seek it but others find it distracts them from delivering their best.</p><p>To deliver results quickly requires a high degree of openness from the coachee; it is time well spent to consider whether you will be able to develop a strong rapport and trust with the coachee from the very first meeting. Of course the coach needs to be <b>credible</b>, <b>capable</b> and <b>confident</b>; it will also be helpful if you are <b>relevant;</b> i.e., if you have an understanding of the client’s context and industry, ways of working, gender and culture.</p><p><strong>3. Engage the appropriate parties.</strong></p><p>To deliver short-term demonstrable impact, the coachee, his or her supervisor, the human-resources representative, and the coach must be open and transparent in discussing the context and outcome; there is no place for a hidden agenda. Of course you’ll handle it sensitively but in addition to discussing the beneficial outcomes, also ask, “What is the consequence for the coachee if the outcome is not achieved?” Will he or she be passed over for a promotion? Fired? Understanding the consequence is part of fully understanding the context. </p><p>The cornerstone of a successful short-term coaching engagement is a coachee who is ready, willing and able to engage fully with the coaching process. Pay attention, trust your instincts and discuss any aspect of the engagement that seems questionable until all parties are satisfied.</p><p>The written coaching agreement will include the usual confidentiality, commercial and logistic sections; it should also include the measurable outcome along with the baseline and target measures (e.g. increase percentage of actual sales vs. target sales from current value of 96 percent (today’s date) to target of 100 percent by a specific date). If there is a management report in which this information is available it can be included in an appendix to the coaching agreement as the baseline. More qualitative measures include 360-degree feedback reports etc. If the coaching fees are offered as a gain-share agreement also state the percentage of the fee at risk for non-delivery and the incentive arrangements for exceeding expectations</p><p><strong>4. Ensure the right “fit.”</strong></p><p>Maximizing the impact from short-term coaching generally requires a challenging—and accelerated—approach, with the coach asking searching questions and often touching on core beliefs and values very early in the coaching process.</p><p>Quickly establishing trust and rapport is aided by having an informal, 30-minute ‘”getting-to–know-you” discussion immediately before the first coaching session. You might even consider including some questions about your coachee’s interests and hobbies in your pre-coaching questionnaire.</p><p>Although following the first three steps increases the likelihood of a good fit, my coaching agreement calls for a formal decision point after the first coaching session so that both the coachee and I have the option to curtail or reshape the coaching agreement if we have concerns.</p><p><strong>5. Find the right balance of “support” and “challenge.”</strong></p><p>Traditional coaching has a tendency towards support but in an organizational context you need more than just a strong relationship—you also need challenge to create sustained change in the individual and the organization.</p><p>Low support paired with high challenge is unhelpful and generates stress, but high support and high challenge equals what I refer to as “tough empathy.” Support is given through trust, rapport and building the relationship. Challenge is given through feedback, accountability, setting courageous goals, balancing the tension to move the coachee out of their comfort zone and looking at situations from a systems mindset that considers the needs of all stakeholders.</p><p>The coach, in taking on a short-term coaching agreement with objective outcomes, is also modeling the behaviors that many organizations seek when they select a coach for a point-need coaching intervention.</p><p>The powerful combination of demonstrating genuine care for the coachee, seeing the world through their eyes, unlocking their full potential through powerful deep questioning and mutual accountability for measurable results in a defined time-frame acts as a catalyst for the coaching and makes it possible to deliver results quickly. To get fast results the coach has to address the root causes, not the symptoms; the underlying beliefs, not the observable actions.</p><p><strong>6. Quantify impact and set the stage for sustainable change.</strong></p><p>Demonstrating the achievement of the target key performance indicators is simply a matter of comparing the baseline, actual and target measures. The coachee will also likely want to include some additional qualitative benefits in the coaching outcomes report.</p><p>Encourage the coachee to include in the coaching outcomes report three ways in which their mind-set has changed as a result of the coaching and three new habits that will sustain the coaching outcomes; it helps the coachee to consolidate the coaching and provides a point of reference when you follow up in six months’ time to confirm that the coaching has led to sustained behaviour change.</p><p>By having the courage and confidence to ask searching questions from the outset and provide consistent challenge it is possible to get fast results that last.</p><p>Author: Angela Armstrong</p><p>Angela is a high-performance coach and trainer. Her sweet spot is developing leaders in IT and engineering using solution-focused coaching and “tough empathy.” She is a leader at the Achievers Academy for Women and a member of ICF United Kingdom.</p><p> </p><p> </p></div>Discovering the new world of Kirkpatrick Evaluationhttps://community.dpgplc.co.uk/blog/discovering-the-new-world-of-kirkpatrick2014-10-08T16:25:53.000Z2014-10-08T16:25:53.000ZMike Collinshttps://community.dpgplc.co.uk/members/MikeCollins<div><img src="https://storage.ning.com/topology/rest/1.0/file/get/2216322?profile=RESIZE_400x&width=400"></div><div><p>This post was recently featured in <a href="http://www.trainingzone.co.uk/feature/training-cycle/discovering-new-world-kirkpatrick-and-exploring-what%E2%80%99s-changed-learning" target="_blank">Training Zone</a> as part of a 3 part series focusing on evaluation.</p>
<p><strong>A long long time ago</strong></p>
<p>When the Kirkpatrick Model was established back in 1954, it was instantly accepted as the industry standard approach to training evaluation. Today it is still the most frequently used model for training evaluation, but does this mean it is still effective in today’s modern world of L&D?</p>
<p>Every model or framework has its critics and I called in to question the validity of the original Kirkpatrick model because it focuses solely on the training event, and not anything that takes place before or after the training event.</p>
<p><strong>Are YOU about to be made s</strong><strong>urplus to requirements?</strong></p>
<p>Fast-forward 60 years to 2014, and the one thing that is more crucial than ever is the need for L&D to measure the success of learning activities and demonstrate:</p>
<ol>
<li>
<p>Tangible value added to the business</p>
</li>
<li>
<p>Measurable improvements in performance</p>
</li>
<li>
<p>Positive changes in behaviour of the learners</p>
</li>
</ol>
<p>We all know budgets are being squeezed, work practices are evolving and technology advancing at an incredible rate of knots, and if us L&D professionals are not careful we are at serious risk of becoming surplus to requirements, side stepped by the very people we are there to support. <span>Now more than ever, </span><strong><a href="http://www.dpgplc.co.uk/what-we-offer/cipd-qualifications/cipd-diploma-in-ld-management-%28intermediate-level%29.aspx">we</a></strong><span> need to be able to demonstrate that the learning solutions we design and develop improve performance and maximise business results.</span></p>
<p>Training is no longer the sole currency of L&D departments; there is a now a strong focus on supporting and driving performance improvements. We need to re-think the whole approach to organisational learning. Training is still part of this puzzle but times have changed.</p>
<p><strong>Welcome to the new world of L&D evaluation</strong></p>
<p>The Kirkpatricks have also changed and evolved their model to underpin this L&D transformation. “<strong><a href="http://www.dpgplc.co.uk/what-we-offer/kirkpatrick-evaluation-programmes/become-an-evaluation-expert.aspx">The New World Kirkpatrick Model</a></strong>”, developed by Don, Jim and Wendy Kirkpatrick, with the help of global Kirkpatrick ambassadors, provides a robust consultative framework for us to use. This framework moves our L&D teams from mere order takers to trusted business partners who help shape and influence the future strategies of the organisations we work within.</p>
<p>As L&D professionals we need to be influencing and guiding, engaging and nurturing, consulting with and talking the same language of the business we are supporting. <span>The L&D team I want to be part of, and be proud to be part of, is one that sits at the heart of the business and has the credibility to stand up and be counted. One that is respected and admired and one that can successfully lead organisations in to the future. So what does this mean for evaluation in this new world?</span></p>
<p><strong>ROI is dead – long live ROE</strong></p>
<p>I believe there needs to be a significant shift in the way evaluation happens. This shift needs to move our industry away from the current ROI (Return on Investment) culture where we attempt to mathematically isolate a single training event using a complex and convoluted formula to give an arbitrary £ or $ value that none of us would feel the slightest bit confident standing up in a court of law to defend.</p>
<p>Instead we must shift towards measuring the learning, behaviours and performance changes and how they compare to the desired and expected outcomes set and agreed with the business at the outset. This is ROE (Return on Expectations.) These results are observable, measurable and lead to genuine improvements in performance. These are results we do feel confident defending in a court of law or more relevantly the boardroom.</p>
<p>I believe the New World Kirkpatrick Model is a framework that enables better conversations to take place and to understand business needs in a way that I’ve not seen from any other evaluation model.</p>
<p>Learning professionals should now be using this evolved model as they transform and evolve themselves beyond the traditional design, development and delivery of training events. It is not about our industry surviving, but about thriving and embracing the opportunities that exist.</p>
<p><strong>The seven rules for evaluation success</strong></p>
<p>Our challenge is to stop thinking of the Kirkpatrick evaluation model starting with Level 1 (Reaction) before moving through the gears on autopilot to Level 2 (Learning) and then attempting to come back to Level 3 (Behaviour) and Level 4 (Results). It’s time to flip the model and start with the end in mind. Ask those business-critical questions in a consultative manner and use a framework that:</p>
<ol>
<li>
<p>Creates strong business partnerships with business leaders as the cornerstones for success</p>
</li>
<li>
<p>Starts any initiative knowing what the targeted and measurable business results will be</p>
</li>
<li>
<p>Introduces return on expectations (ROE) as the method to demonstrating value</p>
</li>
<li>
<p>Connects the levels in planning and practice, as opposed to evaluating them as 'separate entities'</p>
</li>
<li>
<p>Broadens the concept of learning interventions to include all manner of support and accountability to increase on-the-job application and performance support</p>
</li>
<li>
<p>Includes the concepts of ‘required drivers’ at Level 3 and ‘leading indicators’ at Level 4</p>
</li>
<li>
<p>Creates a 'chain of evidence' that is done in partnership with business stakeholders not in isolation.</p>
</li>
</ol>
<p><strong>Making the transition from training to performance</strong></p>
<p>The continued evolution of <strong><a href="http://www.dpgplc.co.uk/what-we-offer/cipd-qualifications/cipd-diploma-in-ld-management-%28intermediate-level%29.aspx">Learning & Development professionals</a></strong> and approaches to workplace learning has seen a major shift in focus from training to performance. The impact of technology, especially social media, has seen new approaches adopted within the workplace that places theories like the 70/20/10 framework at the heart of learning in the organisation.</p>
<p>This shift places more emphasis on learning on the job (and the subsequent increase in performance) and away from traditional classroom-driven events and practices. These new approaches utilise collaboration and peer-to-peer sharing at the heart of improving workplace performance and capability.</p>
<p><strong>The new industry standard evaluation model</strong></p>
<p>The theory and practices of the New World Kirkpatrick Model enable clear strategies to be developed in a truly collaborative way. Strategies that reflect the different stages of delivery, execution and support with clear focus on transferable knowledge and skills, ultimately leading to a positive change in performance and results.</p>
<p>Why do I say this? Because it provides you with a simple to follow methodology to demonstrate how any learning intervention can be measured against a set of pre-determined objectives and critical success factors agreed at the outset and in partnership with your customers and stakeholders.</p>
<p><strong>Moving L&D from the classroom to the boardroom</strong></p>
<p>This is the key to L&D taking a seat at the top table next to our HR counterparts. No longer will L&D be seen as the poor relation to HR once we start to demonstrate the value we know we can. For everyone working in the L&D profession this can only be a good thing.</p>
<p></p>
<p></p>
</div>Evaluation - going beyond the happy sheet!https://community.dpgplc.co.uk/blog/evaluation-going-beyond-the-happy-sheet2013-10-30T23:07:50.000Z2013-10-30T23:07:50.000ZRachel Chalmershttps://community.dpgplc.co.uk/members/RachelChalmers<div><p>Having just completed Workshop 8 on Evaluation, I thought this was a great article on going beyond the 'happy sheet'. Printing a copy for my team meeting tomorrow as this is definitely and area we need to improve :-)</p><p> </p><p><a rel="nofollow" href="http://www.trainingzone.co.uk/feature/superheroes-ld-episode-1/185523">http://www.trainingzone.co.uk/feature/superheroes-ld-episode-1/185523</a></p></div>Collaborate to Evaluatehttps://community.dpgplc.co.uk/blog/collaborate-to-evaluate2013-06-26T15:35:44.000Z2013-06-26T15:35:44.000ZLucy Malleyhttps://community.dpgplc.co.uk/members/LucyMalley<div><p><a target="_self" href="http://storage.ning.com/topology/rest/1.0/file/get/1357312?profile=original"><img class="align-center" src="http://storage.ning.com/topology/rest/1.0/file/get/1357312?profile=original" width="499"></a>In my last blog, I talked about my personal introduction to the world of evaluation with the New World Kirkpatrick Model. Having learnt all about the Four Levels Model and how to effectively apply this theory to display return on expectations, I was really eager to learn more about the other evaluation methodologies out there.</p>
<p> The second programme I attended took a much more quantitative view of evaluation than I had experienced with the Kirkpatrick Certification, although the Phillips Return on Investment (RoI) programme had a similar 2-day, learning-then-doing format. I’d never thought about learning and development from a purely financial view before; for me, it’s very much about soft skills acquisition, building a team and growing as an organisation, so it was interesting to see it all broken down into numbers and financial returns.</p>
<p><a target="_self" href="http://storage.ning.com/topology/rest/1.0/file/get/1357329?profile=original"><img class="align-left" src="http://storage.ning.com/topology/rest/1.0/file/get/1357329?profile=original" width="300"></a>For some areas and in certain industries, RoI works really well and can be a great way to justify that you are getting back the money you’re spending on development. One problem I found with it though; RoI can’t really go into the inherent value of customer service, branding, and all those other things that are not as easily quantifiable as ‘time saved’. It’s these intangibles that, for me, are the basis of Learning and Development- that essential foundation, not particularly quantifiable but nonetheless vital to success, to up-skilling - and ultimately to the bottom line.</p>
<p></p>
<p>An interesting way to look at successful RoI is through the classic call centre training environment- think more efficient phone calls, time saved, all that good stuff. In a situation like this, there are plenty of things that can be measured; and these things all cost money, so you can do an effective RoI on how much money the training has cost vs. the amount of money saved or the increase in revenue (or hopefully, both). Okay, so let’s think about call centres for a minute. Living in an era of technology, getting to actually speak to a real person is increasingly difficult a task. And, by the time you do get through… you might be a little less than cheerful. So, imagine then the relief when you’re put through to someone who is just incredibly good at their job. Friendly, <a target="_self" href="http://storage.ning.com/topology/rest/1.0/file/get/1357352?profile=original"><img class="align-right" src="http://storage.ning.com/topology/rest/1.0/file/get/1357352?profile=original" width="300"></a>helpful, knowledgeable, going the extra mile so that you leave that call with an overwhelmingly positive image of the organisation. You’d mention that to people, wouldn’t you? They might be looking for a new service provider. They remember your conversation, and decide to go with the same company as you because they can be assured that if something goes wrong, it will be resolved. They have a positive experience themselves. They mention it to people in their own circles. And, just like that, it snowballs into a fantastic reputation and great corporate success.</p>
<p> Now, that organisation may well have spent massive amounts of money on training its call centre staff. And it is true that part of the return on this is financially quantifiable efficiency; each employee can now answer more calls, save more time, make the company more money and provide a return on investment for the training. The effect of that training on overall organisational branding, however – this isn’t quantifiable. But it is nevertheless massively important. And it can make you a lot of money.</p>
<p>This was really my main niggle with the Phillips RoI programme, and the gaps that the Kirkpatrick programme filled in for me; it was great learning how to mathematically demonstrate RoI in certain circumstances, but sometimes you need to justify your learning and development programmes by stepping back and taking a look at the bigger picture. Training and subsequent evaluation should be a consultative process; there should be a collaborative approach to ensuring the right solution is selected and delivered so that there is a genuine return on expectations. For me, interesting as it is to add a mathematical element to evaluation, I find that in using RoI exclusively, the roles of consultation and collaboration in effective evaluation are massively underplayed.</p>
<p>Do you evaluate your training?</p>
<p>Which evaluation methodologies deliver for you - and why?</p>
<p>Get in touch, I'd love to hear more about your own experiences of evaluation and evaluation training programmes!</p></div>Evaluation - Between the Happy Sheetshttps://community.dpgplc.co.uk/blog/evaluation-between-the-happy-sheets2013-06-14T10:37:17.000Z2013-06-14T10:37:17.000ZLucy Malleyhttps://community.dpgplc.co.uk/members/LucyMalley<div><p><a target="_self" href="http://storage.ning.com/topology/rest/1.0/file/get/1357469?profile=original"><img class="align-center" src="http://storage.ning.com/topology/rest/1.0/file/get/1357469?profile=RESIZE_480x480" width="444"></a>I consider myself, above all things, a learner. A recent graduate, I’ve spent the majority of my adult life in an educational environment, to the point where I feel, rightly or wrongly, somewhat of an expert in learning as a process on the participatory level. And, above all – I love to learn. Which is why I jumped at the opportunity to take my classroom learning as a benchmark, and from it, learn about (and attempt to become an expert in) something completely foreign to me.</p>
<p>As a newcomer to any industry, there is always a lot to learn; and the L&D world is absolutely no exception. Evaluation is something I’d only ever experienced as a learner, a happy-sheet-ticker if you will. It had never really crossed my mind that an incredible amount of thought goes into designing those tick-boxes, and indeed that a lot can (and should!) be done to evaluate past this point. I’d considered that the comments I’d written on programme evaluations might help to build and positively affect future programmes, but I never looked beyond that at evaluation from the wider viewpoint of view of a university, or any organisation- and how to measure not only the reaction of a participant but the subsequent learning, behaviour change and impact that learning can have.</p>
<p><a target="_self" href="http://storage.ning.com/topology/rest/1.0/file/get/1357483?profile=original"><img class="align-right" src="http://storage.ning.com/topology/rest/1.0/file/get/1357483?profile=original" width="268"></a>I can honestly say that while I may have experienced materials, educators and even programmes that I have not enjoyed, or that haven’t been right for me - I have never had an overall negative learning experience. However, I’ve also never really considered any impact further than knowledge acquisition, perhaps as behaviour change and results are far more transient concepts at University than in a business environment. Thinking about this and attending programmes with learning and development professionals really brought to light for me the relative fragility of L&D- and with this, the absolute necessity to justify training programmes and prove the validity and necessity of development from a business perspective. After a couple of months working in an L&D environment, I became increasingly aware that I’d only experienced one side of evaluation; so when I was offered the opportunity to gain a more in-depth knowledge of the topic, I was excited to explore the value it could provide- not just for the participants, but all the way down to real business impact and positively affecting the bottom line.</p>
<p>My official introduction to evaluation, and the first programme I attended, was the New World Four Levels Kirkpatrick Model. I’d read about the model- and how it been updated from the original 1950s model to apply directly to a contemporary learning environment- but experiencing the programme first-hand really brought to life to me the absolute essential that is effective evaluation at every level. The 2-day programme was split into a day of theory with activities and exercises, and a second thoroughly interactive day. There was a lot to learn in day 1, but having a good group size and more than just one facilitator meant that is was all very engaging and genuinely interesting. Having learned all about taking evaluation from reaction, through learning, and all the way to behaviour change and results, day 2 of the programme was a fantastic opportunity to spend some time in small groups and work one-on-one with the facilitators. Each attendee put together and presented a business case for their respective organisations, working through the leading indicators to show training effectiveness and demonstrating the value of their training programmes, from reaction to results. For me this was essential because it really focused on the practical application of the methodology – using it to justify the value of and need for learning and development programmes within any organisation.</p>
<p><a target="_self" href="http://storage.ning.com/topology/rest/1.0/file/get/1357488?profile=original"><img class="align-left" src="http://storage.ning.com/topology/rest/1.0/file/get/1357488?profile=original" width="400"></a>As well as leaving the 2-day programme with a tangible and effective business case, I found that I was also armed with a bank of very transferable knowledge and skills. I really appreciated how the Kirkpatrick Four Levels model, rather than focusing solely on Return on Investment (which can lead to a side-lining of the Learning and Development department, and an implied loss in its intrinsic value) really emphasised L&D in the context of a business as a collaborative entity. Learning and development in isolation is not an effective way to implement change or a cost-effective way to deliver training that works. The collaborative approach of the Kirkpatrick model showed me how much effective training can occur when individual department/s work together for the greater good of an organisation. </p>
<p>Essentially, the Kirkpatrick Four Levels model is more than just a ‘learn to evaluate’ methodology; it is a consultative tool to help learning and development add maximum value, and it carries through on this promise with each attendee leaving the programme with a bespoke business case proposal to show the real impact of training. To me, that is valuable in ways that discussions on numerical returns alone can’t always be; evaluation of effective training is fundamentally about working together, starting with the end in mind and working collaboratively. The Kirkpatrick model comes through on all these promises and more, and has got me really excited to delve deeper into the world of evaluation- in my next blog I’ll be talking about my experiences on the Phillips ROI programme, and if you have any thoughts or things you’d like to share in the meantime I’d love to hear from you!</p></div>Planting a tree with the 4 levels of Kirkpatrickhttps://community.dpgplc.co.uk/blog/planting-a-tree-with-the-4-levels-of-kirkpatrick2013-04-30T14:00:00.000Z2013-04-30T14:00:00.000ZHenrik Svenssonhttps://community.dpgplc.co.uk/members/HenrikSvensson<div><p>I was attending a fantastic training event last week, the <a rel="nofollow" class="last" href="http://www.kirkpatrickpartners.com/TrainingEvents/tabid/57/Default.aspx">Bronze level of Kirkpatrick four levels evaluation certificate program</a>. I’m going to try to explain the model taught and (of course) used in class with an example a little bit more close to heart than “evaluate the result of a training program”.</p><h2>Let’s start from the back (at Level 4) – What do we want?</h2><p style="text-align: center;"><a rel="nofollow" class="last" href="http://ennoble.se/wp-content/uploads/2013/04/eatapple.jpg"><img class="wp-image-370 aligncenter last" alt="Healthy Snack" src="http://ennoble.se/wp-content/uploads/2013/04/eatapple.jpg" width="277" height="416"/></a></p><p style="text-align: center;"> </p><p>So, I and my family want to be able to pick our own apples in two years’ time and we don’t know a thing about planting or caring for trees. Therefor we’re starting up a tree planting project with homegrown apple pie as our first prize. This is our Level 4 <i class="last">Stakeholder Expectation</i>. Now, why do we start with the fourth and last level and not the first one? Because it actually makes more sense this way! Be patient and continue reading…</p><p>Picking and eating apples from our own tree is our stakeholder’s end result but if we’d take a look at the project as a whole its <i>Desired Result</i> would be to have a fruits to pick as well as a healthy apple tree. How do we <b>evaluate</b> this? The first part is easy: if we are able to pick fruits and eat them, that’s pretty strong evidence. The second part might require several methods. Our <i class="last">Leading Indicators.</i></p><p>With our end-result in mind, the next thing at Level 4 is to ask: What are the short-term observations and measurements (<i class="last">Leading Indicators</i>) that suggest we’re doing the right thing to improve the probability of us eating our own apples? These are the suggested things we must see to be certain that the tree will bear fruit within our time frame:</p><ul><li>Growth a certain amount of centimeters every 3 months</li><li>Blossoming each summer</li><li class="last">Healthy leafs without leaf lice or other pest</li></ul><p>All in all, it’s down to observing if the tree grows at it should. If it grows in the presumed amount each month and quarter – it probably is healthy enough to bear fruit.</p><h2>Down to Level 3 – What do we have to do?</h2><p>With these Leading Indicators in mind we move on to the <i>Critical Behaviors</i> of Level 3 instead. These are the “doing the right thing to improve probability of success” as described earlier, i.e. what do we have to DO to make sure things happen so that the tree bears fruit? For each of the things we need to do we also need ways of monitoring them as well as supporting them to happen with reinforcement, encouragement and rewards (<i class="last">Drivers</i>). Let’s list these in a table as shown below.</p><table border="1" cellspacing="0"><tbody class="last"><tr><td valign="top" width="145"><b class="last">Critical Behavior</b></td><td valign="top" width="179"><b class="last">Monitoring</b></td><td class="last" valign="top" width="276"><b class="last">Drivers (Reinforce, Encourage and reward)</b></td></tr><tr><td valign="top" width="145">Pick out an old enough tree so that it can bear fruit within our time frame</td><td valign="top" width="179"><ul class="last"><li>Asking the experts at the nursery-garden</li><li class="last">Book an appointment with nursery-garden expert to do home visit</li></ul></td><td class="last" valign="top" width="276"><ul class="last"><li>Looking at tree labels</li><li class="last">Asking the wife for allowance to buy a more expensive tree…</li></ul></td></tr><tr><td valign="top" width="145">Fertilize when needed to assure that the tree gets the nutrients it need to grow</td><td valign="top" width="179"><ul class="last"><li>Looking at the tree as a whole</li><li>Get feedback from family if the tree doesn’t look happy</li><li class="last">Blossoming observation</li></ul></td><td class="last" valign="top" width="276"><ul class="last"><li>Reminders in smartphone to check for growth in cm</li><li>Accountability from family (stakeholders)</li><li>Checklist from nursery-garden on care</li><li>Comparison growth-chart from nursery-garden</li><li class="last">The right tools</li></ul></td></tr><tr><td valign="top" width="145">Water the tree when required and some looking after if it rains to much</td><td valign="top" width="179"><ul class="last"><li>Looking at dryness/wetness of soil and looking at the tree as a whole</li><li class="last">Blossoming observation</li></ul></td><td class="last" valign="top" width="276"><ul class="last"><li>Accountability from family</li><li>Looking at weather forecast to see if it’s going to be very sunny or rainy</li><li>Checklist from nursery-garden on care</li><li class="last">The right tools</li></ul></td></tr><tr><td valign="top" width="145">Apply anti-pestilence in correct amount and time</td><td valign="top" width="179"><ul class="last"><li>Observation of amount of leaf lice or pest</li><li class="last">Blossoming observation</li></ul></td><td class="last" valign="top" width="276"><ul class="last"><li>Accountability from family</li><li>Reminders in smartphone to apply</li><li>Checklist from nursery-garden on care</li><li class="last">The right tools</li></ul></td></tr><tr class="last"><td valign="top" width="145">Perform correct branch cutting after each season</td><td valign="top" width="179"><ul class="last"><li class="last">Asking the experts at the nursery-garden</li></ul></td><td class="last" valign="top" width="276"><ul class="last"><li>Checklist (with pictures) from nursery-garden on care</li><li class="last">The right tools</li></ul></td></tr></tbody></table><h2>The learning Level 2 – What do we have to know?</h2><p>When we know what desired result we want and what critical behaviors are required to reach that result, it’s time to look at: What do we have to know to be able to perform those critical behaviors? Also, who, in the family, must increase their knowledge to do it right? Let’s make another table.</p><table border="1" cellspacing="0"><tbody class="last"><tr><td valign="top" width="132"><b class="last">Training Audience</b></td><td valign="top" width="252"><b class="last">Learning Objectives</b></td><td class="last" valign="top" width="216"><b class="last">Evaluation methods</b></td></tr><tr><td valign="top" width="132">The kids</td><td valign="top" width="252"><ul class="last"><li>Demonstrate care with footballs and Frisbees around the young tree</li><li class="last">Explaining the value of carefulness to other kids</li></ul></td><td class="last" valign="top" width="216"><ul class="last"><li>Observation of behavior</li><li>Daddy does it too (Executive modeling)</li><li class="last">Listening in on their conversation with other kids when playing football</li></ul></td></tr><tr><td valign="top" rowspan="5" width="132">The Dad</td><td valign="top" width="252">Know about:<p></p><ul class="last"><li>how and when to use fertilizer</li><li>how and when to water the tree</li><li>common lice and pest and how to combat them</li><li>optimal placement in garden</li><li class="last">Branch cutting</li></ul></td><td class="last" valign="top" width="216"><ul class="last"><li class="last">Self-assessment including demanding speeches from wife</li></ul></td></tr><tr><td valign="top" width="252">Skills:<p></p><ul class="last"><li class="last">Performing the critical behaviors in an accurate way.</li></ul></td><td class="last" valign="top" width="216"><ul class="last"><li class="last">Self-assessment including observation by wife</li></ul></td></tr><tr><td valign="top" width="252">Attitude</td><td class="last" valign="top" width="216"><ul class="last"><li class="last">Post-buy session with wife</li></ul></td></tr><tr><td valign="top" width="252">Confidence:<p></p><ul class="last"><li class="last">Performing the critical behaviors on other non-important plants in the garden</li></ul></td><td class="last" valign="top" width="216"><ul class="last"><li>Self-assessment</li><li class="last">Post-season conversation with nursery-garden experts</li></ul></td></tr><tr class="last"><td valign="top" width="252">Commitment:<p></p><ul class="last"><li class="last">Thinking of happy family all picking and eating apples together</li></ul></td><td class="last" valign="top" width="216"><ul class="last"><li class="last">Family photo album 10 years from now…</li></ul></td></tr></tbody></table><h2>Level 1 – How do we do it to make it a fun experience for the family?</h2><p>We’ve got our end-result and its indicators, our critical behavior and its drivers, and now we know what we have to learn for us to being able to do all of the new things. What’s left to figure out is what most people start with: In what way should we do this so that it’s as enjoyable as possible? Because, you know, if it all feels boring and dull, the risk is pretty big everyone just doesn’t follow through.</p><p>There are some things that should be looked at when considering what way to deliver the apple tree project. The family must feel:</p><ul><li><b class="last">Satisfied</b> during the project</li><li><b class="last">Engaged</b> by the project</li><li>The content of the project is <b class="last">relevant</b> to their need</li><li>It’s <b class="last">focused on them</b> (the learners)</li><li><b class="last">Evaluation is appropriate</b> to the project outcome (not too heavy and often)</li><li>There is <b class="last">variation</b> in the project</li><li class="last">The project has a clear <b class="last">alignment to the goal</b></li></ul><p>Since this apple tree project is what it is we’ll of course not use any classic Level 1 evaluation methods like surveys. That’s too much… We’ll appoint the rest of the family as dedicated observers and actually instruct them to, now and then, ask me questions like “How’s it going?”, “Do you need any help?” and “Is there any more information or tools you need to take care of the tree?”. This ought to take care of the Level 1 evaluation which leaves us with the “How do we do it?”.</p><ul><li>Let it be <a rel="nofollow" class="last" title="Get rid of the Telling – Start Involving!" href="http://ennoble.se/get-rid-of-the-telling-start-involving/">student driven and involving</a></li><li>Get a good classic gardening book with all the basics</li><li>Get a specific “Apple tree caring for dummies” book with checklists and growth chart job aids</li><li>A first meeting with the nursery-garden expert (when buying the tree)</li><li class="last">Tools for fertilizing, watering, branch-cutting etc.</li></ul><h2>Summary</h2><p>Instead of starting with the beginning in mind, i.e. “Let’s go get us a tree!”, we started this apple tree project with the end-result in mind and looked at what things would make us get there. These four levels of Kirkpatrick are not that hard to understand and implement so there really isn’t a reason for you to not do that in your organization. I suggest that you go visit <a rel="nofollow" class="last" href="http://www.kirkpatrickpartners.com/">Kirkpatrick Partners</a> and register for their newsletter and their separate resource page since there’s a wealth of information that’s available for free.</p><p class="last">Thanks Jim for a great couple of days in London and also to the crew at DPG!</p><p class="last">PS: If you enjoyed this post, find more at <a rel="nofollow" href="http://ennoble.se/">ennoble.se</a> ! DS</p></div>5 metrics to measure for effective Learning & Development managementhttps://community.dpgplc.co.uk/blog/5-metrics-to-measure-for-effective-learning-development2013-04-25T12:53:42.000Z2013-04-25T12:53:42.000ZBlake Heneganhttps://community.dpgplc.co.uk/members/BlakeHenegan<div><p>So often training departments are either deemed as ineffective or their effectiveness is completely unknown because of lack of measurement. Training isn’t as simple to measure as other functions such as sales where leads and bottom line sales can easily determine the success of a strategy.</p><p>A common complaint we hear is about managers who approve training with no thought of what success looks like, how it will happen or what success actually means to them.</p><p>Therefore companies should always have solid figures that they can use to measure the effectiveness of their training activities. We think there are 5 broad categories which you can use to judge and compare the success of your training department. They are:<span id="more-3654"></span></p><ol><li>Training spend</li><li>Training recipients</li><li>Time spent training</li><li>L&D Staff</li><li>Training Evaluation & Feedback</li></ol><p><b>Training Spend</b></p><p>Break down the spend into meaningful categories including but not limited to categories like:</p><ul><li>Internal courses</li><li>External courses</li><li>Travel & accommodation</li><li>Training materials</li><li>Technology e.g. pcs, apps, tablets</li></ul><p>These can all be totalled and viewed as a sum that can be compared with competitors and similarly sized companies to see the size and activity of your department.</p><p><b>Training Recipients</b></p><p>Understand who from your employees have undergone training. So you might want to consider categories such as:</p><ul><li>Training category e.g. Management, Sales</li><li>Staff level e.g. senior management, middle management, frontline staff</li><li>Departmental spend</li></ul><p>These are very important as often amount of money spent doesn’t translate to hours of learning per employee.</p><p><b>Time spent training</b></p><p>This is the total amount of hours spent each year on training by employees in your organisation. It is useful to know the combined figure however you need to dig a bit deeper so you know how much time, what topics, who received the training. Senior managers and board level tend to be interested in this figure and the total cost so if you can break it down even more it looks impressive. </p><p><b>L&D Staff</b></p><p>An often overlooked figure. This is focused on the number of all employees that are responsible for organising and delivering training. This will likely include L&D managers, administrative support, coaches, and instructors. This is sometimes difficult to measure as HR managers for example are often responsible for a number of tasks outside of training. In these examples it is best to work out roughly what percentage of their time is concerned with the training tasks and use that to work out therefore how much of their salary is dedicated to training. You need to know these figures and relate back to business metrics such as employee headcount, financial data. Knowing this means you will be able to calculate numbers and costs for when the company either increases or decreases in size.</p><p><b>Training Evaluation & Feedback</b></p><p>How can you ensure that the training organised is having a positive impact and is therefore worth investing in? All L&D professionals should be able to provide this information as this supports everything they do. Directors will often talk about Return on Investment (ROI) however you also need to consider Return on Expectations (ROE) i.e. what did the manager expect to happen as a result of the training. This line of thought is not limited to training and should be encouraged within the business. Some measures will be quantitative such as course scores and some should be qualitative such as comments made.</p><p>These 5 metrics can then be used to work out numerous other important figures – average amount of hours of training per employee, average spend per employee, number of training staff per employee amongst others are useful data to know.</p><p>This data is pivotal for any organisation to know as it can be used to benchmark against other companies to see how effective trainers you are. You can compare against older figures to see what impact a new strategy has had, analyse if raising expenditure has an effect on the other metrics and lots of other analysis.</p><p>If you don’t currently collate this information then you need to start doing this. It doesn’t have to be complicated, a simple excel spreadsheet can be the starting point to record all the information to then analyse it. Or if you work with external suppliers such as ourselves then ask them to produce reports on a regular basis. Why not approach L&D staff from other companies you know or even rival companies, this happens more than you think.</p><p>The approach each company will take should depend on a number of factors specific so the main message is make sure you are doing some form of measuring.</p><p>Have we missed any? Do you have practical experiences you can share? Please comment and let us know.</p><p><strong>Blake Henegan and Scott Drayton – Optimus Sourcing. </strong></p></div>Proactive or reactive L&D?https://community.dpgplc.co.uk/blog/proactive-or-reactive-l-d2013-04-23T09:52:45.000Z2013-04-23T09:52:45.000ZKrystyna Gaddhttps://community.dpgplc.co.uk/members/KrystynaGadd<div><p><a rel="nofollow" href="http://howtoacceleratelearning.files.wordpress.com/2012/08/lightbulb-2.png"><img alt="lightbulb-2.png" src="http://howtoacceleratelearning.files.wordpress.com/2012/08/lightbulb-2.png?w=90" width="90" height="150"/></a>I have recently been participating in a discussion on LinkedIn about "What makes a great trainer" in the "How to Market Training" group and this has sparked off a great debate.</p><p>While some people mention empathy , passion and being learner focused, this question has focussed me to realise that all of the qualities discussed are useful to a point ........however...... if the learning/training you provide does not meet an organisational need, then it is all just window dressing.</p><p>Yes all of these qualities are essential, but if the organisational needs are not met, then L&D, the trainers will have no credibility and in this economic climate, that is a dangerous place to be.</p><p>'But we get told to roll out the training, and that is it. We don't really have time to find out what the organisation needs. And in any case, they say "jump and we ask "how high?"' This was my response one day when I entered into a debate on the subject in a forum in Leeds.</p><p>As I argued why, one L&D manager from a well known telecommunications company kept quiet, but smiled gently. She then shared the story of how they had the same problem 18 months before that, but then began to challenge the status quo. They began by asking for 10 minutes of the stakeholders time to clarify the problem so that they could come up with the best solution. At first there was some resistance, but then slowly the stakeholders would come to her, with the answers along with the required measures to ensure the learning had been effective.</p><p>It did not happen overnight - all in all it took around 18 months. Now learning and training are prioritised according to the needs of the organisation. Isn't that how it should be?</p><p>So what are you - reactive or proactive? When stakeholders say "jump" do you ask "how high" or "why?"</p></div>Evaluation - What's In It For Me?https://community.dpgplc.co.uk/blog/evaluation-what-s-in-it-for-me2013-01-18T11:00:00.000Z2013-01-18T11:00:00.000ZHoward Rosehttps://community.dpgplc.co.uk/members/HowardRose<div><h1><span class="font-size-2"><a target="_self" href="http://storage.ning.com/topology/rest/1.0/file/get/1357125?profile=original"><img class="align-left" src="http://storage.ning.com/topology/rest/1.0/file/get/1357125?profile=RESIZE_320x320" width="200"></a>It's a cold and wet Monday morning and I'm on my way to meet <a rel="nofollow" title="Dr. Jim Kirkpatrick" href="http://www.linkedin.com/in/kirkpatrickfourlevelevaluation" target="_blank">Dr. Jim Kirkpatrick</a> at a hotel near Heathrow. As I climbed in to my car my thoughts reflected back to a conversation I had with Jim during the recent Kirkpatrick Partners Bronze level programme that I attended..</span></h1>
<p class="float">I was thinking about how, when I worked for a worldwide organisation back in the 80s they used to send us to the 'training school' every three or four months to attend a course of some sort. "So what's the problem" I hear you say, "what a great employer", "they looked after you", "developed you", "spent money on you" etc, and yes of course I would agree with almost all of those points except what a great employer! Why? Well they were wasting their money in the worst way possible, in the belief that I would become a better employee with all these new skills. I learnt to solder well enough to assemble a satellite so it would not fall apart on take-off! I learnt how to compile a business report and a whole host of new skills that I did not need!! </p>
<p class="float"><a target="_self" href="http://storage.ning.com/topology/rest/1.0/file/get/1357137?profile=original"><img class="align-left" src="http://storage.ning.com/topology/rest/1.0/file/get/1357137?profile=original" width="275"></a>The training school were fantastic, the courses received great reactions from the trainees and we always passed the end of course assessments, but somehow things were not changing back in the business units, so often we were sent on other courses or refreshers!! Unfortunately some years ago this organisation closed its training school as they just could not see the value in it and weren't getting results. Was sending us to a training school away from the job really the right approach when most learning is done on the job anyway, is it time for new thinking and new approaches?</p>
<p>As I arrived at the hotel my thoughts turned to the day ahead. I was here to learn more about how Jim delivers his two day Kirkpatrick Partners Bronze programme, as I have the great honor of co-delivering with him in April 2013. We spent the day looking at why our profession, learning and development or as Jim prefers learning and performance, have come to rely on and in many cases mis-use the original Kirkpatrick evaluation model. And how the 'New World' kirkpatrick approach puts things straight.</p>
<p>For many learning professionals the Kirkpatrick four levels is a tool that is used towards the end of training to demonstrate the value of the event or programme, some never even get past level two. In a survey carried out by the ASTD "<a rel="nofollow" href="http://www.astd.org/Publications/Research-Reports/2009/2009-Value-of-Evaluation-Making" target="_blank">The value of evaluation: Making Training evaluation more effective (2009)</a> their research suggests that 91% of respondents use level 1 and 80% go as far as level 2, for levels 3 and 4 it drops significantly to 54% and 37% respectively. This is in line with Jim's comments that the training profession can often see evaluation as a way of patting its self on the back or in other words "Look we did a great job, the delegates loved us and answered all our questions correctly, the organisation is going to be so happy with us, don't you think"? And there-in lies the problem, there can be a big disconnect between the training department and the rest of the business! Does the rest of the business really have much more than a passing interest in how well we deliver the training or how good lunch was or even whether our delegates got 100% in a test? No, the rest of the business wants results, and improved performance, they want things to change as a result of the training. Today, stakeholders are asking "So What?" and more importantly, "What's in it for me?"</p>
<p><a target="_self" href="http://storage.ning.com/topology/rest/1.0/file/get/1357159?profile=original"><img class="align-center" src="http://storage.ning.com/topology/rest/1.0/file/get/1357159?profile=original" width="480"></a>This is where the "New World" Kirkpatrick model comes into its own, no longer should it be used towards the end of the training it should be used as a tool to help shape business thinking, to design learning solutions that matter most and along the way provide opportunities to generate evaluation data. The 'New World' approach is much more than a bolt on after training, it is a fresh up front approach to robust consultancy and training design. For mission critical programmes it allows the learning professional, people like you and me to cross the metaphorical bridge to the business. Starting the conversations off with questions that echo Dr. Steven Covey's "Start with the end in mind" such as "what are the business results you are looking for"? and "what behavioural change will give you this"? These are the starting points for business focused conversations. Conversations that will help you and your organisation answer their what's in it for me question and provide data to demonstrate impact and results.</p>
<p>So the starting point is loud and clear from Jim it is Level Four, what results do you want to see, then we can work back through the levels identifying the behaviours and importantly what the business needs to do to support and encourage them, finally we can think about levels one and two.</p>
<p>On the way home I returned to thinking about that organisation I worked for and the training school, if only they had said to the business "so what results do you want to see from training" and reacted by helping the business define clear goals that could then be used to define some smaller measures of success and the behavours needed to get there, they may even be there today. Some questions to leave you with:</p>
<p><strong><em>How are you currently looking at evaluation in your business?</em></strong></p>
<p><em><strong>What conversations do you have with your stakeholders to identify what the desired change and end result is going to be?</strong></em></p>
<p><em><strong>When your stakeholder's ask you "what's in it for me? How do you respond?</strong></em></p>
<p><em>If you are looking to get a fresh perspective on evaluation and take your approach to the next level, join Howard Rose, Phil Wilcox and Dr Jim Kirkpatrick at the next UK Kirkpatrick Certification Programme in London on April 22nd/23rd 2013. For more information on this exclusive event then please email <strong><a rel="nofollow" href="mailto:service@dpgplc.co.uk">service@dpgplc.co.uk</a></strong>or alternatively all the details can be found here</em></p>
<p><em><strong><a rel="nofollow" href="http://kirkpatrick2013.eventbrite.co.uk/" target="_blank">Kirkpatrick Four Levels Programme London 22nd/23rd April</a></strong></em></p></div>