data - Blogs - DPG Community2024-03-29T09:18:36Zhttps://community.dpgplc.co.uk/blog/feed/tag/dataThe role of data and analytics in evaluationhttps://community.dpgplc.co.uk/blog/the-role-of-data-and-analytics-in-evaluation2019-06-20T10:44:39.000Z2019-06-20T10:44:39.000ZAdy Howeshttps://community.dpgplc.co.uk/members/AdyHowes<div><img src="https://storage.ning.com/topology/rest/1.0/file/get/3036011792?profile=RESIZE_400x&width=400"></div><div><p><strong>What is the sort of data we should be gathering and what questions should we ask?</strong></p>
<p>Last week, I saw a session at <em>The Festival of Work</em> about <strong>Learning Data and Analytics</strong> at capacity.</p>
<p>No more space, not one seat, packed, full, people turned away (I myself had to watch from the side lines from the standing room only section).</p>
<p>Data and analytics, it seems is a hot topic right now and as ever you can find this topic explored already in some detail here on the DPG Community. There are articles <a href="https://community.dpgplc.co.uk/blog/i-m-not-a-data-scientist-but">here</a> and <a href="https://community.dpgplc.co.uk/blog/are-l-d-slow-to-change">here</a> that are well worth reading that link to data in L&D.</p>
<p>So why now?</p>
<p>I think the focus on data has always been there but one of the issues is accessibility and the type of data & information we as an industry have focussed on. I also think data is being looked upon to understand relationships and the way in which we draw conclusions and make links to how and why things are as they are.</p>
<p>There is a seismic shift across all industries to use the abundance of data & insight now available. Using this data to help and better understand customer behaviour, customer lifecycles, problems we’re looking to solve, product / service satisfaction, more effective product development, credibility, buying habits and trends and positive and developmental feedback (reviews). We live in a digital world and this has changed our relationship with data, but the principle of data analysis and product development can apply to all aspects of L&D. For me data helps us ask fundamentally better questions or helps us know the right questions to ask.</p>
<p>If L&D were a start-up business, developing a suite of products, this is also exactly the type of data we should be looking to gather or questions to ask in terms of:</p>
<ul><li>Clearly identifying the problem / challenge / pain faced by potential customer</li>
<li>Credibility – consultative conversations with customer to ensure deep understanding and right approach is used with valid recommendations provided that meet that need.</li>
<li>Customer lifecycles – where and at what point is support / training / solutions required to address any problems</li>
<li>Customer behaviour – how do customers interact with a product to address problem, where do they access it or buy it and crucially how does the product change / modify behaviour in a positive way and ‘solve’ problem.</li>
<li>Product satisfaction (feedback on learning solutions / performance support tools)</li>
<li>Product development / continuous improvement based on feedback</li>
</ul><p>Maybe one of the previous challenges for L&D is the ‘product’ has been predominantly a face to face classroom session or a face to face workshop and we just haven’t thought about what we do and how we do it in these terms before. The questions and approach above can be applied to any L&D product but I just don’t think we think as commercially as we should when it comes to demonstrating value and measuring success. As a cost centre L&D need to be able to demonstrate the value they provide, and more is being demanded of us than ever before.</p>
<p>To respond to this demand, more and more L&D teams are working using agile principles and methodology. Rapidly prototyping and developing minimal viable products quickly and far more collaboratively with end users to ensure whatever product is developed meets the need of the end user. Pilots, experiments and beta products are all things you will find in other industries, but we don’t tend to embrace these approaches as much as we perhaps should. One of the things I’ve been thinking about is how we link data & insight to asking and answering the questions above to traditional L&D evaluation.</p>
<p>Evaluation in L&D needs to evolve. I’m not suggesting that the likes of the <a href="https://www.dpgplc.co.uk/courses-qualifications/learning-and-development/kirkpatrick-four-levels-evaluation-certification-programme/">Kirkpatrick Evaluation Model</a> (Link) are obsolete. As a <a href="https://www.dpgplc.co.uk/courses-qualifications/learning-and-development/kirkpatrick-four-levels-evaluation-certification-programme/">Kirkpatrick</a> Partner we are big believers in the KP model. The framework that the KP model sets out helps still provides a structure to follow that can be adapted to each of the elements listed above. It’s a move from an old training mindset to that of a data driven and evidence-based department who focus less on individual training events to that of overall engagement and customer or employee lifecycles and product / performance improvement.</p>
<p>Rather than taking the 4 levels as a linear model and moving through them 1 by 1, analysing the key elements of each and applying them in a way that meets the need of the customer. For example:</p>
<p><strong>Results (level 4)</strong></p>
<p>Start with the end in mind, analyse challenges and pain points – what does success look like and how will it be measured. Consultative approach working collaboratively with customer.</p>
<p><strong>Learning (level 2)</strong></p>
<p>Make a recommendation and begin prototyping with the customer to build an MVP. Get it in front of people to test and use. Pilot, pilot, pilot.</p>
<p><strong>Reaction (Level 1)</strong></p>
<p>Get effective user feedback, is it effective, easy to use, easy to understand, what works what doesn’t, how can it be improved, collaborate and demand feedback.</p>
<p><strong>Behaviour/Transfer (Level 3)</strong></p>
<p>Does it have the desired impact, does it change behaviour, provide new skills, can this be observed, how does this link to the desired success and measures identified.</p>
<p>If yes – release and get back to & track ‘Results’</p>
<p>If no – back to Learning and better product development</p>
<p>Repeat.</p>
<p>OK, so I appreciate this is a crude example but the point I’m making is that we have always looked at evaluation as something that we do to other people or something we apply to the end of the training process as opposed to something that we apply to our process and continually analyse the review the feedback and data we have.</p>
<p>To do this in a much more agile way and to work much closer with our customers to provide better products and service. Working with pilot groups and working in sprints can speed up the development process and make things so much quicker. Today, we don’t need to wait 3-6 months to analyse a change in behaviour and we also don’t have to rely on outdated modes of gathering feedback. Apps, pulse surveys and polls can get us closer to the data we need much quicker if we’re prepared to experiment and explore.</p>
<p>In this age of data, we need to look at our own processes and approaches to stay relevant and keep up with the demands of our customers. Not a complete overhaul of everything we do but looking at what we do and how we do it through other lenses and borrowing ideas and concepts from other industries to make us more resilient and responsive.</p>
<p>Perhaps learning analytics while the latest buzz word is just another way for L&D to evolve its’ practices and that’s why there is such a focus on it. For me though it isn’t about disregarding everything that we’ve done before but using that we can see and experience to make better and more informed decisions. Yes, there is an element of shiny new technology, analytics dashboards and ‘evidence based’ approaches but this needs to blend together with ‘evaluation’ of sorts and we can learn a lot from new forms of data but also our existing data and use it in a more considered and effective way.</p>
<p><em>What do you think?</em></p>
<p><em>How are you using data & evidence-based practice to improve your product development?</em></p>
<p><em>Is the way you do evaluation changing?</em></p>
<p>If you are interested in finding more about how to use a data and evidence based practice to your Learning and Development activities, check out our <a href="https://www.dpgplc.co.uk/courses-qualifications/learning-and-development/kirkpatrick-four-levels-evaluation-certification-programme/">Kirkpatrick Four Levels Certificate</a></p>
<p> </p>
<p> </p></div>I'm not a Data Scientist...buthttps://community.dpgplc.co.uk/blog/i-m-not-a-data-scientist-but2018-04-12T15:41:14.000Z2018-04-12T15:41:14.000ZMike Collinshttps://community.dpgplc.co.uk/members/MikeCollins<div><img src="https://storage.ning.com/topology/rest/1.0/file/get/14413890?profile=RESIZE_400x&width=400"></div><div><p>Not blogged for ages and it's nice to be able to contribute something in this community. I hope it's of interest as I'm sharing new stuff I'm working on. I'm interested in data, and you should be to. According to HBR back in 2012, a <a rel="nofollow" href="https://hbr.org/2012/10/data-scientist-the-sexiest-job-of-the-21st-century">Data Scientist was going to be the sexiest role of the 21<sup>st</sup> century</a>!</p><p><strong>What’s a Data Scientist?</strong></p><p>If you ask Google, you get a simple dictionary definition which is as follows:</p><blockquote><p><em>“</em><em>a person employed to analyse and interpret complex digital data, such as the usage statistics of a website, especially in order to assist a business in its decision-making.”</em></p></blockquote><p>Data is everywhere (and in some places, it shouldn’t be), something that <a rel="nofollow" href="https://www.theguardian.com/commentisfree/2018/apr/11/mark-zuckerbergs-facebook-hearing-sham">Mark Zuckerberg</a> knows only too well right now. With GDPR around the corner, data is in the spot light and a rather hot topic. There are of course different types of data and the <a rel="nofollow" href="https://ico.org.uk/for-organisations/guide-to-the-general-data-protection-regulation-gdpr/">GDPR legislation</a> will serve to protect our privacy and ensure organisations are treating our data with respect and integrity. Those who don’t will be fined heavily. GDPR is a hugely positive thing, if not the latest ‘snake-oil’ product under the banner of ‘<a rel="nofollow" href="https://companyconnecting.com/news/gdpr-expert-%E2%80%93-new-snake-oil-sales-man">let us make you GDPR compliant</a>’.</p><p>Data for L&D and HR is nothing new. Whether you are collecting data from happy sheets to knowledge checks to engagement surveys to exit interviews to trying to demonstrate the unicorn that is Return on Investment and everything else in-between. Data is important, or rather data in this context is important.</p><p>Data that can:</p><ul><li>Help us understand how people are interacting with something (system/content/survey)</li><li>Help us measure a specific metric that could be linked to a L&D initiative e.g. increase in sales conversion, decrease in complaints or completion of a compliance module etc</li><li>Help us understand and interpret what’s happening or the general feeling in our organisations e.g. pulse surveys</li><li>Help us understand qualitative and quantative feedback to spot trends and patterns</li><li>Help us make better and more informed decisions based on the evidence all the above data provides</li></ul><p>Evidence based L&D and HR is something I’ve heard a lot about in recent months. I had the pleasure of listening to <a rel="nofollow" href="https://twitter.com/Rob_Briner">Rob Briner</a> speak on the topic earlier this year. It got me thinking a lot about data and where do I go to get the data that helps me in my digitally focused role.</p><p>I have been fascinated by the <a rel="nofollow" href="https://donaldclarkplanb.blogspot.co.uk/2016/03/the-lms-is-dead-long-live-lms-10-pros.html">LMS debate</a> for years; “they are useless”, “it’s not where learning happens”, “they don’t support work-learning-work”, “the LMS is dead”. These are statements you may have heard from various sources over the years, the link points to a post from Donald Clark that’s worth a read. Attend the <a rel="nofollow" href="http://www.learningtechnologies.co.uk/">Learning Technologies Conference</a> and you will see the LMS is not dead and in fact very much alive and kicking and generating (according to <a rel="nofollow" href="https://globenewswire.com/news-release/2017/01/17/906276/0/en/Global-Learning-Management-System-Market-will-reach-USD-19-05-Billion-by-2022-Zion-Market-Research.html">Statista</a> – $5.2 billion 2016-2017, growing to $16 billion in the next 4 years). As another Donald might say, “those are really big numbers, the biggest numbers in fact”.</p><p>It’s true they are big numbers and the LMS market has grown massively with older legacy systems re-writing their code to be mobile responsive, to LMS & social plug-ins on larger HR systems, open source systems being continually improved by the developer community, new content management systems or digital portals which are xAPI complaint and tonnes of features being driven by ‘consumerisation’, anyone not heard of ‘Netflix for learning’ or ‘let’s gamify learning’?</p><p>It is a fascinating space. I’m learning about the role of artificial intelligence and chatbots and their numerous applications, understanding how machine learning has come on leaps and bounds since Hal in Space Odyssey (1968) and War Games (1983) and it feels like we’re heading down algorithm avenue at frightening speed.</p><p>Whatever portal, system and digital channels you are using and whether you are on the LMS bus or not; let’s park the LMS doesn’t support learning debate for a moment. Instead focus on data and how we can get our hands on the data that systems can provide. Now more than ever we should be more demanding of what analytics (trendy word for data apparently) are available from the systems we use.</p><p>Our colleagues in marketing have been doing this for years. Organisations sell products and services from websites and have teams who are employed to market and promote these products and services; often with decent budgets. Some pay for Google Ads, some rely on SEO and others from referrals or generally a mix of all three. The purpose of getting customers to a website is to get them to convert. In other words, to see something, to interact with something, to build a connection, to develop an understanding and ultimately buy something. Marketing teams use metrics to measure their campaigns, from open rates on emails, to click through rates, they split test their messages, they track times of day and devices used, they analyse everything they can to improve the customer journey / experience and make it as EASY as possible for a customer to purchase and get access to what they offer.</p><p>Let’s pretend that <a rel="nofollow" href="https://www.learningsolutionsmag.com/articles/2410/five-essential-marketing-skills-ld-needs">L&D are in marketing</a> for a second. The website / portal we use is our LMS but instead of having to generate leads we have a captured audience. The employees within our organisation! Wow the hard work is done, we already have a database of customers who could potentially buy our products – whatever they may be. Putting products (content) to one side for a minute and how we are marketing that content, what are the sorts of things you’d want to know about your website and how your customers are using it?</p><ul><li>What time of day is it being used the most?</li><li>What day is most popular?</li><li>What browsers are people using?</li><li>What device and operating system are people using?</li><li>What pages / content is getting the most views?</li><li>How many people are new vs returning?</li><li>What do your user journey’s look like?</li><li>Where are drop offs happening?</li><li>Where are people coming from – direct traffic, referral etc</li><li>Can we measure a campaign or promotion and see an increase in traffic on the system?</li></ul><p>By asking these questions you are seeking to better understand how people are interacting with your system and the data that this provides helps us make better and more informed decisions about what we do and how we do it. This in turn should lead to improvements in what we do and how we do it.</p><p>The good news is this data should be available from your LMS provider and if it’s not it can be gathered from Google Analytics (GA) – a free to use tool that can be easily added to any system that allows HTML code to be added. I’ve been using GA for a few years and I’m no expert. I’m self-taught and have learned from others who know what they are doing, but I see tremendous value in learning more about this great tool and having this data and insight. We used GA to help build and develop this social community over 5 years, I've just added GA to the Moodle LMS I’m now working with, and also experimenting with the <a rel="nofollow" href="https://learninglocker.net/">Learning Locker</a> from HT2 Labs to <a rel="nofollow" href="https://www.ht2labs.com/resources/the-learning-technology-managers-guide-to-the-xapi/">capture xAPI statements</a>. All the data is there, we just need to be curious and be asking the right questions and seek to understand what the data is telling us to help us improve what we do based on the evidence, not the hearsay or subjective opinion. It’s only <strong>one part of the puzzle</strong> but it’s an important part of the puzzle.</p><p>I read recently that <a rel="nofollow" href="https://elearningindustry.com/6-critical-roles-learning-and-development-needs-to-succeed-2017">every L&D team needs a data scientist or a data analyst</a>. I think it’s true but I think EVERYONE in L&D needs to be a bit more scientific and get better at understanding what tools are available and what data we have access to, asking better questions, interpreting that data to gain better insights and then making better decisions because of what we know and improving iteratively as we go.</p><p>I am not a data scientist…. but I see the value in better understanding what data we have and what we can do with it. Data is not for nerds, geeks (with or without pompoms) and the socially awkward. In an increasingly connected, digital and system driven world it’s one of the key tools L&D teams have to being as effective as they can be.</p><p>I’m going to keep sharing this data journey I’m on, I’m keen to understand what you’re doing around data and analytics, specifically around the systems in your organisation.</p><p>Image Source: <a href="http://negativespace.co/photos/seo-stock-photo/" target="_blank" rel="nofollow noopener">negativespace.co</a></p></div>10 areas HR professionals should focus on in 2016https://community.dpgplc.co.uk/blog/10-areas-hr-professionals-should-focus-on-in-20162016-01-14T14:48:42.000Z2016-01-14T14:48:42.000ZMike Collinshttps://community.dpgplc.co.uk/members/MikeCollins<div><img src="https://storage.ning.com/topology/rest/1.0/file/get/2216729?profile=RESIZE_400x&width=400"></div><div><p>The close of one year and beginning of another is always a good time to reflect on what has been and what is to come. HR professionals need to use this opportunity to prepare themselves for 2016.</p>
<p>It almost goes without saying that 2016 is going to be a year of further change for the profession, with new disruptions, innovations and ways of working. A lot of HR teams are still playing catch up with all the change that took place in 2015, so there is no scope for complacency.</p>
<p>To help you, we have come up with a list of the top 10 areas that we think HR professionals need to be focusing on in 2016:</p>
<p><strong>1.</strong> Technology. Technology is behind a lot of the change that has been and will be occurring. HR needs to embrace the possibilities of technology, but without being swept along with the latest fad or piece of kit. Look at the technology you have already got – is it doing what it needs to do? Is it user friendly? Does it enable your business to be agile? Huge amounts of change are predicted in this space for HR in 2016. According to research by <a href="http://www.bersin.com/Blog/post/HR-Technology-For-2016-Ten-Disruptions-On-The-Horizon.aspx" target="_blank">Bersin</a>, more than 40% of all companies are replacing or planning to replace their core HRMS systems. LMS markets are also being hugely disrupted by new video-based learning solutions. Plus there’s video, mobile, etc to think about.</p>
<p><strong>2.</strong> Be employee-centric. Think of employees as consumers. With regards to L&D, for example, how do they learn currently? How do they want to learn? How can you best deliver the type of learning they want, when they want it, on the device they want to use? Employees have become self-directed learners so if L&D doesn’t want to share that journey, they will be left behind.</p>
<p><strong>3.</strong> Business alignment. This is something HR and business leaders have been talking about for a long time, but still needs improving. HR has to be aligned to the business. It needs to really understand business drivers and objectives and its role in achieving them. Talk the language of business. The benchmarking organisation <a href="http://www.towardsmaturity.org/article/2014/10/30/modernising-learning-forward-dr-chai-Patel-HC-One/" target="_blank">Towards Maturity</a> has talked extensively about the need for HR to focus on business outputs, rather than learning inputs.</p>
<p><strong>4.</strong> Data. It’s all about data these days, whether you work in HR, finance or marketing. Want to implement a new training module – where’s the data to support why it is needed and what the results will be. This ties in with talking the language of business and again, this is something that<a href="http://www.towardsmaturity.org/article/2015/05/12/data-optimising-ld-performance/" target="_blank">Towards Maturity</a> has made a big play of.</p>
<p><strong>5.</strong> Skills. What skills does your organisation really need now and in the future? Do you have them? How can you get them, internally or externally? HR needs to keep one eye constantly on the horizon.</p>
<p><strong>6.</strong> Learning culture. Organisations need to really foster and encourage a learning culture, enabling employees to keep learning and learn as part of the workflow. Facilitate collaborative learning and sharing, be it through peer to peer networks, community portals, special projects….</p>
<p><strong>7.</strong> Talent. Do you have the talent you need? Are you retaining your top talent? How can you retain your top talent? With the jobs market continuing its recovery and an ageing workforce who will start retiring, it’s rapidly becoming a candidate’s market.</p>
<p><strong>8.</strong> Mental health and wellbeing. There is so much research that shows that workplace stress levels are high, working hours are long and a culture of <a href="https://www.dpgplc.co.uk/2015/10/presenteeism/" target="_blank">presenteeism</a> exists. The mental health and wellbeing of the workforce is a really important business issue, as Professor Cary Cooper, the new president at the CIPD, made clear at his opening speech at the recent <a href="https://www.dpgplc.co.uk/2015/11/well-being-in-the-workplace/" target="_blank">CIPD conference</a>.</p>
<p><strong>9.</strong> Working patterns. As we know, the workplace is no longer a 9-5, Monday-Friday, office based set up. Increasing numbers of people want to work flexibly, whether it’s flexible hours, flexible location or flexible roles. There’s also increasing numbers of freelance workers. Organisations need to accept and facilitate these different working patterns if they are to hire and retain the best talent. Flexible working arrangements can be highly beneficial to employers as well, when considered properly.</p>
<p><strong>10.</strong> Your own self development. It’s very easy for HR professionals to spend all their time on the career development of others and neglect their own career development. Needless to say, your career and skills development is just as important, particularly as the profession is going through so much change and you have to change with it.</p>
</div>17 Visualisation Tools to Make Your Data Beautifulhttps://community.dpgplc.co.uk/blog/17-visualisation-tools-to-make-your-data-beautiful2015-04-20T14:22:23.000Z2015-04-20T14:22:23.000ZMike Collinshttps://community.dpgplc.co.uk/members/MikeCollins<div><img src="https://storage.ning.com/topology/rest/1.0/file/get/2216469?profile=RESIZE_400x&width=400"></div><div><p><span>Data is key for Learning and HR professionals. Interpreting data to identify trends and patterns providing important business intelligence to make key decisions is a key requirement for any modern professional. The way in which we then display data also needs to be visual, interesting and relevant. It also needs to be easy to understand so when we came across these 17 tools to help bring your data to life and to make them beautiful we had to share it. </span></p>
<p><span>**Curated from the <a href="https://econsultancy.com" target="_blank">Econsultancy</a> website** </span></p>
<p><em>You don’t need to be a seasoned data scientist or have a degree in graphic design in order to create incredible data visualisations.</em></p>
<p><em>It has become a lot simpler to mine your data and interpret your insights in an engaging, attractive, and most importantly easy to understand way.</em></p>
<p><em>This is a list of various different free and premium tools and platforms available which will allow you to communicate data in a variety of different formats, from interactive charts, to infographics, to maps, to word clouds.</em></p>
<p><em>Not all will be suitable for your particular data set or your audience, but its certainly worth experimenting with a few to seek out the most relevant. </em></p>
<p><em>Hopefully you’ll not only learn some new skills and find some new creative tools, but also learn something about your own data.</em></p>
<p>We'd love to hear if you use any of these tools to bring your data to life and make it beautiful.....even in any assignment you might do.</p>
<p>Get going data lovers...</p>
<p><a href="https://econsultancy.com/blog/66131-17-visualisation-tools-to-make-your-data-beautiful/" target="_blank">17 Visualisation Tools to Make Your Data Beautiful</a></p>
<p></p>
<p></p>
</div>