Cammy Bean
Senior Solutions Consultant at Kineo US
Learning data and analytics is a hot topic these days in the L&D field. ‘Big Data’ has arrived and now learning leaders – and learning designers – are starting to dive into an area that may have traditionally felt beyond our scope. We’ve perhaps been doing a good job writing great assessments and getting survey data about learners’ reactions, but now what?
Ultimately, learning leaders and senior business stakeholders don’t care so much about what Josie scored on that compliance quiz, but rather the overall picture of what’s happening in the organisation and whether the investment in learning programmes has been worth it. They want to know if they’re going to get a return on investment - the ROI of learning - and whether they have spent the right budget and energy in the right places. And they want to know how to improve their offerings going forward to ensure they’re really moving the needle on talent development and performance improvement.
The ROI of learning
Traditionally, figuring out the ROI of learning has been a tricky thing. How can we connect the learning outcomes with the things that really matter - the business outcomes? In this day of big data and analytics, the conversation is changing and teams now have more tools and opportunities for deeper insights than ever before.
This impacts learning leaders and instructional designers because we now have to factor in what data needs to be captured with a more eyes-wide-open point of view than ever before. And data analytics is a whole new skillset that most of us have no experience with.
L&D are getting there, but have a way to go
This past month, Preston Gales (Kineo’s Global Head of Learning Design & Product Innovation) and I led a panel discussion with senior leaders from a wide-variety of organisations on data and analytics as part of the OnCon Learning Leaders programme. Participants shared where they were with data analytics. We asked this question with the results following a bell curve – meaning most participants feel like they have a way to go.
How are you doing with data and analytics?
- We’ve not even gotten started.
- We’re collecting traditional LMS data but that’s about it.
- We’re collecting data but definitely have a way to go.
- We’re feeling pretty good about data, but still some gaps.
- We’ve got this. We’re collecting and analysing data and can action on it.
Where does the Data function sit within an organisation?
We talked with participants about who does the data and analytics at their organisation, and this painted a portrait of an emerging maturity model for learning teams. The path to a robust learning data and analytics programme seems to follow a first pass where the learning team upskills and starts learning the basics and trying to “do” data and analytics. “The Accidental Data Scientist”, you might say.
Larger, more mature organisations seem to evolve away from that approach and instead rely on a centralised data and analytics team that partners with the learning side of the business. At these organisations, data from all over the place is getting put into a data warehouse - from the LMS, LRS, performance management, HR, sales results, etc. From there, professional data scientists can look at the big picture and identify the trends and factors that really matter – e.g., those high-performers who beat their numbers every year are the ones who read more articles online and participate in forum discussions. Or Leaders who went through XYZ Leadership Development programme stick around longer and have higher retention.
So where do you start?
Preston shared his experience from past work-lives putting data and analytics programmes in place at large organisations. He laid out a useful framework that you can consider when maturing the learning data and analytics programme in place at your organisation:
- What do we need and want to measure? This will help in learning design, to ensure you’re capturing the right moments. One of our session participants mentioned using Chief Learning Officer Magazine’s Business Impact Measures – looking at employee retention, sales, revenue growth, customer satisfaction, cost reduction, and other factors.
- Do you have senior sponsorship? Chances are this process will take some time and you’ll need to involve individuals from throughout the business. It helps to have the support of a senior leader as you build a culture of capturing data and making changes based on that data.
- How are you going to collect that data? Are you storing data in an LMS, LRS, someplace else? Where will you get the business data from?
- How will you interpret the data? This is where a data science background will help - having someone onboard who can get insights out of the data, identify patterns and trends, and connect the dots.
- How will you visualise the data? Sharing the story with key stakeholders is made easier when you have business intelligence tools that can help you present the data in more meaningful ways, pulling out that narrative.
- What will you do with what you learn? Do you have a decision-making framework to determine what actions to take based on what the data tells you?
Building a robust and cross-functional data and analytics approach will take time to build. Start with small experiments to see if you can gain more actionable insights. Pull in other parts of the business. And don’t feel like this is yet another hat that you need to wear - get help!
Download our guide to Building a Data-Driven Learning Organisation for more.
Cammy Bean
Senior Solutions Consultant at Kineo US
Cammy has been collaborating with organisations to design online learning programs since 1996. An active speaker and blogger, Cammy gets fired up about instructional design, avoiding the trap of clicky-clicky bling-bling, and ways to use technology to create real behavior change.