With so much evolution and advancement in technology today, we now have more ways than ever for capturing data associated with the delivery, consumption, and impact of learning in organizations. Yet, according to Brandon Hall, fewer than 16% of organizations are able to effectively identify and track a series of metrics, including participation, satisfaction, knowledge transfer, behavior change, and business impact for any of their learning. One reason for this shortfall is that the idea of identifying, collecting, and analyzing all of these potential data points can be overwhelming to even the most mature organizations. One potential solution is to focus those efforts on a subset of data, providing insights that align with the most important outcomes. This creates a “measurement brand” for your organization. Continue reading to learn the steps to creating your learning measurement brand.
The Good News: Data is Everywhere
The ability to capture data has always been at the core of the measurement challenge. Thankfully, over the past decade this issue has been addressed with new tools and technologies that improve access to data. These include:
- Learning Management Systems (LMS)
- Learner Experience Platforms (LXP)
- Experience APIs (xAPI)
- Google Analytics
- Learning Content Management Systems (LCMS)
- Feedback surveys
- Enterprise BI Tools
- Location and Device tracking
…and other similar tools and processes. Combined with a comprehensive learning analytics strategy, the data from these tools can be used to measure training consumption, assessment performance, social behaviors, confidence and more.
Is it possible to have too much of a good thing?
Ideally, each organization would capture every bit of data available, analyze for trends, create beautiful graphs and charts to present the findings, and provide regular updates to all the various stakeholders. Unfortunately, for most organizations, it’s not possible to do it all. Different stakeholders will have differing priorities for this data. The Compliance Team may care most about consumption data like starts, completions and time to completion; The Sales Team may care most about starting and ending confidence; The Quality Team may care most about learner’s ability to recall and apply knowledge in a course. Even if you had unlimited resources, providing every data point to every department may leave them overwhelmed. Prioritizing your measurement goals, as part of your overall learning measurement strategy, and taking a point of view on what’s most important to your organization will guide where you put your energy and resources when capturing learner data. This approach will ensure that you successfully deliver on your measurement strategy.
Learning Brand Positioning
So, what’s your measurement brand? As part of a broader learning measurement strategy, here are four potential brand positions to consider:
- Learner-Centered: Meeting the needs of the learner is your driving force. Do you develop confidence? Did they enjoy the experience? Were they able to apply the material?
- Business Outcomes: Delivering an impact on the business. Do those who go through a learning solution perform better?
- Behavioral: Do our learning solutions change behavior? Were we able to get learners to start, stop or continue activities? Can we assign value to those changes?
- Consumption: Are people engaging with the learning? How long do they spend? Are they completing it? Do we hit penetration levels, e.g. 80% of potential audience?
Once you establish the learning brand position for stakeholders, you’ll be able to identify appropriate approaches to capture and measure learner data from your training programs. How does this “data” get there?
It Starts Long Before Data Collection
So let’s say you work with each of your key stakeholders and nail down their priorities. Is that enough? Probably not. The development of a learning measurement strategy involves more than just creating a list of organizational learning priorities. It’s more elaborate because capturing learner data, and subsequently your measurement strategy, will affect not just the data you collect, but also the process of developing learning – from inception to delivery. Here are some examples:
- If you want to track how learners apply knowledge, you might use xAPI statements to collect the number of correct and incorrect responses to questions that require application of knowledge.
- If you want to measure how a learner’s confidence changes from the beginning of learning experience to the end, you need to include questions about confidence at the beginning and end of each course.
- If you want to focus on consumption statistics like # of learners who started a course, finished a course and time to completion, you’ll need to pull data from your LMS.
- If you want to know if training changed behavior back on the job, you’ll want to include a manager or 180 assessment before and after training.
As these examples show, your learning measurement strategy drives your overall approach to learning design and development.
A good set of principles to remember come from Douglas Hubbard, the creator of Applied Information Economics, offering us some insight into collecting and measuring data:
- Rule 1: If it matters, it can be measured
- Rule 2: You have more data than you think
- Rule 3: You need less data than you think
The key to building an effective measurement strategy, therefore, is to set realistic expectations, build relevant data points, and capture and measure specific data from your learning environment. Then, you can measure and re-assess your learning brand and, if required, make informed decisions on repositioning that brand.
Time to Own Your Brand
Designing and implementing a learning measurement strategy can feel overwhelming and unattainable, but it doesn’t have to be. Three keys to keep in mind are:
- Focus your efforts based on the needs of stakeholders
- Plan your data capture strategy early in the project (before you build it!)
- Be consistent - once you’ve decided your approach or approaches, implement it on every project. There’s value in seeing trends over time.
To learn more about all the different data you might consider collecting, and how to measure training data to assess the effectiveness of your learning brand, download our Measurement Guide now.
Still feeling left in the dark about how you can get the most out of your data? We can help. Replace inflexibility with Kineo’s versatile open LMS, built with totara inside.
Our Suite of Totara solutions are powered by our world class support and customised services, giving your learning strategy the flexibility it needs.
Reports and Guides
Drowning in data? Seven approaches to data to help you measure success
If you’ve been in L&D long enough, you’ve probably been confronted with questions about the impact of learning solutions. Your internal voice may say ‘yes, of course we make a difference’ but you probably also have a sense of unease thinking about your ability to prove it. You’re not alone. In...READ MORE