When you invest a lot of time and effort into developing an online training program, you want to know that it’s effective.
How do you know if the knowledge and skills learned throughout the course are improving day-to-day job performance?
While certain metrics of employee engagement and retention can be measured objectively, others can be harder to define.
That’s why it’s important for employers or managers to know how to measure eLearning engagement and when things aren’t working as they should.
Here are a few basic things to know about measuring the success of your eLearning training.
Using the Kirkpatrick Model of Engagement
The Kirkpatrick Model of training evaluation is one of the more popular methods of gauging the effectiveness of a training initiative.
It’s comprised of four basic levels, using both subjective and objective reporting:
- Reaction – How did learners perceive the training? Did they find it useful and relevant?
- Learning – Did the learner gain specific knowledge related to their job? Did learning improve over time?
- Behavior – Did employee behaviors change (positively) after training? Were there noticeable productivity improvements?
- Results – Did the training deliver on its initial objectives and goals? Does management believe it was worth the investment?
Because measuring engagement can often be subjective – maybe the employee felt like they learned more, but managers found no noticeable difference in performance, for example – managers must have a plan for measuring objectively.
Using LMS Reports to Gauge Objective Results
LMS reports should help managers identify which learning initiatives are having a real impact.
When measuring engagement objectively, it’s essential to consider:
- Completion Rate – How many employees logged into the Learning Management System (LMS), how often did they log in, and how many completed the training?
- Self-Learning – Did learners participate in self-learning and how many completed self-learning courses?
- Questions – How many employees asked questions, either of their peers (in an online forum or equivalent, or of their managers or course instructors?
- Responsiveness – Did employees create content to help their colleagues (answered questions via messages, etc.)?
- Response Rate – How long does it take employees to complete a learning course?
- Weekly Logins – Are specific employees or departments logging in more often than others?
It’s important to look for measurable signs of participation.
If someone reported completing a training course that required peer-to-peer or social participation, but they never posted a question or created any content for their peers, it can be harder to know whether or not they learned what they were supposed to learn.
If most employees finished an eLearning course quickly, it might indicate that the learners are receiving the right notifications and that they’re engaged enough to move through.
You also want to take note of any Key Performance Indicators (KPIs) throughout the organization. If completion rates are high but KPIs are unchanged, or even lower than average, then it might indicate that the eLearning course is too easy and not producing the right outcomes.
Using Employee Surveys to Gauge Subjective Results
Aside from looking at your KPIs and LMS reports, it’s important to gain an idea of how employees felt about the training, and whether or not it improved employee relationships and attitudes about work.
You want to gain this feedback both during and after coursework has been completed.
One of the best ways to do that is through the use of a post-training employee survey. When measuring engagement with the use of a survey, it’s important to consider the questions being asked.
Here are a few tips for gauging employee attitude and response (level 1 of the Kirkpatrick Model) to the coursework once it’s completed.
- Choose open-ended, impactful questions.
Use open-ended questions (those that can’t be answered with a yes or no) that can help you gain insight, like, “How would you rate the amount of materials covered in the course?”
Some other examples might include:
“Rate your confidence level for completing the knowledge or skills presented”
“What part of the e-learning course did you find most useful and interesting?”
“What additional material would you like presented in the course?”
- Create (moderate) incentives for survey completions.
You could make surveys mandatory, but you may not get full or completely honest answers from a mandatory survey. Instead, consider making the survey optional but include an incentive for completion.
Choose something that would improve the employee’s work experience, like a small gift card, rather than a big financial incentive. A large incentive might skew the answers to be more favorable.
- Use feedback to shape future coursework.
Keep in mind that an employee survey can’t measure actual KPIs, but should be used to measure thoughts, emotions, and general attitudes surrounding your eLearning course.
This includes whether or not employees found it dull or unhelpful (in which case, changes to the coursework may need to be made), or whether or not they enjoyed certain aspects of the course (in which case, you could use those again in future training courses).
More Engagement Measurements to Consider
You may also want to look at other subjective and non-subjective tools to measure engagement.
Consider adding any of the following metrics to your assessments:
- Gamification results – If you’re adding gamification to your eLearning course, whether through a separate application or tracking system, make sure to incorporate those results into your engagement metrics.
- Company sales – Pay close attention to your sales numbers in your sales system, especially after implementing a new eLearning course for those employees that have a direct impact on sales.
- Learning expectations – If certain parameters were set for success before the course began, take note of whether or not those expectations were met, either by the employees or the managers.
- Management surveys – Employees may have a different experience with the outcomes of eLearning training than managers or course instructors. It’s equally important to gather insights from leaders to ensure expectations are met.
Keep in mind that measuring eLearning properly will require multiple approaches.
Measuring eLearning isn’t a one-time event, but will require ongoing measurement of both subjective and objective results.
You should also consider measuring effectiveness and engagement levels both during and after a training course, in order to gain better insights.
Weekly and monthly LMS reports, in addition to employee (and manager) surveys, and other KPI metrics, like sales or company growth, can help give a clearer picture as to whether or not an eLearning training course is truly effective.