In the last five years, I have seen many articles, videos on Youtube, and even TV news with the following title: X Jobs that will disappear in the Y years. Probably you’ve seen them too. These pieces conclude like this: companies will need to train their employees for other jobs, sometimes jobs that don’t exist.

I recommend taking these pieces with a grain of salt.

My father works as an electrician in the same job since 1988, before I was born. While the machines that he operates renewed a few times over the last 33 years, he still performs the same job, with the same results. Just way easier because of the advances in technology and automation.

A couple of years ago, I asked my dad what he would be doing if his job would be fully automated and become replaceable? He still has some years until retirement, so he should still be working for several years.

He said that he probably would look for an electrician job somewhere else. I asked him if he would consider switching careers.

With a smile on his face, he said: “I’m too old to learn something new.”

We are now at the end of an era where someone could have worked in the same job his/her entire life, retiring after 40-45 years. For instance, the position of a payroll clerk did not change in the last 40 years too much; the result it’s the same: the employees need to receive their salaries. In 1980 they did the payroll on paper, then the payroll register moved in an Excel file, then in a mainframe payroll engine. Every technological milestone came with a learning piece. But now, Payroll Systems are brilliant. Once configured, they can calculate payroll based on timesheets, consider all HR data changes, calculate taxes, both state and federal, create and send payslips by email. The job of an entire team it’s now done by software.

We are facing something new, probably for the first time in human history: people will have multiple careers in their lifetime.

And they will need to learn a lot to be able to switch careers.

In The Future of Jobs Report 2020, done by the World Economic Forum, one of the top 15 skills for 2025 will be Active learning and learning strategies. To support an entire generation switching their careers, people specialized in learning and development will be in high demand. Together with L&D specialists, systems will need to be easy to implement and use. The circle closes with Learning Analytics: we will need to understand if the learning programs deliver results or are any gaps or issues.

Education versus Learning

Before jumping into the practical part of this article, I want to emphasize the difference between education and learning.

When you hear about changing careers, you might think about starting a college in the 40s or 50s.

That may be because education is an industrial system – a system based on command and control. It makes people compliant, from elementary school to higher education. Often, corporate training follows the same rules.

Learning should be a voluntary engagement process – engagement should be the core of learning and the backbone of Learning Analytics.

How to start the Learning Analytics journey?

Probably you should start with the technology stack. Modern Human Resources Informational Systems (HRIS) has a dedicated module for Learning (LMS). Most of the time, it’s not part of the core suite, and it needs to be acquired separately.

This system should contain all data points about Learning, data points that will transform into metrics, and metrics will convert into insights.

In my experience, with an LMS in hand or without, L&D professionals still keep a data track record in Excel files. While this is a good backup solution, I strongly recommend having the data held as the primary source in this manner. Excel files can be subject to human error. Most often than not, spreadsheets cause issues because of a broken formula, a forgotten filter when dragging down a cell, a typo that can return #N/A without knowing why.

Another issue with spreadsheets it’s scalability. Once the headcount exceeds a few hundred employees, there are a series of downstream effects: the number of new hires increased proportionally, these new hires need to become productive fast, the training needs adjustments to innovations and technology advances – in the end, training need increases and so does the number of data points that have to be maintained. Even the most advanced tracking spreadsheets, once over a few thousands of lines of raw data, become hard to operate, and often they crash.

That’s why the best practice is to have a Learning Management System that fits your needs.

There are hundreds of systems you can choose from. If you ask me what the best system for X, my answer is: it depends. Here are a few things you should consider when looking for an LMS:

• Your current processes and how flexible are you with changing/redesigning them. Start by analyzing processes and map them against the system’s features – that’s a great way to observe what enhancements and new functions you can develop in the future.

• Integrations with other systems are crucial, especially for companies where the tech stack is not part of an enterprise software ecosystem like Oracle, SAP, or Microsoft. Ask the vendor about the possibility of integrating via API.

• Reporting capabilities – start by asking for standard reports, already built-in that you can use from day 1. Then move to dashboards – I always recommend choosing systems that have dashboarding capabilities – it saves a lot of time and is an excellent way of sharing insights with managers and leaders.

• User Experience (UX) – well, in the end, it’s all about our employees, and since they will use this system, it’s crucial to offer them a pleasant experience. The focus should be on ease of navigation, as many clicks as possible, 1,5-2x play speed, and anything else that would make using the software more enjoyable. Think of your favorite app and try to identify why you like it so much, and then look for the same things in the system.

What metrics are important?

Short answer: the ones that might solve you a problem. Easier said than done, but the truth is that without a problem, it’s hard to focus on the right metrics.

In the below part of the article, I will present the Learning Metrics split into three categories:

Standard Metrics (aka must-have metrics)

1. Participation rate is the percentage of employees participating in development opportunities. Also known as training participation rate, the formula used to calculate it is the number of employees who participated in a training divided by the number of eligible employees. This metric is an excellent indicator of the relevance of specific training, especially the non-mandatory ones.
2. Completion rate is the percentage of employees who finished training, and it’s calculated using the formula: the number of employees who completed a training divided by the number of employees who started it. Low completion rates might indicate that employees find the class difficult or not applicable.
3. Abandonment rate – it’s the opposite of completion rate: the percentage of employees who did not complete training. Some companies look at this metric and have targets around it.
4. Cancellation rate is the percentage of employees who signed up for a training and canceled before it started. It’s calculated using the formula: the number of employees who signed up for a training divided by the number of employees who showed up for it.
5. Time spent in training – let’s be honest, we all clicked next – next – next without actually reading the info, or maybe we skipped some parts of a video training. It can be measured at the employee level, and we can see who spent less time than the actual time needed to go through the training.
6. Pass rate – is the percentage of employees who passed a test at the end of training divided by the total number of participants. Training with a test is a great indicator to understand how the participants assimilate the information provided. If multiple takes are allowed, this can also be measured.
7. Devices used
8. Learning hours per employee/FTE – the name is self-explanatory. I highly recommend tracking this metric and comparing it to external benchmarks. From 2009 to 2019, the learning hours per employee increased, and it’s essential to keep pace with the market.
9. Knowledge retention rate – this involves assessing the knowledge after the training is completed. While this is a compliance requirement for some classes, verifying the knowledge retention after a training completion can be a way to understand where employees struggle to understand some parts and maybe address them to solve this issue.
10. Cost of training – again, another metric that can be compared to external benchmarks. But why I like it so much, it’s because you can cross-check it with high-performance employees/teams.
11. Questions rate – this refers to the number of questions submitted by participants in instructor-led programs. It is an excellent indicator of engagement. Even when questions are addressed verbally, the trainer or a training assistant can note each question, either for online or offline training.
12. Confusion Flags – this is a cool feature that I wish many LMS have: when a participant clicks it, it means that the concept it’s hard to understand, or it’s not explained well. It is an excellent marker for instructors to review the most marked pieces of content.
13. Time to competency – it’s the metric that measures the time since an employee has been trained for a particular skill until it’s proficient in performing that skill. Please note that it’s different than time to productivity, a metric used for new hires from their start date until they become fully productive.
14. Performance rate increase – this metric is a great key performance indicator when evaluating the outcome of training. Let’s be honest: in the end, it’s all about increasing performance as this increases the revenue per hour. To measure it, we should measure the performance before the training and check the delta (variance).
15. Impact of Training on Revenue – probably the most down-to-earth meme regarding L&D is saying: What if we train them and they leave? What if we don’t and they stay? – could not agree more. I would measure the numbers of hours spent in training against the revenue – more often than not, you will see a dependency there.
16. Return on Investment – the queen of all metrics, the famous R.O.I. Traditionally, the formula used to calculate it is Benefits (Return) divided by Cost (Investment) = R.O.I. The problem with this formula it’s how we measure the upper part of this fraction, the benefit. There is no general way to do it, so this comes to establishing some goals initially, and those should be considered the benefit. If all goals are achieved, we should be able to attach some units of measurement like money, time, or productivity.

Why it’s important to track these metrics?

Learning Analytics can change how we measure the impact and the results of teaching programs in our companies. Measuring and tracking these metrics can provide a more comprehensive picture of how well initiatives are performing, what areas can be improved, and in the end, how we can better help our people.

We always need our stakeholders’ buy-in. Having data points that can show the progress and outcomes of any program reported to productivity and revenue is always a good idea.

Conclusion

Learning Analytics is something new in the industry. Unfortunately, the People Analytics community does not focus on this subject as intensive as it should do. In my research for this article, I found very few academic papers or studies. One explanation for this might be that learning analytics has been the same as traditional education analytics for a long time. One of the ideas I want to emphasize is: learning it’s a voluntary engagement process, while education is mandatory.

If you want to explore this topic more, I would suggest starting by creating a list of problems you encountered or seeing and collecting data that you might analyze. If data is not available, look for a Learning Management System. If there is no budget for this, start tracking all activities in an Excel file. Start small, with a team or department, then scale. Create a list of recurring reports, maybe monthly or quarterly, and see the progress. Cross-check data with other People Analytics like Turnover or Revenue and look for dependencies.

L&D is on its way to becoming the area of Human Resources with probably the most significant impact on employees’ life.

‍