What does the CLO really want? The ideal learning data set
Nov 09, 2018 by Robert M. Burnside
As former CLO of Ketchum, I have a good idea of what my peers want when it comes to data. We want data that gives us the king—le roi—the ROI, the return on investment that results from the precious organizational funds we just spent on helping our employees learn. Throughout my career, the L&D function has labored over the disconnect between funds spent on learning, and how that learning has resulted in higher achievement of organizational goals, among them the all-important increased revenues and profits.
Along the way, I gathered and used various data sets that don’t quite do the job:
Average # of hours of learning time per employee
Self ratings of value of the learning and ability to apply it on the job
Quizzes, tests for understanding
Record of completion of the learning tasks (Learn more on Why can’t I find stats on e-learning completion rates?)
Efficiency of learning spend, e.g. $/hour of learning per employee
Skills ratings before and after
Qualitative assessments by customers and managers
But now, with AI algorithms emerging that push our interpretation of data to the next level and give us a broader picture, it feels reasonable , to dream a bit and imagine the ideal data set to gauge the value of our investments in learning:
The time the employee took to learn the new skill is correlated with the reduced time – greater efficiency on the job – and the net result is the learning increased the efficiency of the employee in reduced time investment to achieve the task
The changed behavior that resulted from the learning is correlated with how this new behavior impacts greater sales and profits
The share of employees learning the new skills and behaviors is correlated with the increased share of market for the employer
The average performance of employees going through the learning is compared to the average performance of employees not doing the learning
The investment in the learning is correlated with the satisfaction ratings of customers, and the satisfaction rating of customers is correlated with organizational profits and revenues
Okay, let’s stop there for now. Let’s assume our AI algorithms can do the above job – have we gained our goal? Not really, as the primary goal of data collection by CLOs is to prove the learning process and knowledge gained is real, sustainable, and can be applied in constantly changing conditions. And to attain this, it matters that the learner is interested in it and sees it as valuable and meaningful in the context of her or his life. And the concepts need to be clear and rigorous, proven. Finally, the enactment of the knowledge on the job needs to be efficient and effective. And in the end, the new learning and knowledge needs to be shared among the overall group tasked with the organization goals, it can’t really live in a silo.
Finally, the disruption in the current world of digital transformation is such that one doesn’t really want people to learn something too well—that is, to get certain behaviors and tools ingrained as it must shortly change.
Our current world requires learning that is flexible, fungible, fluid, mobile, seeking constantly to question its assumptions. How exactly do we measure that? Let’s step back and take the long view, where have organizations failed, where have they thrived? Kodak failed to keep up with the evolution of photography – what were the characteristics of its learning function? Apple has been able to ride and evolve in the current technology evolution – what are the characteristics of its learning function? When we are able to identify and accurately measure the learning function of an organization and what differentiates failing from succeeding, then we can look to correlate it to the attainment of the organization’s goals, including revenues and profits. It’s up to us humans to figure this out first, then we can manipulate our AI algorithms to gauge it.
Learn how we gather and present meaningful data on our award-winning platform, book a demo today