What does the CLO really want? The ideal learning data set

Nov 09, 2018 by Robert M. Burnside

Get Our Newsletter

Sign up to get the latest expert commentary, analysis, and news on enterprise learning and capability academies delivered straight to your inbox.

As former CLO of Ketchum, I have a good idea of what my peers want when it comes to data. We want data that gives us the king—le roi—the ROI, the return on investment that results from the precious organizational funds we just spent on helping our employees learn. Throughout my career, the L&D function has labored over the disconnect between funds spent on learning, and how that learning has resulted in higher achievement of organizational goals, among them the all-important increased revenues and profits.

Along the way, I gathered and used various data sets that don’t quite do the job:

  • Average # of hours of learning time per employee
  • Self ratings of value of the learning and ability to apply it on the job
  • Quizzes, tests for understanding
  • Record of completion of the learning tasks (Learn more on Why can’t I find stats on e-learning completion rates?)
  • Efficiency of learning spend, e.g. $/hour of learning per employee
  • Skills ratings before and after
  • Qualitative assessments by customers and managers
  • But now, with AI algorithms emerging that push our interpretation of data to the next level and give us a broader picture, it feels reasonable , to dream a bit and imagine the ideal data set to gauge the value of our investments in learning:

  • The time the employee took to learn the new skill is correlated with the reduced time – greater efficiency on the job – and the net result is the learning increased the efficiency of the employee in reduced time investment to achieve the task
  • The changed behavior that resulted from the learning is correlated with how this new behavior impacts greater sales and profits
  • The share of employees learning the new skills and behaviors is correlated with the increased share of market for the employer
  • The average performance of employees going through the learning is compared to the average performance of employees not doing the learning
  • The investment in the learning is correlated with the satisfaction ratings of customers, and the satisfaction rating of customers is correlated with organizational profits and revenues
  • Okay, let’s stop there for now. Let’s assume our AI algorithms can do the above job – have we gained our goal? Not really, as the primary goal of data collection by CLOs is to prove the learning process and knowledge gained is real, sustainable, and can be applied in constantly changing conditions. And to attain this, it matters that the learner is interested in it and sees it as valuable and meaningful in the context of her or his life. And the concepts need to be clear and rigorous, proven. Finally, the enactment of the knowledge on the job needs to be efficient and effective. And in the end, the new learning and knowledge needs to be shared among the overall group tasked with the organization goals, it can’t really live in a silo.
    Finally, the disruption in the current world of digital transformation is such that one doesn’t really want people to learn something too well—that is, to get certain behaviors and tools ingrained as it must shortly change.

    Our current world requires learning that is flexible, fungible, fluid, mobile, seeking constantly to question its assumptions. How exactly do we measure that? Let’s step back and take the long view, where have organizations failed, where have they thrived? Kodak failed to keep up with the evolution of photography – what were the characteristics of its learning function? Apple has been able to ride and evolve in the current technology evolution – what are the characteristics of its learning function? When we are able to identify and accurately measure the learning function of an organization and what differentiates failing from succeeding, then we can look to correlate it to the attainment of the organization’s goals, including revenues and profits. It’s up to us humans to figure this out first, then we can manipulate our AI algorithms to gauge it.

    Learn how we gather and present meaningful data on our award-winning platform, book a demo today

    Nomadic’s Approach to Learning Design

    At Nomadic our approach to learning design lies on the fact that we don’t want to teach - our focus is on what we want people to learn. That small shift informs every aspect of our process. And, leads to more effective collaborative learning.

    The Key Ingredient For Creating Exceptional Learning

    Extraordinary content, design, and technology get us some—but not all—of the way to exceptional learning. There’s one more key element: social interaction. In this article we show you how to include social interaction to create exceptional learning experiences.