This Year’s Best LRS is….Hold on….

Recently, Craig Weiss published a ranking of Learning Records Stores (LRSs) on his blog – see LRS Rankings (Learning Record Store).  When I first saw this, I was excited that xAPI was getting such great coverage.  Craig’s rankings of eLearning technologies are well respected and his paying attention to LRSs is a great sign for the xAPI movement.

Unfortunately, Weiss missed the mark here.

I wish I could say that this error was along the lines of Steve Harvey’s Miss Universe mistake in 2016 or the Academy Award’s flub in naming Best Movie winner this year.

But Weiss’s errors in his assumptions for his rankings have to do with a fundamental misunderstanding of what an LRS is required to be under the ADL specification and a confusion between what the xAPI data specification delivers and what it might enable other tools to deliver.

LRSs are required to:

  • receive and store xAPI statement that complies with the xAPI technical specification.
  • reject statements which are not compliant
  • consider statements received to be immutable – ie, the cannot be deleted
  • provide access to data by other LRSs or 3rd party systems that request the data

That’s it.  LRSs are about verifying xAPI formatted data, storing it and sharing it.  Recently, ADL released a tool with which LRS vendors can verify that their LRS is conformant to ADL specification at

A “by specification” LRS will likely never be seen by an end user – learners, instructors, data analysts.  Some LRSs are building “back door” basic analytic interfaces, but those are not required by the specification.  (More on analytics below.)

Weiss outlines several “Keys to Remember” when analyzing LRS (and by extension, xAPI).  The first is:

There are some standards in terms of what data is captured that is seen in most LRSs. These include top influencers, xAPI, statement activity, most popular content, search extraction, data visualization (but all over the place from that standpoint), and connectors.

LRSs only accept, store, and share valid xAPI statement.  xAPI statements, depending upon how they are constructed, may release the data needed to determine top influencers, most popular content, and search extraction information.  xAPI data, like any data, can be used for data visualization.  But none of these things are inherent in xAPI statements.  xAPI provides access to new data to be analyzed (more on this below). As to connectors, yes, xAPI and LRSs utilize standard API protocols for connectors to other systems.  This is not unique to xAPI – which is a powerful aspect of the specification.

Weiss’s second and third keys are true.  Some LMS vendors are incorporating LRSs into their architecture which has advantages and disadvantages.  This post by Rustici Software outlines the various configurations that can occur with an LRS.  He seems confused that some of the LRSs have both an open source and a commercial version.  xAPI is an open source standard.  The “instructions” for building an LRS are open source,

He seems confused that some of the LRSs have both an open source and a commercial version.  xAPI is an open source standard.  The “instructions” for building an LRS are open source. Anyone is welcome to build their own.  However, the reality, building your own LRS is not easy.  (there are 1300 tests to pass for an LRS to be determined conformant by ADL.)  So naturally, there will be many customers, or LMS vendors, who will be more interested in buying an already built, conformant LRS from a vendor than taking on the cost and effort to build their own.

Weiss then says that vendors have forgotten one of the “premises of LRSs”:

The premise (besides what it can do and its benefits) was that each learner has this data record and it captures everything (which it does), BUT and here is the kicker, if the learner leaves the company, school, etc., they take their data record with them.

This is and has never been a premise of the LRS nor the xAPI specifications.  What he is confusing here is an aspirational goal that might be achieved if xAPI is widely implemented.  The role of the LRS in this aspirational vision is that it is a vessel to hold and transfer xAPI statements which can be transferred to any other LRS with no worry about data configuration.  Thus, ideally, a student’s learning activity, once recorded in xAPI statements technically becomes portable – dependent upon either the student or their new school or employer having an LRS it can be transferred to.  These technologies are being developed.  There are issues of data ownership and privacy that will impact this vision, but that is not relevant to this current discussion.

He then makes an accusation that is totally false.

Some vendors though have changed the premise of the data record transference.  How?

  • They delete the record if the person leaves (regardless if they quit, fire, bolt, go the route of the school angle above, etc.)

As I stated above, a key feature of xAPI and LRSs is that statements are considered immutable once created.  They can not be deleted per the specification.  If an error has been made in the creation of a statement or statement, a second VOID statement can be generated to negate the first statement.  But this is a very laborious procedure and, as far as I know, is generally used only to negate test statements so that they won’t appear in any analytics.  Even in cases of test statements, I’ve been advised it is easier to simply create a new LRS and start over than to try to VOID all of the incorrect statements.   This is a strawman on the issue of why the aspirational goal of lifelong learning records.  Vendors are not deleting valid statements willy nilly.

Finally, the main basis on which Weiss founds his rankings on is the learning analytics interfaces that some of the LRS vendors have opted to add to their LRS offering.  This add-on interface is not part of the ADL specification conformance.  Why are the vendors doing it, then?

  1. Value.  As has been discussed, building an LRS is free to anyone who wants to take on the task, so there is little value-add in building one – at least not enough to entirely support a re-selling business model.
  2. Visibility.  A free standing LRS, if it were physical, is basically a box sitting there, running quietly, holding data.  I don’t want the job of trying to sell that to someone!  So there needs to be at least some minimal backend that shows the data is there and safe.
  3. Demonstration.  At the same time, until very recently, no one has been paying attention to learning analytics (except many of these vendors who have been at the heart of the xAPI movement) so demo’ing the learning analytics capability of xAPI in a value in pushing both xAPI and the LRS.

Some of the vendors clearly are aiming at participating in the Business Information and Analytics Marketplace and it shows in their “dashboard”.  Others as simply interested in providing some basic “control” information to show that the LRS is healthy and operating and will expect data will be drawn out to bigger beefier BI and Analytic tools.

The irony of all of this is if I had to rank the LRSs he has listed, I’d probably have the same top four.  But Weiss’s assumptions underpinning his selections misrepresent the role of LRSs and ultimately do a disservice to the aspirations of the xAPI standard.

What do you think?  Should LRSs be ranked?  On what criteria?  Please share your thoughts below in the Comments section.


Feature image by Ryan McGuire provided by Gratisography.

Times are a Changing – That’s Good!

[Reflection is a key part of the learning process.  Time for me to incorporate it into my blogging!]

This week (which isn’t over yet!) has been crazy busy, but inspiring and edifying.  Between preparing for teaching/leading a second of my CPLP study group, the Masie Systems and Tools conference, presenting our project as a part of the xAPI Spring 2017 Cohort, and my usual personal learning activities, it’s been a week of seeking>sensing>sharing at its best.

Some of the insights from others and that I’ve had this week:

  • The breadth of knowledge we are expected to understand as L&D professionals is astounding – yet this week, I met and interacted with dozens of folks who navigate between learning theory, finance, analytics, social dynamics, impact of future technologies, technological infrastructure, group and organizational dynamics, etc. and do so effortlessly.
  • Elliott Masie shared three fundamental changes that are occurring in how humans learning:
    • Memorization for learning is declining being replaced by Familiarization as a tactic.
      • Based on the research of Betsy Sparrow
      • We aren’t as concerned about absorbing information as we are in learning how to navigate to it
    • Reading is being replaced by watching
      • 60% of all web landing pages incorporate video
    • It is no longer the WHAT that matters, but the WHO
      • learners will search for information, determine WHO provided it, and contact them for the learning
  • “We’ve become a world of self-service.  Except in corporate learning.” – Bob Mosher
  • Meeting people is easy, building network connections is hard.
  • Millenials (and other learners as well) want intense, immediate experiences to learn from.  They don’t want to wait until a workshop in June or a webinar next week.
  • Rob Lauber’s (CLO at McDonald’s) answers to rapid response questions from Elliot Masie:
    • xAPI – “Could change the game – personalized and transparent; aspirational”
    • search – “I’m already there.”
    • interoperability – “Has to happen for the future.”
    • badges – “I don’t see recognition from outside organizations”
    • virtual reality – “Not serious. I have people pitching me how great it would be for employees to be learning in a virtual restaurant while they are in a real restaurant.”
  • Getting back into a full-scale learning/study mode is challenging!
  • Building a data strategy is probably the last thing most L&D Professionals want to do, but not building one is much more perilous.
  • Irony – one of the two courses I got a C in back in college is fast becoming a component of my everyday life.
  • No matter if it becomes the standard or not, xAPI is driving a conversation about what learning is and how learning analytics should be formulated.  It is driving data knowledge/literacy in L&D.  These are good things.
  • In her article The Future of Learning Measurement has Arrived on, Caroline Brant says, “now is the time for innovation in L&D”.  She points to innovations like Artificial Intelligence, micro-credentialling, intelligent learning, and xAPI that are promising to radically change L&D.   She makes the point that it’s time for us to prepare ourselves so we can be a part of the innovation.

I couldn’t agree more with Caroline’s reply to a comment on LinkedIn where she reposted her article, “It’s a great time to be in the [L&D] space.”

What are your thoughts on the state of L&D and the challenges we face?  Are you optimistic or apprehensive?  Please comment below and let me know where you’re at.


10, no 11, Trends that Promise to Disrupt L&D

Recently, Josh Bersin posted about the changes happening in workplace learning in The Disruption of Digital Learning: Ten Things We Have Learned.  I’ve known Josh and his work for 12 years now and from the beginning, I’ve found both his research and his analysis to be rock solid.  This post is no different.

He generally isn’t swayed by today’s latest fad.  Microlearning and gamification are variables in the mix, but not what Josh views as trends.  He looks at more foundational/structural trends that the fads may be an element of.

He opens the post by talking about the gravity of the change ahead.  L&D has grown into a $140 Billion dollar industry.  He also notes that 83% of companies see delivering compelling, digital learning experiences as urgent or important. Add to this the data from so many other sources that C-Suites around the world are growing weary of L&D to become a strategic partner in the enterprise and you have a massive, volatile transformation in progress.

He points out that this isn’t just a shift in the tools we use it’s a complete shift in what L&D professionals do.  It’s not about changing textual content to video or making small chunks of learning.

…so our job now is simply to “deliver learning to where people are.”

It’s about phones or VR.  It’s about bringing learning to where employees are.  We’ve been talking about delivering what the learner needs, when the learning wants it, whenever the learner wants it, where ever they are for ages now.  Well, rehearsals are over – the curtain is going up!

Here are Josh’s 10 Trends that will disrupt L&D. (my commentary is in blue)

  1. The traditional LMS is no longer the center of corporate learning, and it’s starting to go away.  Why?  It’s old.  Based on a 30-year-old mindset focused on a course catalog and compliance.  The paradigm has shifted.  As Bersin quips, “their cheese has been moved.”  I agree that the “traditional LMS” is on its down slope.  But I’m curious to see how well those who are dancing on its grave will react to the new world order.  The expensive, controlling LMS is out, but L&D has had the luxury, in most cases, of being left alone with its big toy.  The learning ecosystem will be dependent upon systems that L&D has limited control over.  New rules of usage may impact learning implementations.  Whether a needed functionality is turned on or not will often be in the hands of IT or Sales or Marketing.
  2. The emergence of the X-API makes everything we do part of learning.  Everything we do is part of how we learn at work.  xAPI will enable delivery and tracking of all of it.  Bersin indicates that vendors will be building more and more tools that are xAPI compliant. No doubt, you know this brought a smile to my face.  If it gets rolled out properly and quickly, xAPI will be a game-changing enabler of new ways of guiding and tracking learning.
  3. As content grows in volume, it is falling into two categories: micro-learning and macro-learning.  His point here is that there is micromacro-1an appropriate time for all “sizes” of learning during an employee’s learning journey.  I believe that microlearning has been blown totally out of proportion in the past 2 years.  Microlearning isn’t new.  We used to call it “chunking.”  It plays a role in effective learning at different times in the learning process.  But it isn’t the square peg that finally fits into the round hole.
  4. Work has changed, Driving the Need for Continuous Learning.  Reading and answering emails takes up 28% of our time. 19% of our time is spent searching and gathering information.  Combined with the statistics Bersin quoted at the beginning of the post about the need for more learning because of rapid change and growing complexity, the 24 minutes employees spend, on average, in learning activities clearly isn’t enough.  No one has time for “course level training” anymore.  Combined with the trend above about crafting learning journeys and the trend below about spaced learning, this trend is obvious.
  5. Spaced learning has arrived.  No need to go into depth here.  We’ve finally discovered that research begun in the late 1800’s shows we forget things.  And with all the information flowing at us, we forget more.  But we’ve also discovered that spacing out learning and reviewing and questioning for retrieval increase our ability to retain and recall information.  OK, Bersin wasn’t as snarky as I may have just reflected it.  This is a major shift in the way L&D thinks about learning.  It’s also a trend that will give us some quick wins if we measure it well.  Retention up.  Scrap learning costs down.
  6. A New Learning Architecture Has Emerged: With New Vendors to Consider The LMS isn’t dead, but it’s only one of the players on the field.  There is a wide range of new tools hitting the market to meet the needs of the learning that is more personalized, self-directed, and just-in-time.  Some of the new tools are from vendors we know, but many are by new players.  The landscape is going to be shifting for a while through this transition.  Not much to agree or disagree with here.  I would advocate that L&D professionals put their curiosity caps on and invite vendors in to pitch these new products or sign up for their demos online.  Sure, it will take up some of your limited time, but it will be well worth it. There is some amazing stuff out there.  Make it a team activity one a month.  Yes, you will get a biased view of the world skewed to that vendor’s sweet spots, but they’ve also had to spend alot of time synthesizing some of the issues in this blog post to get to a point of being able to program a solution.  You’ll learn and, maybe, find a new tool.

    Today learning is about “flow” not “instruction,” and helping bring learning to people throughout their digital experience.

  7. Traditional Coaching, Training, and Culture of Learning Has Not Gone Away  With all of the “new toys” to play with, two key factors in high-performing organizations are mainstays of current practice – culture and coaching.  (I’m not sure why “Training” is in the title of this trend)  Bersin talks about the importance of the four E’s of learning at work (education, experience, environment, and exposure) to generate sustainable development. He shares that he feels there will soon be a tighter linkage between L&D and performance management tools.  Culture is clearly vital to the success of this vision of learning.  Peer support, knowing the organization supports you in taking the time to learn, linkage between learning and organizational objects are examples.  Coaching by managers not only can provide direct support for learning, but it also should generate a “my manager cares” and “I’m not just a number to leadership” feelings, which increases engagement.
  8. A New Business Model for Learning  With the diminution of the LMS and the de-emphasis of 3rd party content collections, L&D will no longer be in the massive capital investment game.  Bersin encourages a “pay by the drink” approach and encourages L&D purchases to push back on vendor pricing.  He also warns that the technology marketplace is going to be volatile for a while.  Vendors will come and go and there were be mergers and acquisitions.  He argues that signing long-term packages might be risky until things settle out.  My reaction is mixed to this trend.  One of the things that got many L&D departments “to the table,” sometimes briefly, was the acquisition of an LMS for millions of dollars.  There are numerous new tools out there that are testing out “pay by the drink” pricing methodologies.  Unless you have strong historical data that can inform what your potential usage might be, these methodologies could result in much larger invoices than you are expecting.  Bersin also doesn’t address the use of open source tools which in some cases are as powerful as their commercial competitors and just as secure.  Finally, APIs, webhooks and other connecting tools like IFTTT, Zapier, and Apiant are making it easier to mix and match vendors and to short cut review periods.  I’m not even going to go into the impact of the Internet of Things that is coming.
  9. The Impact of Microsoft, Google, Facebook, and Slack Is Coming  Tools from these companies are radically changing the digital experience at work.  Up until now, L&D has generally stayed away from email and messaging tools as part of the learning ecosystem, but these tools are incorporating learning capabilities.  Bersin encourages us to think about Microsoft incorporating LinkedIn’s courses into Excel.  He suggests we need to open a dialogue with IT regarding the next generation of messaging tools they are likely considering today.  This is a very important call to action, in my mind.  I am concerned that this may prove to be L&D’s kryptonite.  For numerous reasons, we have stayed away from using normal workplace tools as learning tools.  Microsoft Office, Salesforce, Slack, Google, Github are all already incorporating learning functionality.  If we don’t incorporate these and other tools into our learning experience designs, we may be perceived as out of touch and irrelevant.
  10. A New Set of Skills and Capabilities in L&D Roles in L&D are going to shift and, likely, some will go away as we take on these new challenges.  New roles will arise. new LD technical needs Many companies are already re-training their L&D teams learning design thinking, MVP (minimal viable product) approaches to new solutions, and understanding the “employee experience.”  He does point out that the overarching principle that has historically driven L&D’s work: Our job is to understand what employees jobs are, learn about the latest tools and techniques to drive learning and performance, and then apply them to work in a modern, relevant, and cost-effective way. This can’t be overstated. L&D and it’s professionals are in an “adapt or die” situation.  To use two overused, but familiar terms; we need to be responsive and agile to deal with this changing environment.  Some of us may be sitting in the same chair in 5 years, but the work in front of us is likely to have little resemblance to what is there now. 
  11. (Wait, you thought there were 10?) Based on a comment to Bersin’s post by W. Nema, I’m adding an 11th trend – the Need to Understand and Incorporate Business Structure.  Nema is specifically advocating that business-specific ontologies, taxonomies and metadata are necessary to enable effective contextual search (which is a mainstay of the modern workplace).  Of course, to add to the degree of difficulty, these structures are rapidly changing due to Big Data, Cloud-based interoperability, the Internet of Things, and other factors.  On this specific concept, I totally agree. Those working on xAPI are spending 4 months this spring re-evaluating the role of Profiles in the standard.  Profiles include what Nema is suggesting.  Without rigorously developed profiles xAPI is clunky and hard to program to – at best.  Well defined profiles will enable vendors and practitioners to fully exploit the full potential of xAPI.  But I would expand it to include more than what Nema is calling for.  We need to understand IT, Processes, and Cultural Structures in our development of learning experiences.

PHEW!  Hat tip to Josh Bersin for his ability to synthesize all of this.

Now it’s your turn.  What do you think of any or all of this?  Is Josh on the mark?  Am I a suck up for agreeing with most of it?  What do you think the challenges are that L&D must address?  Please feel free to use the comments section to share your thoughts. Or do like me and refer to this post and comment on your own blog.

Feature image: “Wheel of Disruption 2014 by Brian Solis” by Brian Solis is licensed under CC BY 2.0

xAPI Resource Center Update

The xAPI Resource Center has been updated with new resources covering new developments  with the cmi5 profile (in particular a comparison chart with SCORM by the cmi5 workgroup), the new effort to revise the role of profiles to be more informative and clearer for adaptation by authoring tool vendors, and various other use cases, scenarios and descriptions of xAPI.

Please let me know what you think of this resource center.  One commenter warmed my heart with her comment:

…And although people in the tech industry are extremely smart, they sometimes have difficulty explaining certain things in a non-technical way. This website really helped me to understand what an xAPI is and what it can do. Thank you so much!

That’s exactly why I’ve created it.  To help L&D practitioners understand what xAPI can do and the opportunities it offers.  Glad to know it’s working!

Loopy v1.0 Games the Systems

I stumbled across this fun open source simulation this afternoon.  Despite its lack of nearly any documentation, Loopy 1.0 is an impressive serious game. In Loopy you can play with out of the box challenges or create your own.

The player interacts with the system model by adding “nodes” and “arrows”.  Arrows can have a positive or negative effect upon the relationship between the two nodes.  Consider foxes and rabbits.

LOOPY_ a tool for thinking in systems

In this example, the bottom arrow is positive- indicating that an increase in rabbits will likely result in an increase of foxes.  The top arrow is negative indicating that an increase in foxes will likely result in a decrease in the number of rabbits.  The system runs adding or subtracting from a node based upon the +/- of the arrow(s) that lead to it.

You can add more than one arrow between the same nodes to indicate a strength difference.  For instance, if you double the top arrow above it would basically represent the idea that every fox would lead to a reduction of two rabbits in the system.

You can add whatever nodes you’d like to the system (hunters, a new housing developent, etc) that represent the complexity of the system.

When ready, you click “Start” and then chose one node to either add (up arrow) or reduce (down arrow) and the system starts to run.  You can see where the system stresses.  My one criticism of Loopy is that there is no point where the system “breaks”.  The rabbits are never all killed off by the foxes and hunters.  The banks never go bankrupt.

The creator of Loopy provides three challenges to play with.  So I did, choosing the automation and job loss challenge.

The challenge is at this link You can start the challenge and observe how it works.  This one, as presented puts amazing stress on the “frustration” and “political unrest” nodes because of unemployment caused by jobs being automated.

loopy v1.0 challenge

Keying off of the big clue – “??? what goes here ???” – in the middle of the  challenge system, I began with asking what, if positively affected could reduce job loss.  But it didn’t seem like the answer would come from that route.  Lo and behold, I came to the conclusion that what government needed to do was use some of that tax revenue that goes no where in the original system to reinvest in public education and other programs that help humans who are loosing their jobs to robots and AI to find new purpose in the new world.

My solution can be found at this link

loopy v1.0 dave answer

Education will create a more educated workforce that will be trained in unique human competencies.  I couldn’t figure out a simple way to slow the impact of automation on jobs, but realized that the key node was the frustration that led to political unrest.  With a re-education program for the unemployed and new community outreach programs to empower non-workers to improve their self-image and to find worth in volunteerism.  Ah, a bit utopian in concept, but when I hit the Start button and the up button on tax revenue, the system kept working and working.  Sure frustration would build and there would be substantial political unrest occasionally, the systems would

Ah, a bit utopian in concept, but when I hit the Start button and the up button on tax revenue, the system kept working and working.  Sure frustration would build and there would be substantial political unrest occasionally, the systems would relieve the tensions and keep on going.

I found Loopy easy to use and it did make me think about how the system would accept change and interaction between nodes.  As you can see from this post, it enables the sharing of the model I create.  It can the be manipulated by others.  Construction of systems can be collaborative.  All in all, a nice little tool.

Your turn: What do you think?  Do you see a way that a tool like Loopy could fit into a learning experience? What type of learning would you try to implement it with?  Please leave a comment below in the reply area.

Feature photo by NASA via