Hey L&D! Your Underpants are on Fire!

This is the second of a 2-post series on the challenges L&D faces and solutions to overcome them.  In the first post, I addressed the challenges.  In this second post, I offer a suggested solution.

In my last post, Hey L&D, Your Underpants (gnomes) are Showing!, I discussed the black box that training and development has historically worked in.  Mimicking a South Park episode, I suggested that we have utilized the following model:


If we get honest, we don’t have any real clue if what we create actually will increase the ability of our learners to do their jobs better and impact business goals in a positive fashion.  We’ve gotten away with it because senior leadership knows that talent development is crucial to the organization’s health and ability to deal with ever more complex and changing markets.

But they are being pressured to justify every expenditure in the budget to ensure is is driving business priorities, goals and profits.  The free pass we in L&D have been receiving is about to be revoked.

We need to overcome what Matthew Syed calls “Black Box Thinking.”  We have been enabled by our organizations and the general culture to cover up and hide from our failures. One example.  If we don’t look at long-term retention of information, ignoring the forgetting curve, we can be satisfied with Kirkpatrick Level 1 and 2 responses that say our training was “great”, “just fine.”  Our learners don’t want to point out that they don’t remember what they learned a month ago – because it might make them look stupid.

We have to open up that black box and determine how to work in a new way and we don’t have much time.  Now a full-scale overhaul of L&D will take time and I’m sure that most senior leaders will work with us. If we set out a strategic plan to transform learning in our organizations and get working on the needed changes to make it so, they will give us the time to do it.  But the time to take action is now.

So what do we have to do now that we haven’t been doing?  (NOTE: these actions are interrelated.  This is not a step-by-step process.)

Data, Data, Data

And I don’t mean a compilation of smile sheets. We need data-driven evidence of the business impact of learning.  As CEB put it in a Learning Analytics whitepaper last year:

You can’t defend L&D activities to executives with anecdotal evidence, buzz from hallway conversations, or surveys that show program participants liked their instructors and were satisfied the learning experiences.

We need to build a comprehensive Learning Data and Analytics Strategy that will enable the collection, storage, analysis, and reporting of data.  Data about training efficiency, learning effectiveness, target behaviors, and business results.  Some of this data doesn’t exist today, some exists but our stakeholders “own” it (ie; sales, performance data), some might be at our fingertips, but we don’t know how to access it) and we may even need benchmarking data from outside sources.  It is vital that we work to develop an understanding of what data we need in order to answer the questions and then figure out where it will come from – is it collected manually or automatically?  where will it be stored?  who will analyze the data?  who needs to see reports of what? when? how?

We’ll have to understand how data, particularly big data, is handled, warehoused, and managed with our organizations.  What software tools do we need?  Are there IT resources to connect APIs?  to write scripts to link programs?

Change and Promote a Learning Brand

Even if you don’t think your learning has a brand, it does.  Jeff Bezos, CEO of Amazon, said: “Your brand is what other people say about you when you’re not in the room.”  More importantly, the quality of a brand determines how much people will trust you, how loyal they will be to you, will they have your back when others attack.

What do you want L&D to be known for in your organization?   A trusted business colleague?  A problem solver?  A group that uses evidence to make decisions?  A brand is a consistent message of who you are.  That consistent message is built through conversations and commitments fulfilled.  It is enhanced by actively listening and collaboratively determining business goals with your stakeholders.  Using data to drive decision making and in reporting results.

A Learning Culture focused on Continous Learning

Move away from event-based training and toward socially energized learning experiences.  Executives and Managers need to understand that helping their employees to enhance their knowledge and skills both specific to their current job and as professionals is a part of being a leader.  L&D needs to provide support, tools, and training so that managers can fulfill this role.  If you can get learning objectives incorporated into the performance management process, all the better.

Help employees to become self-directed learners.  Our employees know that in today’s world it is imperative that they expand then knowledge, hone their skills, take on new challenges, and become better professionals.  We need to aid them in this process.  Help them to learn how to learn.  Scaffold social learning experiences online and in their everyday interactions. Reflect on their work – on their own and with others.  We need to provide them with easy access to the resources they need to do their jobs.

Change Everything about L&D

We need to take a deep and honest look at how we are, or more likely aren’t, meeting the needs of our organizations and ourselves.   This will require the courage to face the fear of admitting past deficiencies and proposing radical change.

This goes to how we design our learning experiences which need to be grounded in business goals and data-driven decisions.  How we deliver learning needs to move to a multi-faceted, extended multi-c0ntact sequence of experiences.

We need to take the time and make the effort to learn the businesses we work in.  We must become peers with our stakeholders by understanding how they contribute to the success of the business.  We need to be able to engage in conversations that explore the details of their goals and the knowledge and skills necessary to meet them.  We need to walk the walk of a true business partner.

We also need to open ourselves to new ways of thinking about learning and living in a new world that is fast approaching. The science of learning, neuropsychology, big data, learning analytics, artificial intelligence, the internet of things are changing the world around us.  Think about how mobile devices has changed not only how we need to design and deliver learning, but also how humans interact with each other and the information they need.  The changes coming are even bigger than the impact of mobile.

Partner with Stakeholders as Never Before

Any major change in an organization that affects all employees (as learning does) is going to be disruptive, difficult, and met with resistance.  In order to effect the changes discussed above, we will need partners who are committed to our success.  Partners who trust that we have their best interest in mind and who will in return have our backs when things get rough.

Change the conversation.  Let them know that you know what their goals are and that you understand their needs.  We need to find an outcome or set of outcomes and related measures for learning that they can feel co-ownership because they tie to their business goals.  We have to be consultants to their business.

But simply setting agreed upon outcomes and measures isn’t enough.  We also must demonstrate a knowledge of the influence chain within our organizations.  Who do they listen to?  Does their boss delegate authority or do we need to help them advocate to their boss?  Do they defer to another manager or operations group (if IT says it’s ok, then….).  Are there roadblocks that they may not be aware of?  Knowing the political terrain around the solution will help both of you.  Because L&D isn’t siloed, you may have an oversight of the organization that they don’t have.

Communicate Early and Often

L&D should have a comprehensive, well articulated Communication Plan.  How do we plan to promote individual programs?  How do we plan to share our brand? What reports or dashboards can we provide to our stakeholders? sponsors? learners?

We need to be transparent with the organization about would we are doing, why, what our goals are and the results.  Reports and dashboards for our stakeholders and sponsors that show how we are contributing to their success.  Work out loud.

What “Phase Two” Should Say

In the end, we want to be able to say what Phase Two involves.  It won’t be easy, but we need to break the cycle of accepting less than we are capable of.  Our new model then would look like this:


What do you think?   Do we need to stop being underpants gnomes?  Can we make these changes?  What do you see as obstacles to making these changes?  Please share your thoughts in the comment section below.  I’d love it if you used the sharing buttons below to share this post with your social network. 

An xAPI Resource Center for L&D Professionals



Over the past several months I’ve been learning about the xAPI standard for learning experience data interoperability that is gaining traction and is poised to replace SCORM.  This Resource Center is a result of my studies, conversations, and reflections on this exciting advancement for Learning and Development.

My xAPI journey began on September 13 when I saw a Twitter post for something called “xAPI Camp” which was being held that Friday at Lurie Children’s Hospital here in Chicago.  Having no plans, I checked out the link.  The price (free) was right so I sent off a hopefully request for a seat.  What a great happenstance.  The projects that were presented blew my mind.  All because of a standard based on the basic sentence “I did this”?!?!?

Back in the late 1990’s  my boss at Universal Learning Technologies, Barb Ross, was on one of the workgroups developing the IMS (then version 0.4) standard for interoperable content cartridges and she involved me in her review of the early specification.  I sat in that room at Lurie’s thinking, “They’ve finally figured out how to do what Barb and I were wanting way back then.”

Since then, I’ve thrown myself into understanding this new specification.  I’ve attended the xAPI Camp at DevLearn in Las Vegas (where I ended up winning WatershedLRS’s xAPIgo challenge).  I’m completing HT2Lab’s Learning xAPI MOOC (both the technical and non-technical tracks).  I’ve even had the opportunity to have lunch with Aaron Silvers to learn from him directly.  I’ll be participating in Torrence Group’s Spring xAPI Cohort beginning on February 9.

Of course,  I’ve also combed the web and curated what I’ve found.  This Resource Center is the product of that curation.   These pages are living documents.   I’ll be adding and deleting resources.  Please provide your feedback via the thumbs up/thumbs down poll associated with each item.  Let me know what you’d like to know more about via the comments at the bottom of each page or directly to me on Twitter, the Contact page here, or email me directly if you have my email.

I see this Resource Center as the first step in an effort to help the everyday L&D professional understand the power and potential of xAPI to drive true learning analytics that cover a far broader swathe of learning experiences than we’ve dreamed possible in the past.  If implemented correctly, xAPI will enable us to analyze targeted behaviors, to create learning experiences to affect the desired changes, and to measure whether we have met the organizational goals we set.  So click on the image

So click on the image or link at the top of this post and start your journey in xAPI!

twitterI have created a Twitter List of people and organizations that tweet about xAPI.  Please follow it.  If there is someone or an organization that tweets regularly about xAPI, please send me your suggestions (Direct tweet me, use Contact page here at new eelearning, or email me if you have my email.)



Is there something you don’t understand about xAPI?  Questions about something said in one of the above resources? General thoughts on these resources?   Add a comment below.

If you have any ideas on resources you feel should be on this page or in this Resource Center, feel free to use the comment section below or contact me via the Contact page here at new eelearning.

xAPI Data Talks! Page Layout May Influence Interaction

In her post, Supporting Social Learning Through Page Design, on HT2Labs’ blog, Janet Laane-Effron talks about analysis she and her colleagues did on two of HT2Labs’ MOOCs.

The question is:

How can page design best support social learning?

The test:

Janet and her colleagues placed the comments section in one of their MOOCs below the content it was related to.  In another, the placed the comments section next to the content.

The result:

The two MOOCs had statistically the same number of total comments once moderators and other HT2Labs folks were removed from the data.  However, when they looked at whether the comments were original comments or replies to comments,  the MOOC with the comments section next to the content came out as the clear winner for interaction.  (The assumption here was that replies to a comment reflected interaction between participants.)

While Janet states in her post that this finding is not conclusive and there are other issues around UI and general layout for responsive design, it definitely suggests that there is more to consider on this question of the positioning of the comments section in relationship.

The xAPI win:

The only reason Janet and her colleagues were able to do this analysis was the MOOCs were created in Curatr, which creates xAPI statements.  In the xAPI standard for comments, original comments and replies to those comments generate statements with different verbs which can be sorted for.  In addition, the MOOC facilitators and other HT2Lab admins can be removed easily by sorting on the actors and the roles they have in the course.

Without xAPI, none of this data would have been created.  Sure, you could manually go in and created a data set my viewing each comment section and notating the comments in a spreadsheet.  But that would take far to long.

With xAPI, it would be very simple to expand this study to 10 or 100 MOOCs – if they are all set up in authoring systems that comply to xAPI.

Usage data on our learning designs can be at our fingertips with xAPI.

Bersin by Deloitte: 2017 – the Year Everything Goes Digital

Bersin by Deloitte has just released its Predictions for 2017: Everything Is Becoming Digital report.  Amongst the 11 predictions for HR there are specific predictions for Learning and Development.

2017 will be a tipping point for L&D and we will see dozens of world-class “digital learning” solutions all over the world.

Overall, the report says that, due to the rapid and non-stop changes in  technology and expectations from the business, Learn and Development has fallen behind and will continue to struggle in 2017.  The challenges L&D faces will be in a context of Massive HR transformation to a performance oriented, employee responsive digital culture that can address the impact of future work.

They predict that (comments in blue are mine):

  • Real-time feedback and analytics will explode in maturity.  While L&D isn’t mentioned specifically in this prediction, it does discuss HR as a whole.  With the advent of xAPI and cmi5 to better report today’s ongoing and often social learning focused learning experiences, we should be able to make a remarkable move forward in this area.
  • 2017 will see the launch of new and highly restructured LMS’s.  This will be refreshing for the companies in a position to adopt new LMS’s as the current LMS’s aren’t built to deal with the challenges that social and blended learning ecosystems create.  In addition, a myriad of new social tools are being launched and improved as well.  Add xAPI to the mix and L&D technology and its impact will begin a radical change in 2017.
  • Video and self-directed learning will become the dominant factors in organizational learning moving forward.  The shift to a micro-learning, “always-on” model of learning will accelerate in 2017.  Social Learning and learning eco-systems design will also contribute a transformation of the work of Instructional Designers and Facilitators.
  • L&D functions will need to be rethought and restructured.  Bersin by Deloitte points out that while L&D is aware of this change and is driving the discussion, two-thirds of all corporate learning organizations are structured with a centralized training function based upon old models.  L&D personnel and competencies need to be built into the business units to become more responsive.  New competencies, some relatively alien to many L&D professionals, will be incorporated into existing jobs  and new roles will be created.
  • L&D needs to take the lead on driving a culture change in organizations to an always-on learning focus.  They task us with showing managers that they need to drive learning because it is tied to the success of business.  We need to tie learning inextricably to performance , leadership skills, and organizational success and become marketing stars.  
  • “2017 will be a tipping point for L&D and we will see dozens of world-class “digital learning” solutions all over the world.”   2017 will make heroes out of agile learning leaders who can drive innovative and creative solutions that change the  course of organizational learning.

What do you think?  Is Bersin by Deloitte on the mark in their predictions?  is your L&D organization ready to lead this massive change?  Are you ready?

You can download this report on a complementary basis, for a limited time.

Towards Analytics Literacy


In his post, Analytics Literacy is a Major Limiter of Ed Tech Growth, Michael Feldstein argues that there is a lack of basic literacy in the Ed Tech and Learning and Development communities.  He points out that analytics is as much about storytelling and sensemaking as it is about data.  We intuitively create stories about the data we see, it is the core of hypothesis creation.  Through repetitive and progressive testing of hypotheses, we come to trust the story that algorithms tell.  This builds analytic literacy.

Feldstein provides several excellent examples of the role of analytic literacy in evaluating student performance based upon logins to software,  weather forecasts of precipitation, and the recent US Presidential election.  When we lack the literacy to get the story right, we start to distrust the data or the analytics, not our literacy.

If we, as a culture, lack the basic literacy to have clear intuitions about what “a 70% chance” means, then how likely is it that we won’t have shocks that cause us to distrust our learning analytics because we didn’t understand their assumptions and limitations?

He uses the medical community’s move to scientific methodologies a century ago as an example of the transformation that the learning community now needs to undertake regarding analytics and performance.  But he also points out that we need to avoid placing all our analytical trust in various technology tools.

Using a personal story about a medical issue, he discusses how dependence upon various diagnostic tools didn’t reveal the cause of back pain he was experiencing.  It was a doctor touching his back and feeling the benign fatty tumor that was present before he was correctly diagnosed.

He finishes by concluding that the “training of learning and development professionals needs to make a radical change to  transform our teaching culture into one of learning science and data science literacy.”  While not losing the intuition and observation skills that have propelled our field to date.

I’m not sure that I agree with Feldstein that incorporating learning analytics into our profession is going to take a radical transformation.  15-20 years ago, in the early days of elearning, there were predictions that the advent of online, digital tools would decimate the L&D profession.  Which, supposedly was so rooted in brick and mortar, face-to-face training delivery that it wouldn’t be able to adapt.

Well, time has shown the world what we already knew about ourselves, we will adapt in whatever way we need to achieve our goal of helping people learn.  In the case of elearning, not only did we adapt, we thrived.  We turned the change to our advantage to improve learning across the board.

Analytics, statistics, and Big Data are a bit of a foreign language for most L&D pros, but it fits well with our trained skills of needs analysis and evaluation.  We cherish any information that will enable us to design better learning experiences.  There will be an additional benefit in that we will be able to demonstrate our link to not only the success of the businesses we work within, but we will resonate more closely with the management culture of our organizations.   Our evidence of success will look like their evidence of success.  Learning analytics has the promise of finally putting us in a position of being peers among peers in our organizations.  That may well be the best carrot for L&D professionals in taking on this new challenge.