By Daniel Hickey and Chris Andrews
The previous post in this series explored LinkedIn endorsements and recommendations. Chris and Dan used those features to locate a potential consultant with particular skills and considered recent refinements to those features. We also explore the new LinkedIn Learning site made possible by the acquisition of Lynda.com. In this post, we explore how endorsements and recommendations might help LinkedIn earn back the roughly $300,000 that they paid for each of Lynda.com's 5000 courses.
With the acquisition of online training giant Lynda.com, LinkedIn moved beyond recognizing learning to actually supporting learning. One writer (a Forbes blogger who happens to be a Lynda author) called claims that LinkedIn’s $1.5B “is the best money it’ll ever spend.” Lynda.com’s high-quality training videos and courses can now be accessed at the new LinkedIn Learning site. LinkedIn launched this new site in September 2016 after adding practice activities and quizzes for most of the courses.
Apparently, some tech insiders were puzzled by the purchase. Many were surprised by the high price tag (Consider that Volvo just sold for $1.3B). According to one estimate, Lynda should have around 5000 courses. That works out to $300,000 per course! While Lynda’s videos are a lot better than typical, this still begs the question of how LinkedIn and its new owner Microsoft might make that money back. One writer pointed out that since many of Lynda’s most popular courses are for Microsoft products, it might be particularly useful for MS employees and customers. We checked and found that about 500 courses are tagged with the term “Microsoft.” While that is a lot of courses, it is only around 10% of the total. So we set out to explore the quality of these courses and the corresponding value of the eCredentials that are issued for completing them.
Is LinkedIn Learning Really Just “Another Netflix of Education”?
Phil Hill, an influential eLearning blogger published a commentary in the Chronicle of Higher Education arguing that LinkedIn’s purchase of Lynda.com “should give us pause.” On one hand, Hill commends LinkedIn for not embracing the MOOC-fantasy of displacing higher education. Hill also provides a nice summary of LinkedIn’s Economic Graph that aims to profile workers and workplaces in a manner that will generate high-priority competencies to be covered in new Lynda.com resources.
 |
| LinkedIn Economic Graph (image from Standup Strategy) |
Nonetheless, Hill extends the most common critique of MOOCs to Lynda.com and LinkedIn Learning. Pointing to the standard video-quiz model (and the absence of additional readings and interactivity), Hill argues that LinkedIn Learning is “still stuck on the idea that idealized chunks of content are equivalent to learning.” He elaborates
LinkedIn believes that one type of course…is what it needs to develop knowledge and skills. From a pedagogical standpoint, the LinkedIn Learning platform is a very limiting one…The company doesn’t explicitly make this claim, but LinkedIn Learning has the look of an effort to be another Netflix of education. If we can just create consumable content chunks and then apply data science to deliver the right chunk at the right time to the right person, the thinking goes, then we’ll achieve nirvana (emphasis added).
We are still not entirely clear about how this traditional “expository” model of instruction relates to Netflix. But this is precisely the sort of concern that we here at RMA love to explore. Most Learning Scientists assume merely exposing learners to specific procedural and factual information is insufficient for many training goals (and most educational goals). This concern is amplified when learners are then assessed with quizzes that only require recognition (rather than recall or use) of that information. But we wonder whether the combination of LinkedIn Learning and LinkedIn endorsements and recommendations might push individuals to learn more from these courses.
How Do Lynda.com Courses Work?
As an IU employee, Dan had occasionally turned to Lynda.com to learn how to use a new software program. So, he was disappointed when the contract was canceled in June 2016. To work on this series, Dan signed up for the free 30-day trial of LinkedIn Learning ($30 month for individuals—about the same as Lynda.com was for individuals). As a high school career and technical education teacher, Chris had regularly turned to Lynda to brush up on some of the technical skills that his students were learning, especially since the software he was teaching was constantly updated with new features.
While working on the previous post, Dan had searched for courses about e-Learning. This revealed 143 resources, including 88 courses and 3 multi-course “Learning Paths.” This abundance of high-quality resources is what made Lynda.com so popular with companies and individuals needing training towards very specific competencies.
 |
| LinkedIn Learning’s Resources for E-Learning Skills |
The Become an Online Instructor pathway consists of 11 courses that include a total 32 hours of video. All the courses in this learning pathway are broken up into chapters consisting of brief clips. Most (but not all) of the courses include activity files and/or quizzes for each chapter. For example, the course for Camtasia, a screen capture program, consists of over six hours of video broken into eleven chapters. Each chapter concludes with a 2-3 question quiz. The quizzes are quite low stakes but cleverly designed. Clicking on a wrong answer immediately displays the 1-2 minute clip of the video that contains the correct answer. While learners can click through to guess the correct answer, the formative feedback clips are very brief and the questions are quite easy. It seems likely that most learners will watch the clips. Such quizzes are no substitute for a secure proctored exam completed from memory. Even a time-limited test that included some challenging "best answer" items that can't be quickly located on the Internet. Such tests could provide evidence needed to grant more formal recognition for the learning in these courses.
 |
| Example Quiz Item from LinkedIn Learning |
Apparently, after purchasing Lynda.com. LinkedIn quickly added quizzes to many existing courses and stepped up the pace of adding new courses. It seems possibly that they may continue to refine the quizzes. But the most important changes are likely the ones that many users will never even be aware of
How LinkedIn Adds Value to Lynda.com Courses
How LinkedIn Adds Value to Lynda.com Courses
While LinkedIn Learning and Lynda.com appear quite similar, the course descriptions hint at one of the new features that add value to the videos. As shown in the course descriptions below, the new site now shows (a) how many people have watched the course, (b) what those people do, and (c) where those people work. This is just a fraction of the data that LinkedIn Learning is apparently already able to access from the new Economic Graph. Readers may have noticed recently that their LinkedIn homepage now makes recommendations for courses based on their skill profiles and endorsements.
 |
| LinkedIn Learning’s Camtasia 8 Essential Studio Course |
binding your accounts allows your data to be used to personalize your experience across LinkedIn and Lynda. For example, your LinkedIn profile may be used to power suggestions on Lynda.com courses, and Lynda.com coursework may be used to power job suggestions on LinkedIn.
 |
| The LinkedIn Placements Website for Testing Users and Linking them to Employers |
Given LinkedIn’s ability to continually refine its platform and extend its reach, these features might create entirely new synergies. Consider, as a blogger at TechCruch has, that LinkedIn is piloting a LinkedIn Placement, and job placement service in India that tests skills and suggest jobs. It seems likely that they will soon connect such a testing system with LinkedIn Learning. However, at least part of this potential rests on the validity of the eCredentials associated with these courses or tests. And that depends in large part on the assessments and tests used in these courses and websites.
How Valid is the Evidence from Online Open Courses?
Of course, the concern here is with validity. As we are exploring in another series of posts here at RMA, the broad move to online learning and the explosion of open learning resources like Lynda.com, MOOCs, and Khan Academy has created a big hot mess of validity questions and concerns. As a reminder, validity concerns the extent to which evidence (typically from assessments) supports claims. Whereas reliability is a property of assessments, validity only concerns the claims (there is no such thing as a “valid assessment”).
One big issue with the online quizzes and tests is that that they usually need to be machine scored. In most cases, this requires using multiple-choice or true-false items, which limits the competencies and types of knowledge that can be assessed. In some domains (particularly mathematics and coding), it is relatively easy to machine score numerical and short answer responses. And with enough resources, new approaches to measurement and digital technology makes it possible to create some very sophisticated tests that are machine scoreable.
However, even though a test is sophisticated, it may or may not provide valid evidence of claimed competencies. As for LinkedIn’s Placements service, it currently only tests skills for software jobs. The site relies on HackerRank.com and its sophisticated suite of practice problems and tests. It is beyond our scope and expertise to investigate the extent to which HackerRank can predict an individual’s ability to code on the job. While Venture capitalists and the US Department of Education’s EQUIP program appear ready to give HackerRank a chance, some writers have been quite critical of the quality of the assessments, while other writers explain how to use Google to cheat on the tests.
Of course, there is a big difference between the credentials issued by HackerRank and the certificates of completion issued by LinkedIn Learning. Technically speaking, the validity of the certificates only concerns the claim that the holder “completed” the course. Presumably that means that they watched the videos, completed the exercise and took the quizzes. The obvious issue with LinkedIn Learning courses is that you don’t actually have to pass the courses. But we are getting ahead of ourselves. A recent scandal around MOOCs raised a bigger issue with completion certificates.
One problem with the “online video-then-quiz” format is that people can log in a second time, and use the “guess and check” strategy as one user to answer the items while using the other login to input only correct answers. A 2015 study found this practice to be so widespread in EdX courses that the authors gave it a name: Copying Answers using Multiple Existences Online (CAMEO). The study was widely reported and seemed to undermine the credibility of EdX certificates of completion. There was some discussion of taking steps such as revoking certificates and making changes to the platform to prevent the practice. However, we could find no evidence that these changes are being undertaken, as “such changes could entail significant pedagogical tradeoffs for instructors.”
One of the important points of the CAMEO study is that it highlighted the way that new forms of learning generate new kinds of problems. In an article about the CAMEO paper at MIT, renowned MOOC innovator George Siemens argues that
This paper addresses a growing challenge as higher education digitizes: ensuring that the work that students do is their work. MOOCs, and in the case of this paper, the development of multiple online identities, present a new set of challenges for academics. The stakes in learning can be high since academic performance can influence job opportunities. Ensuring that work is done by the student proclaiming to have done it will continue to be a challenge online and in classrooms.
Of course, technology innovators are hard at work searching for new solutions such as online proctoring and simplified identity verification schemes. But the fact remains that online courses, and particularly open online courses are now hard pressed to provide convincing evidence.
How Valid are Lynda.com Certificates of Completion?
Illustrating Phil Hill’s point above, Lynda.com is careful to not overstate the value of their certificates. One FAQ states that the certificates are “not endorsed or recognized by any third parties. Another FAQ states
The Lynda.com Certificates of completion program is not the same as a degree program or a software certification program, and it is very important that it is not confused with such. For example, training programs that software and hardware companies have for their own software are not the same as the Lynda.com program. Completing a course does not mean that the member is then certified in that respective software, it only reflects that they completed the course on Lynda.com.
The first question this raises is whether it is possible to earn Lynda.com/LinkedIn Learning completion certificates without actually completing the courses. Because you don’t get penalized for wrong quiz answers, there is little incentive to bother with the CAMEO strategy. So, Dan accessed the Camtasia Essentials course and clicked through all the quizzes quickly, just guessing until he got the items correct. He was able to do so in about five minutes. But the course did not generate a certificate of completion. So we went digging.
We found a Lynda.com FAQ under Certificates of Completion state that the Lynda.com system is able to track when a member watches and finishes a movie in a title. Apparently Dan’s failure to watch all of the videos precluded getting a certificate. While there does not appear to be any way of verifying whether an individual completed any exercises, our initial investigation suggests that the LinkedIn Learning’s certificates of completion may well be valid evidence that the earner really did play the videos and then complete the quizzes in the course.
Of course somebody could just let the videos play in real time in the background and then click through the quizzes as Dan did above. But given the brief videos, easy quizzes, and the wrong-answer feedback, one might as well just complete the course. We are currently trying to track down a 2015 study of this issue in vocational classes published by Jamie Clark in the UK. But for now we suspect that Lynda.com certificates are reasonably valid evidence that someone at least watched the videos.
Thought Experiment: Can LinkedIn Endorsements Enhance the Validity of Lynda.com Certificates?
The previous post explored how LinkedIn makes it simple to endorse your connections for the skills featured on their profile. The post also explored LinkedIn’s efforts to increase the validity of those endorsements. This raises the question of whether endorsements might enhance the value of completion certificates. They might do so by providing valid evidence of the particular skills covered in those courses. This seems like an eCredentialing question worthy of serious study, and perhaps a dissertation. For now, we are going to do a thought experiment.
Let us imagine that Chris wanted to supplement his graduate assistantship by designing and teaching online courses starting next summer. With an undergraduate degree, MA, and extensive experience in vocational, technical, and arts education, there are probably plenty of community colleges and high schools out there where his teaching credential and experience could be valuable. And some of those institutions are probably already using LinkedIn Recruiter to find potential instructors.
The problem is that Chris has never taught online. Not surprisingly, his LinkedIn Profile does not include Distance Education or e-Learning as skills. We do see, however, that Chris very recently completed the introductory course on SCORM and Tin Can API from Lynda.com. These are both important for designing an online course. But that is just a one-hour course (that he completed as part of the research for this post).
 |
| Chris’ Current LinkedIn Skills, Endorsements, and Certifications |
It seems quite plausible that Chris could choose to complete the Become an Online Instructor Learning Path that we explored above. The path consists of eleven courses which contain a total of 32 hours of video. While it would be a tough semester, he could probably manage one course a week and earn all of the eleven certifications. If so, he should be able to claim skills on his profile that would appeal to recruiters and online schools. These include skills with specific online teaching tools like Lectora, Camtasia, and Captivate, knowledge of issues like Accessibility, and broader skills such as Distance Education and e-Learning. This sort of part-time “skilling up” to advance or expand a career is a pretty good example of the sort of alternative certification that was explored in the second post in this series.
But if Chris’ new skills are to be valued by recruiters, they will need to be endorsed. As we learned in the previous post, what he really needs are endorsements from people who (a) are also endorsed for those skills and (b) have worked or currently work with Chris. This is because LinkedIn Recruiter emphasizes potential applicants whose sought-after skills have more such endorsements. Even outside of using Recruiter, endorsements are more likely to be valued when they are awarded by individuals who are more connected to the viewers (as was illustrated in Dan’s location of a potential consultant in the previous post).
After completing the Learning Path, Chris would want to quickly and deliberately solicit his past and current colleagues who are endorsed for the new skills to endorse him for those same skills. This part of our thought experiment lets us examine why these endorsements are more valuable than other endorsements:
- As we illustrated in the first post, experts in specific skills are able to readily (and often quite accurately) assess that expertise in others. In particular, they can ask a few specific questions that quickly detect truly fraudulent claims of skill.
- Experts are more likely to be familiar with relevant educational resources. A connection with a widely endorsed skill on LinkedIn is especially likely to be familiar with the quality of relevant LinkedIn resources. For example, they might believe that a specific course covers a claimed skill quite poorly and decline to endorse a skill on that basis.
- Experts are familiar with what kind and level of training a particular skill requires. For, example, perhaps one of Chris’ colleagues is particularly skilled (and endorsed) for SCORM. SCORM (Shareable Content Object Reference Model) is a complicated set of e-learning standards that is not very meaningful without extensive contextualized practice. That expert might not be willing to endorse Chris’ SCORM skills based on a one-hour course. But that same expert might be willing to endorse Chris’ skill for Camtasia (a more discrete skill) for completing the six-hour Essential Skills course.
- All of these characteristics of expert endorsements are heightened when they occur between colleagues and close connections. This is because the relationship increases the likelihood that the endorsement will be scrutinized by knowledgeable “third-party” peers. Furthermore, experts are less likely to risk undermining their credibility in settings among colleagues that give their credibility its value.
Certainly many readers will question whether LinkedIn endorsements are a sign of true expertise as defined by Anders Ericsson and popularized by Malcom Gladwell. But that misses the point. The algorithms in LinkedIn Recruiter only care about relative expertise. This is because the queries push individuals with more valued endorsements to the top of the search. This means that they necessarily push down individuals with less valued endorsements. As this reality become more widely known, LinkedIn users who care about their rankings will get better at gaining valued endorsements and may even stop accepting some endorsements. We will explore additional nuances for endorsements in a subsequent post. But for now, our thought experiment leads us to conclude that endorsements can indeed increase the value of certificates for completing Lynda.com courses in LinkedIn Learning—but only certain kinds of endorsements.
Can LinkedIn Recommendations Enhance the Value of Certificates and Endorsements?
As we explored in the previous post, recommendations in LinkedIn function somewhat different than endorsements. While the interface that LinkedIn provides makes it a lot easier to provide recommendations rather than providing a formal letter of recommendation, it is still a lot of work.
 |
| Chris’ Current LinkedIn Recommendations |
Right now, Chris only has two recommendations on his profile, and both are from working in the BYU library. Certainly, Chris would want to work to earn recommendations from an administrator, colleague, and perhaps even some students about the quality of his online instruction. But he would be quite hard pressed to do so until he starts teaching.
In the meantime, Chris would want to ask his colleagues and students from his seven years as a high school teacher and program leader for recommendations. That process would be similar to gaining endorsements, but more time consuming and personal. However, the process would be much more efficient and transparent than requesting actual letters of recommendation. While one or two actual letters might help, it seems that a handful of plausible recommendations on his profile might be very helpful.
Once Chris was actually working, designing, and teaching online courses, he would likely be in a position to start gaining recommendations associated with that role. Such recommendations would be particularly helpful when they provide additional support for the skills and certifications that are most valued by prospective employers. While it is unclear whether LinkedIn’s search algorithms consider recommendations, it seems that recommendations might be the key to getting an interview once a recruiter or employer finds Chris’ profile. And then the same kinds of value and validity processes described above for endorsements are likely to apply to recommendations, but in a more personalized and individualized manner.
Conclusions about LinkedIn as an eCredential and Next Steps
In conclusion, yes it seems clear that LinkedIn certifications, endorsements, and recommendations can all provide valid evidence of valuable skills. But it seems pretty clear certifications are only valuable with valued endorsements, and that valued endorsements are more valuable when they are supported by valued recommendations. This phenomenon is actually a rather messy version of convergent validity in classical testing theory, where one examines the correlation between two different tests that purport to measure the same thing.
Our next post in the series will explore some rather complex but crucial phenomena that this post raised. The first is the way that some of the features described in this post should minimize the blanket distribution of trivial endorsements. This phenomenon (dubbed “carpetbadging” by Kyle Bowen in 2013) is currently one of the biggest challenges facing the open badges eCredentialing community.
The second phenomenon we will explore is the anthropological notion of “prolepsis.” This refers to the way that anticipation of events in the future shapes activity in the present. If Chris were to actually go down this path, it is certain the manner in which he completes his certification courses and then approaches his first online teaching job will all be carried out with an eye towards gaining more valued endorsements and recommendations. We will argue that the phenomenon is central to the development of true expertise.
In exploring these two phenomena, we will also explore the notion of productive disciplinary engagement as introduced by the late Randi Engle in 2002. This concerns the manner in which more static disciplinary knowledge (i.e., abstract concepts and specific facts and skills) of the certification courses would be made meaningful by enlisting them in more specific disciplinary practices. We will argue that a wealth of experience doing so is essential for true expertise, and should be central to any educational endeavor.
Posts
No comments:
Post a Comment