In August 2017, Skillsoft and Accenture kicked off a sponsored neuroscience research initiative with The Massachusetts Institute of Technology. The goal of this research is to scientifically validate which instructional design methodologies have the greatest impact on the effectiveness of video-based instruction.
This research is divided into three phases:
- Phase I – This involved the survey and evaluation of learners having consumed similar instructional content using six different instructional scenarios. The survey gathered learner reaction to the video, a brief personality profile (growth mindset, etc.) of the participants and a targeted knowledge assessment
- Phase II – The second phase includes brain scans of study participants as they engage with the sample learning content in the neuroscience lab at MIT
- Phase III – The final phase will involve brain scans of participants as they consume learning content amid the distractions in the workplace
During Phase I, the research team conducted a rigorous evaluation of six different video instruction strategies within the subject areas of both Business Skills and IT. The instructional strategies, or “interest factors,” included the following:
- Instructor-led Baseline – The baseline video, against which all others were measured, featured an instructor, on camera, delivering the material in lecture form.
- Scenario-based – Presenting the same video content in a new and innovative way. In this video, Skillsoft’s scenario-based, role-play video was used for testing purposes.
- Pre-Test – Starting the instructional video with a pre-test
- Interpolated Quiz – Interspersing quiz questions within the instructional video
- Course Outline – Starting the instructional video with an outline of the video course content
- Social Shares – Altering the number of “likes” and social shares of the video example to understand if peer recommendations stimulate interest in the video
In each case, one Business Skills training video (The Art & Science of Communication – introductory level content) and one IT video (T-SQL – content which requires some level of SQL competency) were used to study the effectiveness of the instruction. We completed Phase I of this research in the spring of 2018, which involved more than 700 Accenture employees (368 completions).
Phase I revealed several intriguing insights:
- For the business skills example, the instructor-led video treatment, the baseline, proved to be the least interesting approach. Instructional strategies, such as introducing an interactive outline, as well as presenting the content within a scenario-based format, materially improved the overall learner “interest.”
- For the business skills example, the presence of a growth mindset directly correlated to the expressed level of interest in the video and improved performance on the final assessment. Interestingly, this turned out to be more of an influencer than the selected instructional strategy. Of note, those participants who exhibited grit, also found the videos to be more interesting.
- For the IT example, there was no direct correlation between the instructional strategies and the learners expressed interest in the video. However, when looking at the subset of IT video participants who rated themselves as being less familiar with the content (1-3 out of 5), there was a positive correlation with their expressed level of interest and outcome on the final assessment.
- For the IT example, participants in the pre-test instructional strategy had significantly higher exit scores than participants in the baseline. Interest positively correlated with grit and growth mindset overall. Growth mindset positively correlated with exit scores in the course outline and interpolated quiz instructional strategies, but not with any other IT interventions. This again suggests that the presence of a growth mindset can influence the performance of the learner.
While these findings are intriguing, what initial practical perspectives might we draw from them?
- For business skills, the treatment and design of the video materially influence the expressed level of engagement by the learner. Simply delivering an instructor/expert lecture on camera, interspersed with bulleted concepts, is the least engaging way to present critical business skills content. The findings suggest that learners crave a storyline, a narrative and some degree of self-directed, progressive disclosure of the concepts. This finding informs design considerations for instructional content for the modern enterprise. While buyers are drawn to “instructor-driven” videos, they may compromise the engagement of the learning.
- For IT instruction, recommending the appropriate level of instruction is critical to overall cognitive engagement. This is a fairly intuitive concept. One would expect those who already know the subject matter of the video to find it less appealing than those who are filling in gaps in their knowledge. However, this finding may suggest something even more profound, which is that IT learners crave depth and richness of the instruction to fill self-perceived skill gaps. When considering the relative indifference to the instructional strategy, the finding suggests that substance and depth prevail. One might conclude, that IT audiences will value learning modalities beyond video alone, such as digital books, as a vehicle for depth of study.
- Regardless of the subject matter, a learner’s mindset (growth vs. fixed) is ultimately the greatest determinant of success. Again, this is a relatively intuitive concept. If a learner is motivated and believes they can learn, then they are more likely to learn. Much of Carol Dweck’s research centered on this very point. However, the more provocative question is whether a growth mindset is something that can be instilled at scale. Said differently, if leaders can effectively influence or change one’s mindset to turn their organizations into hyper-learning machines, then this may yield the most significant impact on learning effectiveness and return on investment.
As the study moves into the next two phases, we will seek to prove just that. Can we materially influence a learner’s mindset, so that they become better learners? More importantly, can we scientifically validate that their level of engagement is improved as evidenced by an EEG signature? Fully understanding this dynamic could lead to one of the most profound breakthroughs in digital learning design in recent years. In the coming months, Skillsoft and Accenture will be in the lab at MIT working to understand this. We have jointly developed a growth mindset intervention, which we will test in the lab.
Stay tuned for more insights over the coming months!
UPDATED BLOG POST: June 10, 2019: The Evidence for Scenario-Based Training for Interpersonal Skills – Skillsoft Completes its Research with MIT and Accenture
Mark Onisk is the Chief Content Officer for Skillsoft.