Supported by:
September 2018 // Volume 56 // Number 5 // Feature // 5FEA5
Adaptive Learning: An Innovative Method for Online Teaching and Learning
Abstract
The term adaptive learning refers to a nonlinear approach to online instruction that adjusts to a student's needs as the student progresses through course content, resulting in a customized experience for the learner based on prior knowledge. This concept is emerging in the field of online learning. Through a project funded by the eXtension Foundation, we reviewed and conducted pilot testing on adaptive learning tools for Extension programming. We found that the adaptive learning format aided learners in mastering content. A significant contribution to the Extension community resulting from our project is improved understanding of an innovative way of teaching online.
The term adaptive learning refers to a nonlinear approach to online instruction that adjusts to a student's needs as the student progresses through course content, resulting in a customized experience for the learner based on prior knowledge. This concept is emerging in the field of online learning. Through a project funded by the eXtension Foundation, we reviewed and conducted pilot testing on adaptive learning tools for Extension programming. We found that the adaptive learning format aided learners in mastering content. A significant contribution to the Extension community resulting from our project is improved understanding of an innovative way of teaching online.
Introduction and Background
Adaptive Learning in Higher Education
The term adaptive learning refers to a method of online instruction that involves providing personalized learning experiences resulting from a data-driven approach to curriculum design. This method builds on customized learning paths derived from algorithms, course analytics, assessment data, and feedback from students and can be used both to provide remediation and to help learners achieve mastery of content (Educause, 2017). In contrast, a standard course is typically more linear and does not include customized learning paths for students. Support for adaptive learning has been seen recently across higher education. According to a survey of 338 chief information officers and senior campus information technology (IT) officials, adaptive learning technologies have great potential for improving student outcomes (Green, 2016). Additionally, the fact that eight universities received funding from the Association of Public and Land-grant Universities to develop projects for the Accelerating Adoption of Adaptive Courseware Initiative (Adams Becker et al., 2017) further indicates support for the use of adaptive learning methods in higher education.
Students and faculty have reported both advantages and disadvantages of adaptive learning. Adaptive learning allows students to apply prior knowledge, gain information they need to know more quickly, and create a customized learning plan. This method can ensure that students reach mastery before moving on and helps educators avoid "teaching to the middle," a strategy that fails to account for advanced students or those who are lagging (Educause, 2017). In an adaptive learning pilot project at the University of Central Florida, students indicated that they spent more time on adaptive coursework as compared to work for a traditional course, but they overwhelmingly suggested that they would take another adaptive course and that they spent their time valuably by participating in the adaptive course (Freda, 2016). There are advantages for instructors as well, as they can more easily monitor which students need assistance, measure curriculum performance, and maximize learning outcomes, in addition to having a better sense of content areas in which students are struggling (Educause, 2017). Challenges to adaptive learning include high costs of implementation, inadequate support, need for training, and inability to manage extensive amounts of data (Freda, 2016). Implementing adaptive learning also requires time for creating detailed curriculum mapping and content that supports learning objectives (Educause, 2017).
Adaptive Learning in Extension
As technology improves and is available to more target audiences, Extension must focus on reaching out to clients in new and creative ways. According to the 2017 Pew Research Internet/Broadband Fact Sheet, roughly nine out of 10 American adults use the Internet (Pew Research Center, 2017). Another Pew Research study of lifelong learners and technology indicated that 73% of respondents identified themselves as lifelong learners and that those who had access to technology tools were more likely to rely on the Internet to access learning resources (Horrigan, 2016). Moving traditional Extension face-to-face programming to an online format is a method Extension educators often use to reduce expenses and reach audiences who may not have been served previously. In taking this approach, educators must be responsive to the needs and preferences of online learners and offer high-quality, relevant content.
For over 100 years, Michigan State University (MSU) Extension has played an important role in improving the lives of residents of Michigan and beyond through the dissemination of quality science-based educational materials. To meet the needs of communities and stakeholders, MSU Extension offers many programs online through mediums such as web-based articles, webinars, online courses, and mobile apps. In 2013, more than 300,000 people participated in MSU Extension programming, and over 1 million people viewed nearly 3 million pages on the MSU Extension website (Michigan State University Extension, 2014). It is imperative to continuously respond to the changing needs of our audiences and explore new ways to disseminate science-based information.
Our team of MSU Extension professionals received a grant from the eXtension Foundation in 2016 to explore the use of adaptive learning in Extension. The project connected adult learning theory, adaptive learning concepts, and MSU Extension online courses. Adult learning principles relate to the learner's need to know, self-directedness, prior experience, a readiness to learn, problem-centered learning, and the motivation to learn (Knowles, Holton, & Swanson, 2005). Adaptive learning models reflect these principles in that the learner is given a personalized learning path based on previous knowledge. According to a case study by Tyton Partners (n.d.), application of adaptive learning can make significant contributions to improving retention among students, measuring student learning, aiding the achievement of better outcomes, and improving pedagogy, outcomes that also are focus areas for those creating online courses in Extension.
The overall goal of our funded project was to evaluate the effectiveness of adaptive learning tools for Extension programming. Specifically, we sought to answer the following research question: Is adaptive learning effective for outreach and education in Extension?
Methods
We explored a variety of adaptive learning software tools and identified which tool was best suited for a selected pilot project based on an existing MSU Extension online course. We then invited past participants to take the course a second time using an adaptive learning method.
Constructivism is a theory regarding the social construction of knowledge. According to this theory, learners construct knowledge by applying active learning techniques as well as prior knowledge, beliefs, and experiences (Stravredes, 2011). The application of adaptive learning techniques is similar in that learners gain access to content they need to know on the basis of knowledge they already have attained through formal and informal learning. For our project, we chose to test the effectiveness of adaptive learning by using the existing noncredit online course Purchasing and Owning a Horse 101.
Purchasing and Owning a Horse 101 is an online course offered through the My Horse University program, which is based out of MSU Extension. My Horse University offers noncredit online courses and digital resources to horse owners and enthusiasts, with topics ranging from nutrition and health to behavior and horse ownership. We chose Purchasing and Owning a Horse 101 for our adaptive learning project because it had a high number of past participants, included a variety of content based on learning objectives, had existing assessments, and was built in the D2L learning management system, which MSU uses for credit and noncredit online courses.
The adaptive learning tool we chose for the project is Brightspace LeaP. This tool is part of the Brightspace suite of products, which also comprises the D2L learning management system. Brightspace LeaP was a logical choice because the Purchasing and Owning a Horse 101 course already was housed in D2L and the MSU IT Services Department was interested in pilot testing Brightspace LeaP to determine whether it would be effective for other courses on campus.
Brightspace LeaP is used to build personalized learning paths that can help course participants gain access to content in areas in which they are deficient or prepare them for future assessments. Participants in our project initially were given a pretest. On the basis of the questions they answered correctly and incorrectly, they were presented with their own personalized learning paths containing content items relating to areas in which they had failed to show mastery on the pretest. Each content item also included optional practice questions. Once a participant had gone through the content in his or her learning path, the participant was presented with a posttest. Participants could repeat the process and retake the posttest as often as desired.
We derived the sample for the project from past participants of the Purchasing and Owning a Horse 101 course. As an incentive, those who completed our adapted version of the course were given access to a premier online course from My Horse University (a $200 value) for free. To complete the adapted course, participants had to (a) submit the pretest, (b) access the presented content, (c) submit the posttest, (d) complete a survey to evaluate the course and tool, and (e) post feedback in an online discussion forum.
We analyzed data from the Brightspace LeaP adaptive learning course quantitatively and qualitatively. We used Stata, Version 13.0, to run descriptive statistics and conduct a paired t-test on the pretest and posttest scores from the course. Qualitative data analysis includes the following systematic actions: (a) affixing codes to transcriptions, (b) sorting through transcriptions to identify similar themes, (c) taking note of thematic patterns, (d) trying coding schemes, (e) developing a set of generalizations that cover consistencies, and (f) confronting these generalizations with a formalized body of knowledge based on theory (Miles & Huberman, 1994). We coded open-ended question responses into open, axial, and selective codes according to the constructivism theory. Data that were not pervasive in the findings were not included in the coding process.
Results
The Purchasing and Owning a Horse 101 online course had been offered for free before it was adapted. Of those who had completed the course, 84% were female and 70% had taken some college courses or received a bachelor's degree or higher. For 47% of the participants, Purchasing and Owning a Horse 101 was the first online course they had taken, whereas 53% had taken one or more online courses previously. As we have noted, we asked past participants of the Purchasing and Owning a Horse 101 course to be a part of our adaptive learning project. A total of 1,839 past enrollees of the course were invited to participate, and 192 people registered for the course. Of those who accessed the course, 85 completed the pretest, and 50 attempted the posttest. The posttest was not required and could be retaken. The evaluative survey was attempted 95 times by a total of 75 individual adults. Some individuals completed the survey more than once.
Participants' scores on the pretest and posttest provided information about their changes in knowledge. Participants' average scores were 85.50 on the pretest and 90.81 on the posttest. In looking specifically at the mean scores, we found that the posttest had a higher average as compared to the pretest. On the basis of paired t-test results, we determined that there was a difference between the pretest and posttest mean scores of 5.31, with a standard deviation of 13.97, and that the difference in the scores was statistically significant (p < .0001). It can be concluded that there was a significant difference between the pretest and posttest averages for the adaptive learning course.
Through our analysis of participant responses to open-ended items on the evaluative survey, we found a common theme in that technical problems were minimal for most users but that mobile accessibility would increase both the effectiveness of the experience and users' ability to participate in adaptive learning courses. Another theme we identified was that adaptive learning allows for a more enjoyable, flexible, and specific learning experience in which users can apply prior knowledge and experiences. Table 1 showcases the qualitative common themes and codes we applied to the open-ended data we collected. Examples of specific quotations from users reinforcing the codes are "Saves time in studying and provides a good focal point," "Short lessons make it a lot easier to study on the go," and "Easy to follow and navigate."
| Item topic | Open codes | Axial codes | Selective codes |
| Technical problems | Not mobile friendly Some text cut off One of the videos did not work No problems Directions to access the site were vague | Most users did not experience technical problems except for encountering a video that did not work and finding directions to be unclear. Adding a mobile access option would improve the course. | Technical problems for users throughout the course were minimal, although mobile accessibility would increase the effectiveness of adaptive learning. |
| Open feedback | Not clear how adaptive learning works Enjoyable Saves time in studying, provides a focal point Short lessons make it easier to study on the go Course was run and written well Objective and specific Basic information but brief and palatable Easy to follow and navigate Fairly easy and convenient Concise and easy to navigate | Adaptive learning modules provided a clear, concise, and specific learning option that saves time and is enjoyable to use. | Adaptive learning allows for a more enjoyable, flexible, and specific learning experience in which users can apply prior knowledge and experiences. |
The evaluative survey also included questions regarding the effectiveness of the adaptive learning method and the functionality of the Brightspace LeaP tool. Of those who completed the survey, 96% indicated that they would take more courses featuring the adaptive learning format. Table 2 provides additional information about participants' responses. Overall participants' responses were positive, with 83% either agreeing or strongly agreeing that the adaptive learning method was helpful in mastering course content, 60% agreeing or strongly agreeing that navigating the adaptive learning tool was easy, and 63% agreeing or strongly agreeing that they preferred the adaptive learning method over a traditional online self-paced course.
| Item | Strongly disagree (%) | Disagree(%) | Neutral(%) | Agree(%) | Strongly agree (%) | NA(%) |
| The adaptive learning tool was helpful in mastering the content of this course. | 5.26 | 5.26 | 5.26 | 37.89 | 45.26 | 1.05 |
| It was easy to navigate through the adaptive learning tool. | 6.23 | 7.37 | 26.32 | 23.16 | 36.84 | 0.00 |
| I preferred this method of learning compared to a traditional self-paced course. | 4.26 | 9.57 | 21.28 | 34.04 | 28.72 | 2.13 |
Participants also were asked to respond to items regarding other aspects of the adaptive learning tool. Results are presented in Table 3. Again, participants' responses were positive, with 81% agreeing or strongly agreeing that the practice tests were effective, 89% agreeing or strongly agreeing that the recommended readings were valuable, and 69% agreeing or strongly agreeing that information about access to the personalized path, its objectives, and its overall use in the course was communicated clearly.
| Item | Strongly disagree(%) | Disagree(%) | Neutral(%) | Agree(%) | Strongly agree(%) |
| Practice tests were effective in allowing review of the material presented. | 4.21 | 2.11 | 12.63 | 48.42 | 32.63 |
| Recommended readings were valuable for further remediation. | 4.21 | 1.05 | 5.26 | 54.74 | 34.74 |
| Access to the personalized learning path, its objectives, and the overall use in this course was clear and well communicated to me. | 4.26 | 10.64 | 15.96 | 37.23 | 31.91 |
Conclusion
Adaptive learning has been a focus area within higher education as a personalized learning method for improving student outcomes. However, there has been little focus on its use by community-based organizations such as Extension. Feedback from participants in our project indicated that the adaptive learning course was effective for allowing participants to master specific content. A majority of the participants also indicated that they would take additional courses in an adaptive learning format. This finding aligns with feedback from students at the University of Central Florida, who also indicated that they would take additional online courses in an adaptive learning format (Freda, 2016).
Future program delivery will include the use of tools and best practices for creating customized learning paths for online course participants. Pew Research results from 2016 showed that among adults with Internet access, 82% had done some personal learning activity online in the preceding year (Horrigan, 2016). This number is set to increase in the future, suggesting that the demand for online learning also will increase. Additionally, 61% of adults in the 2016 Pew Research study had little or no awareness of distance learning (Horrigan, 2016). As with other technology avenues, increased awareness will increase the need and desire for adaptive learning. Three years ago, New Media Consortium's NMC Horizon Report: 2015 Higher Education Edition indicated that adaptive learning technologies were 4 years away from widespread use and that the next step was to develop best practices (Johnson, Adams Becker, Estrada, & Freeman, 2015).
Implementing adaptive learning requires aligning learning outcomes with assessments and building content to ensure that users are provided with the required knowledge to achieve the given outcomes. This process requires an instructor to follow sound instructional design practices and is characterized through learning models such as backward design, an approach to designing learning experiences that align with intended goals and outcomes. Backward design was introduced by Grant Wiggins and Jay McTighe in 1998 (Wiggins & McTighe, 1998). Furthermore, an Extension study of the 4-H web-based course Teens Reaching Youth Through Innovative Teams (TRY-IT!) showed that effective course design could help educators reach and teach Extension clientele through interactive web-based e-learning (Edwards, Flood, & Safrit, 2004). This study of the TRY-IT! course supports the idea that implementation of adaptive learning, or development of any web-based course, requires sound instructional design practices.
Across the tools and methods we tested and deployed for our study, numerous strengths emerged, with Brightspace LeaP shining the brightest. The significant difference we found in the pretest and posttest average scores emphasizes the effectiveness of this tool. Future research endeavors may include investigation of cost-effective adaptive learning tools or methods that could be used in Extension and exploration of the effectiveness of adaptive learning methods within other disciplines across Extension.
Our project was funded by the eXtension Foundation. Additional collaborators include Ellen Darnall, former Michigan State University Extension staff member; Christine Skelly, equine Extension specialist and founder of My Horse University; Michigan State University Information Technology Scholarly Services team members; and Karen Vignare, executive director of personalized learning for the Association of Public and Land-grant Universities.
Recommendations for Further Reading
The following resources provide more information for individuals interested in developing adaptive learning courses:
- Personalized Learning for Adult Learners—Project Overview, http://www.canr.msu.edu/od/educational-technology/personalized-learning;
- Learning Objective Outline for Designing Online Courses, http://www.canr.msu.edu/od/educational-technology/learning-objective-outline;
- Articulate Storyline Adaptive Example, https://tinyurl.com/adaptive-example; and
- Articulate Storyline Adaptive Template (opens only in Articulate Storyline), https://tinyurl.com/adaptive-template.
References
Adams Becker, S., Cummins, M., Davis, A., Freeman, A., Hall Giesinger, C., & Ananthanarayanan, V. (2017). NMC horizon report: 2017 higher education edition. Austin, TX: The New Media Consortium.
Educause. (2017). 7 things you should know about adaptive learning. Retrieved from https://library.educause.edu/~/media/files/library/2017/1/eli7140.pdf
Freda, B. (2016, August 26). Clearing the hurdles to adaptive learning: Overcoming six challenges encountered in adopting adaptive learning. Retrieved from https://www.universitybusiness.com/article/adaptive-learning-clearing-hurdles-higher-ed
Green, K. (2016). The 2016 national survey of elearning and information technology in US higher education. Retrieved from https://kenneth-green-pln7.squarespace.com/content/2016/11/21/the-2016-campus-computing-survey
Horrigan, J. B. (2016). Lifelong learning and technology. Retrieved from http://www.pewinternet.org/2016/03/22/lifelong-learning-and-technology/
Johnson, L., Adams Becker, S., Estrada, V., & Freeman, A. (2015). NMC horizon report: 2015 higher education edition. Austin, TX: The New Media Consortium.
Knowles, M. S., Holton, E. F., & Swanson, R. A. (2005). The adult learner: The definitive classic in adult education and human resource development (6th ed.). Burlington, MA: Elsevier.
Michigan State University Extension. (2014). Michigan State University Extension catalog of programs and services. Retrieved from http://msue.anr.msu.edu/uploads/236/59719/MSU-Extension-catalog-FULL.pdf
Miles, M. B., & Huberman, A. M. (1994). Qualitative data analysis (2nd ed.). Thousand Oaks, CA: Sage Publications.
Pew Research Center. (2017). Internet/broadband fact sheet. Retrieved from http://www.pewinternet.org/fact-sheet/internet-broadband/
Safrit, R. D., Edwards, H. C., & Flood, R. W. (2004). An initial assessment of an interactive web-based Extension curriculum to engage and prepare teens as volunteer teachers. Journal of Extension, 42(3), Article 3RIB1. Available at: https://www.joe.org/joe/2004june/rb1.php
Stravredes, T. (2011). Effective online teaching. San Francisco, CA: Jossey-Bass.
Tyton Partners. (n.d.). Learning to adapt: A case for accelerating adaptive learning in higher education [White paper]. Retrieved from http://tytonpartners.com/tyton-wp/wp-content/uploads/2015/01/Learning-to-Adapt_Case-for-Accelerating-AL-in-Higher-Ed.pdf
The Journal of Extension
The Journal of Extension (JOE) is the official refereed journal for Extension professionals. JOE expands and updates the research and knowledge base for Extension professionals and other adult educators to improve their effectiveness. In addition, JOE serves as a forum for emerging and contemporary issues affecting Extension education.
Published bimonthly online by Extension Journal, Inc.
© 1962-2018 Copyright by Extension Journal, Inc. ISSN 1077-5315.
Skip to contentSkip to main menu
Making Multiple-Choice Exams Better
The relatively new Scholarship of Teaching and Learning in Psychology journal has a great feature called a “Teacher-Ready Research Review.” The examples I’ve read so far are well organized, clearly written, full of practical implications, and well referenced. This one on multiple-choice tests (mostly the questions on those tests) is no exception. Given our strong reliance on this test type, a regular review of common practices in light of research is warranted.
This 12-page review covers every aspect of multiple-choice exams, at least every aspect I could think of. What follows here are bits and pieces culled from the review. For teachers serious about ensuring whether their multiple-choice exams (both those administered in class and online) assess student knowledge in the best possible ways, this article should be kept on file or in the cloud.
Perhaps the most important ongoing concern about multiple-choice tests is their propensity to measure surface knowledge, those facts and details that can be memorized without much (or any) understanding of their meaning or significance. This article documents studies showing that students’ preference for multiple-choice exams derives from their perception that these exams are easier. Moreover, that perception results in students’ using studying strategies associated with superficial learning: flashcards with a term on one side and the definition on the back, reviewing notes by recopying them, and so on. Students also prefer multiple-choice questions because they allow guessing. If there are four answer options and two of them can be ruled out, there’s a 50 percent chance the student will get the answer right. So students get credit for answers they didn’t know, leaving the teacher to wonder how many right answers indicate knowledge and understanding the student does not have.
In one of the article’s best sections, the authors share a number of strategies teachers can use to make multiple-choice questions more about thinking and less about memorizing. They start with the simplest question. If the directions spell out that students should select “the best answer,” “the main reason” or the “most likely” solution, that means some of the answer options can be correct but not as correct as the right answer, which means that those questions require more and deeper thinking.
Another strategy that promotes more thinking includes a confidence level indicator given along with the right answer. The greater the certainty that the answer is correct, the higher the confidence level. When scoring, a right answer and high confidence level are worth more than a right answer and a low confidence level.
Perhaps the most interesting way to make students think more deeply about the question doesn’t use questions per se but contains several sentences presented as a short essay. Some of the information in the essay is correct, and some of it is incorrect. The students must identify mistakes in the essay. The multiple-choice options list different numbers of errors so that students need to select the correct number.
There’s also some pretty damning evidence cited in the article. A lot of us don’t write very good multiple-choice questions, especially those who write questions for test banks. The most common problem in an analysis of almost 1,200 multiple-choice questions from 16 undergraduate courses was “nonoptimal incorrect answers,” so designated if they were selected by fewer than 5 percent of the test takers. In other words, they’re obvious wrong answers because almost nobody is picking them, and that means more correct answers are selected by guessing.
Students want test questions to be fair. If they think tests are, research indicates students are more likely to study and do, in fact, learn more as a result of their studying. What they mean by “fairness” is pretty straightforward. The questions need to be clear: students should be able to figure out what the question is asking even if they can’t answer it. There should be a broad range of questions. If the test covers three chapters, there should be content from all three chapters. The authors recommend giving students sample test questions so they know what to expect and can’t persuade themselves that memorizing a few definitions will be all it takes to conquer the test.
The article recommends several guides for writing good multiple-choice questions. It’s good to remember: the easiest questions to write are those that focus on specific information bits and have one right answer. Other details also merit consideration. For example, how many answer options should a question have? Four? More? A fairly recent study looked three, four, or five options in terms of test reliability, response behaviors, item difficulty, and item fit statistics and found no evidence of significant differences between the numbers of answer options. A meta-analysis of 27 studies (across different content areas and age levels) recommends three options. There’s an interesting reason that favors three-answer options. On average, students can answer three-option questions five seconds faster than those with four or five options. That means more questions can be included on the exam. Imagine how popular that will be with students!
“Consistent meaningful feedback (e.g., detailed explanation of why certain answers were correct or incorrect) is an important component of student learning outcomes, enjoyment, engagement in the course and rating of teaching quality” (p. 151), are the findings of another study referenced in the review. This argues for more than posting the right answers on the professor’s office door or on the course website. The authors recommend an interesting way of providing this high-quality feedback, which is giving students opportunities to self-correct. Research shows that students who were allowed to turn in a self-corrected midterm performed better on the final than students who weren’t given this option. Both the in-class (or online) exam and the self-correct exam are scored. Students earn full credit if the answer on both exams is correct, partial credit if the question is missed on the in-class test but corrected on the take-home version, and no credit if it’s wrong on both tests.
As these highlights illustrate, this is an article packed full of good information. Students take tests seriously. We need to do our best to make those exams fair and accurate measures of learning.
Reference: Xu, X., Kauer, S., and Tupy, S. (2016). Multiple-choice questions: Tips for optimizing assessment in-seat and online. Journal of Scholarship on Teaching and Learning in Psychology, 2(2), 147–158.
Reprinted from Teaching Professor, 30.9 (2016): 3,7. © Magna Publications. All rights reserved.
