Adaptive Hypermedia: A New Paradigm for Educational Software

Abstract

Traditional online dental education courses follow the broadcast paradigm which centers on the teacher, not the student. This one-size-fits-all approach resembles a mass-production idea which cannot take individual learner characteristics into account. Most online course designs do not address the issue that users with different goals and knowledge may be interested in different pieces of information about a topic. Adaptive hypermedia (AH) is an emerging field in education research which investigates how computer systems can overcome this problem. AH can be applied to any course content. This learner-centered approach first considers the learning goal(s), then evaluates the user’s abilities and determines the individual learning style, to structure and tailor the curriculum most efficiently. The presented AH environment exploits various concepts of AH. The system collects data to create a model of the individual user, which is continuously refined based on test results throughout the course. The system then adapts the learning material dynamically, using active and passive curriculum sequencing and adaptive presentation.

Introduction

Common sense tells us that we should teach different learners differently. Parents demonstrate this intuitive wisdom when they communicate differently to their children according to their specific ages. Current Bush Administration advisors formally recognized the instructional superiority of individual tutoring by suggesting that one-on-one tutoring be accepted as the gold standard for evaluating technology-based educational systems (President’s Information Technology Advisory Committee, 2003). Nevertheless, when students enter higher education (and pay a substantial amount of money for their education), we fall back to a collectivistic model of instruction, that denies them the opportunity to be taught according to their individual goals, prior knowledge, and learning style.

This paper attempts to address this contradiction by introducing a new paradigm for technology-based education. Adaptive Hypermedia (AH) moves away from broadcast-based teacher-centered applications to individualized learner-centered education delivery systems. Application of the AH paradigm to dental education might be able to address some of the challenges we face in dental education, such as the nationwide shortage of dental faculty (Haden and Valachovic, 2003), the increasingly diverse professional and educational background of students entering dental schools, and the need to take advantage of the knowledge explosion in the biomedical sciences and apply such knowledge to public health (DePaola et al., 2002).

This paper assumes that the reader has either worked on the development of educational software or has simply used educational software as an instructor or learner.

Before suggesting a new paradigm for educational software, we will start with an analysis of the current state of learning environments and their weak points.

Current State of Learning Environments in Dental Education

Distance education has come a long way since 1840, when Sir Isaac Pitman, the English inventor of shorthand, came up with the idea of delivering instruction via correspondence courses by mail (Phillips, 1998). But only with the advances of modern technology has distance education grown to a multibillion dollar market. The past decade has seen the design of many different kinds of online courses and e-learning environments. While most of them were not thoroughly evaluated for their educational effectiveness and user acceptance (Phipps and Merisotis, 1999), they were often referred to as “advances in learning”. Especially in the late 1990s, their creators predicted that online courses would soon replace traditional classroom-based instruction in many areas of education. Neglecting the difference between instructional method and delivery media (Clark, 1993), they claimed that the advantages of these online courses, such as the independence from location and time and their unlimited scalability, made them far superior to traditional courses.

There is a general and increasing trend toward usage of e-learning in education, which has been observed in dentistry as well. After an exhaustive search for Web-based continuing dental education (CDE) courses in 1998, Schleyer and Pham (1999) could identify only 158 courses. In 2003, just one online CDE provider, DentalxChange, offers 489 courses in 23 different categories. However, previous studies uncovered major deficiencies in existing Web-based CDE courses. For example, 30% of the courses did not indicate an author, no time of last update was shown in a majority of the courses, and the relationship between course length and credit hours varied significantly (Schleyer and Pham, 1999). Usually no formative or summative evaluations were performed, resulting in a lack of corrective feedback (Corbett et al., 1997). Dental educational software developers (and the publisher-assigned peer-reviewers!) are often satisfied with evaluations which use phrases such as “anecdotal evidence”, “unstructured student feedback”, and “students expressed that they liked...”. These existing deficiencies allow one to conclude that online courses are prepared with less care and effort than peer-reviewed papers, whose underlying research is repeatedly scrutinized on various levels (e.g., departmental review to initiate a new line of research; review by funding agency to gain financial support; review by IRB for ethical considerations; review by journal editor for entrance into peer-review process; blind peer review for scholarly value; and review by editorial staff for style and consistency). We can only speculate about the reasons why the increased development and usage of dental online educational systems fails to translate into improved quality. One reason could be the risk for online course developers of losing content, because the lack of standards prevents the transfer of content to other environments. Another reason could be that the rewards system in higher education offers little incentive for instructors to make the substantial investment of time and effort required to develop high-quality courses. Yet another reason could be overly enthusiastic encouragement by administrators who mistakenly assume that distance education can solve some of higher education’s ills. Program directors, for instance, might assume that online continuing education courses can attract thousands of dentists who are willing to pay for their CE credits—and don’t even need parking (Austin, 1999; Carnevale, 1999; O’Neill, 1999)! The sobering truth is that in dental online learning, no one has yet reported profits (Spallek et al., 2002).

A decade of work in the field of Web-based dental education systems and a careful review of the literature allow me to suggest the following summary of the current state of learning environments in dental education:

Web-based online courses and supplemental lecture material are widely used.

Most of these courses consist of a collection of static Web pages which are simple translations of existing lecture notes or course manuals.

Most positive experiences reported in the literature show that this approach works well for prepared, motivated students, in reasonably homogeneous virtual classrooms, who have access to teachers to fill in possible gaps and resolve misunderstandings (Brusilovsky et al., 2002).

E-learning approaches can be categorized into three generations (Table 1).

Toward a Learner-centered Education

The following section will explain educational AH on a conceptual level, describing it as a move toward learner-centered instruction, which structures and tailors the offered material in the most efficient way—that is, in a way that addresses the individual student’s learning goals, prior knowledge, and learning style.

AH is an emerging research direction that focuses on the design of Web-based software that adapts to the user (Brusilovsky, 2001). While education research represents 2/3 of the total body of all AH research (De Bra et al., 2002), we find many other applications, such as: information retrieval systems that take not only the query but also the user’s long-term interest into account (Gates et al., 1998); individualized help systems (Encarnacão and Stoev, 1999); and personalized information kiosk systems (Bullock and Goble, 1998).

Educational AH systems try to match different learning styles and levels of pre-existing knowledge to make learning more efficient and effective. Users with different learning goals may be interested in different pieces of information about a certain knowledge item. The term “knowledge item” can be substituted with “subtopic” or “little piece of information about the domain”. AH systems build a model of the individual user and apply this model to the content adaptation geared toward the learner’s knowledge and goals. The “user model”, also referred to as “mental state” of the user (De Bra and Calvi, 1998), stores some value which is an estimation of the user’s knowledge level about a knowledge item. For each knowledge item belonging to a domain—or, in a more practical sense, belonging to an online course—such a user model exists and is used to adapt the teaching sequence and the presentation of the material to the user. More details about the user modeling process would exceed the scope of this paper, but can be reviewed elsewhere (Brusilovsky et al., 2003).

Now we turn to a description of what can be adapted with the help of a user model, focusing on only two examples of AH methods, adaptive presentation and curriculum sequencing (Brusilovsky, 2001).

With adaptive presentation technology, Web pages are not no longer static, but can be adaptively generated for each user. Such an adaptation could, for instance, offer a novice in the field only the basics about a certain topic without any technical jargon, while the same topic presented to an expert could include the newest research findings and advanced concepts. Only after the novice had mastered a certain knowledge level would more advanced concepts be offered about the topic.

Curriculum sequencing, also referred to as “instructional planning technology”, provides the learner with the most suitable individually planned sequence of knowledge items to learn. It helps the learner to find an “optimal path” through the offered material.

We can distinguish active curriculum sequencing, which requires a stated learning goal and then builds the best individual path to achieve this goal, from passive sequencing, which starts when the learner is unable to solve a problem. Passive sequencing offers the learner a subset of available learning material, which can fill the gap in the learner’s knowledge. Now let’s turn to the architecture of the newly developed educational AH system.

Development of an Adaptive Hypermedia Course for Dental Education

Already in the planning stage of our AH course for dental education, we aimed at the development of an easy-to-use learning environment—easy to use not only for the learners, but also for the authors of the courses. While the level of computer literacy of incoming dental students continuously increases, most dental educators are equipped with only limited knowledge about information technology.

We identified the course “Information Retrieval for Dental Professionals” as an appropriate prototype course because we expected a wide range of potential course participants with various levels of computer literacy. We therefore predicted the full utilization of the adaptation feature, which would give us the opportunity to evaluate the system thoroughly. (Currently, the course shell is in the process of being deployed for tissue engineering courses taught at the University of Pittsburgh. This effort is supported by seed funding from the University of Pittsburgh [Innovation in Teaching Award 2003].)

Applying principles of user modeling research, we identified the characteristics of the learner that can be used as a source of the adaptation (Brusilovsky, 2001)). We knew that the successful design of a highly individualized course could be achieved only if we collected enough information about the learner (Table 2).

After identifying useful information about the learner, we designed various methods to collect and use this information as part of the learning experience. The following section describes a learner’s typical path through the newly developed AH learning environment.

During course enrollment, the learner provides demographic data, access rights, and privacy settings. The privacy settings, for instance, determine if the learner’s e-mail address is displayed when annotating the learning material. Based on the learner’s selected learning goal, a set of knowledge items is compiled. These sets of knowledge items are assembled by the author during course creation. Because the adaptation is determined by the system’s estimation of the learner’s knowledge level for each knowledge item, the system compiles a pre-test whose results build part of the user model. After completion of the pre-test, the learner selects his learning style, which results in an adaptation of the presentation. Now the system compiles the individually tailored curriculum and presents the adapted course pages. The results of quizzes throughout the course update the user model in a positive or negative way, depending on whether the question was answered right or wrong—a process called knowledge-tracing. During course creation, the author determines the level of knowledge necessary for the learner to be eligible for the final test. Only after the learner reaches this level is the final test presented. The result of the final can be printed as a certificate including demographic information, such as address and license number, for continuing education certification.

Technical Implementation of the System

The developed AH system includes two main applications, an authoring tool for developing courses and a learner environment. A highly granular relational database (MySQL, version 3.23.56) stores the learning material (content, questions, answer options, examples, learning goals, course settings, etc.), which is shielded from the programming code (PHP version 4.3.1). This architecture allows for the separation of course content and programming logic, which facilitates, for instance, the export of course content for other purposes.

Authors are authenticated via username and password when entering the Web-based authoring tool, which is hosted on a Web server housed in the School of Dental Medicine (IIS version). The learning material of a course consists of all knowledge items which are entered into the system through this tool. For each knowledge item, the author enters and edits various views on the same topic, differentiated by sophistication. Later, the author enters the associated questions and examples according to the appropriate levels of difficulty. Multiple-choice and fill-in-the-blank questions include an author-defined level of difficulty used to stratify the randomized questions for the knowledge assessments.

Following Mark Weiser’s key idea about ubiquitous computing—that “[t]he most profound technologies are those that disappear” and “weave themselves into the fabric of everyday life until they are indistinguishable from it”—we purposefully structured the learning environment (from the learner’s perspective) to resemble a first- or second-generation Web-based course (Weiser, 1991). Applying this design philosophy eliminates the need for a time-consuming training session prior to the use of the new learning environment. The learning environment resides on the public Web server of the School of Dental Medicine under the URL http://di.dental.pitt.edu/ir/ and is open for enrollment as a 3-credit-hour continuing dental education course. Furthermore, the course is used as one element of the required freshman course “Introduction to Computing”.

The learning environment exploits the emotional intelligence “embodied” in a personal tutor which guides the learner through the course. An emotionally intelligent tutor can improve user acceptance of the educational experience. Using empathy and encouragement, the tutor can help the user to relieve frustration and recover to a positive emotional state (Klein, 1999). The scope of this paper does not permit expansion on the emotional aspect of learner environments, but I would like to recognize the research findings which support the claim that humans relate to computers as social actors (Reeves and Nass, 1996).

Evaluation

We developed a multi-step evaluation process for the new educational AH system using the course “Information Retrieval for Dental Professionals” as a prototype. Before bringing the course to users for testing, we performed cognitive walkthroughs under the observation of the developer, to solve any unanticipated problems. As our test users encountered obstacles in the course, we were able to identify problems and fix them after each iteration.

For the alpha test, we asked experts in Web design and educational software to complete the course and provide informal feedback about their general impression, and to generate a list of all deficiencies in the areas of education, instructional design, technical implementation, design, and layout. All experts supported the use of AH for dental education, and repeatedly pointed out that the use of emotional intelligence was beneficial.

The formative evaluation is divided into two parts, a usability inspection performed by experts and an end-user testing for educational effectiveness. Following common usability engineering principles, a detailed protocol for the expert review was developed (see electronic copy of the protocol at http://di.dental.pitt.edu/expertevaluation/). The overall feedback from the experts was positive, but all of them found violations of certain usability heuristics which we addressed as they were brought to our attention. Table 3 shows the summarized feedback and how it was addressed.

We used a general-purpose heuristic for Web user interface design developed by Jakob Nielsen (2002) and a specialized heuristic for educational software development which was extracted from the ANSI Standard “Guidelines for the Design of Education Software” (ANSI, 1999).

Education research has thus far failed to suggest a standard instrument for reliable measurement of the effectiveness of an educational intervention (Donovan et al., 2000). Because we cannot measure effectiveness directly, we decided to evaluate the following aspects of the new learning environment, based on suggestions by various schools of education researchers (Clark, 2000):

— changes in student learning,

— values,

— motivations, and

— ability to transfer knowledge acquired outside the instructional setting.

Motivation and values regarding the instructional method will be correlated with users’ attitudes toward the media and their accomplishments in the learning environment itself.

The results of both parts of the formative evaluation will shape the further development and design of a summative evaluation.

Conclusion

Applying design principles of AH research, we were able to create our own course shell for a learning system using a state-of-the-art programming environment. For the first time, a dental educational delivery system can adapt itself dynamically to a learner using active and passive curriculum sequencing and adaptive presentation. While the system has not yet been formally evaluated (Part two of the formative evaluation is still ongoing), initial feedback suggests that it increases efficiency of learning and user satisfaction. Working on the authoring of the prototype course let us believe that what is more efficient on the learner’s end is more time-consuming on the course author’s site. The authoring of different views of a single topic to achieve adaptation to tiered levels of expertise is more difficult and more time-consuming than we had anticipated. Other investigators conducting similar studies experienced the authoring process as a challenge as well (De Bra and Calvi, 1998; Brusilovsky et al., 2002).

Challenges for Online Dental Education

Large numbers of unsatisfied learners, low enrollment in online courses (Spallek et al., 2002), and a lack of evidence from the current literature that online courses are significantly better than traditional courses (Ramage, 2003) show that the currently used model for online learning is weak.

Web-based education can reach a wide distance audience which would most likely be more diverse than a local classroom audience. But a compilation of static Web pages cannot take individual learner characteristics into account.

There is no one technique or procedure to teach all students effectively and efficiently (Donovan et al., 2000). Education researchers have spent a great amount of time and resources trying to find the single best instructional technique, but have thus far failed to deliver.

AH personalizes instructional material based only on information about the user’s cognitive processes. Yet there are other factors which influence students’ learning, such as the impact of emotions, intentions, and social interactions (Martinez and Bunderson, 2000). Thus, the social and emotional relationship between instructor and learner must also be electronically facilitated as an integral part of the learning environment.

The presented project exploits AH research to move instructional technology to an individualized teaching approach. In doing so, our model addresses many of the identified challenges for online dental education. Combined with the sound application of usability engineering principles and the facilitation of peer-learner contacts as well as learner-instructor contacts, adaptive hypermedia could serve as a new paradigm for educational software.

In Seymour Papert’s 1996 work, The Connected Family, he predicted that, “[i]n the learning environment of the future, every learner will be ‘special’”. With the development and utilization of adaptive instructional media, we can move slowly toward this future.

TABLE 1 —

The Three Generations of e-learning Approaches

Online course	First-generation	Second-generation	Third-generation
Key feature	collection of static Web pages	static Web pages adapted by author to fit a few stereotypes	adapt dynamically to individual learner
Source material	translations of existing lecture notes, course manuals, or papers	first-generation course saved at different Web addresses	versions of knowledge items belonging to a domain differentiated by sophistication
Conceptual approach	course material is still implicitly oriented for a traditional on-campus audience	stereotyping: every user fits into one of a few pre-defined groups	every learner is special and changes characteristics over time
Characteristic	one-size-fits-all	group-specific	individualized

TABLE 2 —

Characteristics of the Learner Used as a Source of Adaptation

Learner Characteristic	Information Source	Examples from Prototype Course
Knowledge	results of pre-test and quizzes throughout the course	multiple-choice and fill-in-the-blank questions
Interests	selection of a specific learning goal	“conceptual understanding of the information retrieval process” vs. “ability to find information on the Internet”
Traits	self-selected learning style	“emphasis on examples” vs. “emphasis on concepts”

TABLE 3 —

Expert Review Response and Developer Response (content-related issues omitted)

Problem	Developer Response
Initial settings regarding learning style and goal cannot be changed later.	Profile page was created to allow changes to all user settings at any time.
A logout button is missing.	Logout button was added.
Login should be labeled as “login”, not only as “e-mail address.”	Login is labeled consistently throughout the course as “e-mail address” to decrease short-term-memory load for learners (no need to remember separate username). No change.
Sometimes answers are determined as correct and wrong.	The bug affecting fill-in-the-blank questions was fixed.
The font is too small.	The font size will be adaptable by learner preference in version 2.
There is no difference between “learning by example” and “conceptual learning, no examples.”	The naming of the learning style was changed from “conceptual learning, no examples” to “omitting extra examples (only some important examples will be displayed)”. The new wording emphasizes that some examples (flagged by the author as mandatory) are still displayed.
The questions at the end of the pages disappeared after a while, preventing the mastering of KIs.	The number of questions associated with each KI was increased. Because of the system rule that a question can only be displayed again for the same user after 5 days, some KIs run out of questions after prolonged usage.
Pre-assessment questions were too difficult.	To evaluate the knowledge level of the learner, the pre-assessment test includes two easy and two difficult questions for each KI. No change.
Annotations: “private”/“public” appeared to be part of the annotation.	The type of annotation (“private”/“public”) is displayed in colored font to emphasize that this is a system message.
Confusion after completing an end-of-the-page quiz: What next?	Instructions were added to go on with the next topic from the curriculum.
Pre-assessment test was not completed, but submission of answers was possible without warning.	The pre-assessment test can be submitted incomplete because a novice learner might be unable to answer a large portion of the questions. Random guessing is not encouraged. No change.

Footnotes

Publication supported by Software of Excellence (Auckland, NZ)

Presented at “Dental Informatics & Dental Research: Making the Connection”, a conference held in, Bethesda, MD, USA, June 12–13, 2003, sponsored by the University of Pittsburgh Center for Dental Informatics and supported in part by award 1R13DE014611-01 from the National Institute of Dental and Craniofacial Research/National Library of Medicine.

Acknowledgements

The design of this course shell was greatly influenced by the work of Peter Brusilovsky, Department of Information Science and Telecommunications, School of Information Sciences, University of Pittsburgh, a pioneer in the field of Adaptive Hypermedia. Furthermore, the author and developer thank all individuals who gave input during the different development stages of this course. Special thanks go to Titus Schleyer, who gave input regarding the specifications, and to Ronald Kaiser, Gisela Spallek, and Petra Hiebl, who spent a great deal of their valuable time on the formative evaluation of the course. Very valuable expert reviews were performed by William Bartling, Carol Washburn, Andrea Hyde, Amy Gregg, Patricia Corby (all University of Pittsburgh) and Phil Eschallier (Comcation, Inc.). Marcos Kreinacke contributed to the project by providing his expert knowledge of JavaScript and PHP4. The author also expresses his thanks to Ms. Andrea Hyde, University of Pittsburgh, Center for Dental Informatics, for her thoughtful review of the manuscript.

References

ANSI Standards Committee on Dental Informatics (1999). Guidelines for the design of educational software. [Online] Available at http://di.dental.pitt.edu/edswstd/. Accessed 3/23/2004.

Austin E (1999). The caliber concept. University Business 2(2):24–31.

Brusilovsky P (2001). Adaptive hypermedia. User Modeling and User-adapted Interaction 11(1/2):87–110.

Brusilovsky P, Eklund J, Schwartz E (2002). Web-based education for all: a tool for development adaptive courseware. http://www7.scu.edu.au/programme/fullpapers/1893/com1893.htm.

Brusilovsky P, Corbett A, de Rosis F (2003). User modeling (lecture notes). In: Artificial Intelligence. Vol. 2702. Berlin: Springer Verlag.

Bullock JC, Goble CA (1998). TourisT: the application of a description logic based semantic hypermedia system for tourism. HYPERTEXT ’98. Proceedings of the Ninth ACM Conference on Hypertext and Hypermedia: Links, Objects, Time and Space—Structure in Hypermedia Systems, pp. 132–141. New York: ACM Press, ISBN 0-89791-972-6.

Carnevale D (1999). U of Maryland University College forms for-profit company for on-line courses. Chronicle Higher Educ 46(17).

Clark RE (1993). Media will never influence learning. http://www.usq.edu.au/material/unit/resource/clark/media.htm. Accessed 2-6-2003.

Clark RE (2000). Evaluating distance education: strategies and cautions. Q J Distance Educ 1(1).

10.

Corbett AT, Koedinger KR, Anderson JR (1997). Intelligent tutoring systems. In: Handbook of human-computer interaction. Helander MG, Landauer TK, and Prabhu PV, editors. Amsterdam, The Netherlands: Elsevier Science B.V., pp. 849–874.

11.

De Bra P, Calvi L (1998). AHA! An open adaptive hypermedia architecture. New Rev Hypermedia and Multimedia 4:115–139.

12.

De Bra P, Brusilovsky P, Conejo R (2002). Adaptive hypermedia and adaptive Web-based systems, AH2002. Berlin: Springer-Verlag.

13.

DePaola D, Howell H, Baker CG, Boy-Lefevre ML, Hull P, Holmstrup P, et al. (2002). Research and the dental student. Eur J Dent Educ 6(Suppl 3):45–51.

14.

Donovan SM, Bransford JD, Pellegrino JW (2000). How people learn: bridging research and practice. Washington, DC: National Academy Press.

15.

Encarnacão LM, Stoev SL (1999). An application-independent intelligent user support system exploiting action-sequence based user modeling. Proceedings of 7th International Conference on User Modeling. Vienna, Austria: Springer, pp. 245–254.

16.

Gates KF, Lawhead PB, Wilkins DE (1998). Toward an adaptive WWW: a case study in customized hypermedia. New Rev Multimedia and Hypermedia 4:89–113.

17.

Haden NK, Valachovic RW (2003). The ADEA-NIDCR National Research Conference on Putting Science into Practice: The Critical Role of Dental Schools. J Dent Educ 66:912–917.

18.

Klein JT (1999). Computer response to user frustration (thesis). Cambridge, MA: MIT Report No. TR480.

19.

Martinez M, Bunderson CV (2000). Foundations for personalized Web learning environments. Asynchronous Learning Networks Magazine 4(2). ISSN 1092-7131 (http://www.aln.org/ publications/magazine/v4n2/burdenson.asp).

20.

Nielsen J (2002). Heuristic evaluation. [Online] http://www.useit.com/papers/heuristic/. Accessed 8-28-2002.

21.

O’Neill JM (1999). For-profit Temple spin-off will offer online courses. The Philadelphia (PA) Inquirer, Cover.

22.

Papert S (1996). The connected family: bridging the digital generation gap. Marietta, GA: Longstreet Press.

23.

Phillips V (1998). Virtual classrooms, real education. Nation’s Business, pp. 41–45.

24.

Phipps R, Merisotis J (1999). What’s the difference? A review of contemporary research on the effectiveness of distance learning in higher education. Institute for Higher Education Policy Report No. CSD1488 (http://www.ihep.com/Pubs/ PDF/Difference.pdf).

25.

President’s Information Technology Advisory Committee (2003). Report to the President: using information technology to transform the way we learn (http://www.hpcc.gov/pubs/ pitac/pitac-tl-9feb01.pdf).

26.

Ramage TR (2003). The “no significant difference” phenomenon: a literature review. e-journal of instructional science and technology (http://www.usq.edu.au/electpub/e-jist/docs/ html2002/ramage.html).

27.

Reeves B, Nass C (1996). The media equation: how people treat computers, television, and news media like real people and places. Stanford, CA: CSLI.

28.

Schleyer T, Pham T (1999). Online continuing dental education. J Am Dent Assoc 130:848–854.

29.

Spallek H, Pilcher E, Lee JY, Schleyer T (2002). Evaluation of Web-based dental CE-courses. J Dent Educ 66:393–404.

30.

Weiser M (1991). The computer for the 21st Century. Scientif Amer 265(3):66–75.