Inspired by Cook and West’s approach [20] to conducting systematic reviews in medical education and existing review papers [21-24], the screening and classification process conducted is presented subsequently.

The focused questions addressed the population, intervention, comparison, and outcomes (PICO) model as recommended by Cook and West [20]. In addressing the issues above, the research questions addressed in this review are:

Eligibility criteria for the selection of studies for review were also determined in light of the PICO guidelines. The population was limited to postsecondary education, specifically in dentistry, medicine, and speech and hearing sciences, in which PBL was the key educational pedagogy and curricula. Three types of educational technologies were identified as interventions used to support PBL: learning software and digital learning objects (video/3D models), IWBs and plasma screens, and LMSs. Three types of technologies were selected based on their relatively frequent implementation and innovations as indicated in on-site visitations and communications with health sciences PBL curricula across the globe. Regarding comparisons, although studies adopting experimental designs were included, this was not considered an exclusive criterion given that much educational research in the field is case-based. Finally, included studies indicated outcomes of the effects, both positive and negative, of the use of educational technologies on student learning and staff engagement in PBL. Evidence was determined from both databases and the grey literature.

A comprehensive computerized database search of full-text articles published in English from 1996 to 2014 was carried out using 3 education databases: ProQuest, Scopus, and EBSCOhost. Initial search terms were (“educational technologies” OR “learning technologies”) AND (“problem-based learning” OR “problem based learning” OR “PBL”) AND (“clinical” OR “dent*” OR “med*” OR “speech and hearing”). To narrow down the number of studies retrieved in each database, search terms in title/keywords/abstracts were selected in the initial search stage. The titles and abstracts of retrieved papers were first screened and rated for inclusion based on the PICO inclusion criteria. Additional cross-referencing uncovered grey literature in the form of articles and book chapters. Reviews and commentaries were excluded. The review flowchart (Figure 1) indicates the educational database search method and criteria as well as the final number of studies yielded for analysis (N=28). Search results indicate 3 types of educational technologies, learning software and digital learning objects, IWB (Figure 2 and plasma screens, and LMS, were investigated. Given that LMS combines a range of course or subject management and pedagogical tools to offer a means of designing, building, and delivering online learning environments [25], LMS in the search process includes examples of what are also termed course management systems or CMS (eg, WebCT/Blackboard, Angel, Sakai, and Moodle). Following Cook and West’s approach [20], key information (ie, author, year, research design, research purpose, findings) for each article were included. The results were then analyzed and synthesized by narrative or quantitative pooling, exploring effects of educational technologies in PBL-based applications.

Educational technologies have been increasingly used in health sciences education to support or substitute traditional didactic approaches to teaching and learning with inquiry-based approaches. Of the final 28 studies, 20 examined applications of various software and digital learning objects, 5 studies examined the application of IWBs and plasma screens, and 3 explored the application of LMSs in PBL across the 3 clinical disciplines.

Table 1 indicates studies that have implemented, evaluated, or explored a variety of software and digital learning objects in problem-based clinical education. Software included an interactive distance learning program in obstetrics and gynecology [26], concept mapping [27], 3D visualization [28], CD-ROM (ErgoROM [29]), a text and image database on CD-ROM [30], a Web-based learning portfolio (SkillsBase [31]), online virtual/simulated patients [32-36], video case(s) [37,38], online cases [39,40], an online resource simulating surgical clinical decision making (SURGENT [41]), Interactive Case-based Online Network [42], virtual PBL [43], and computer-based software [44,45]. The majority of these studies analyzed questionnaire data to investigate user perceptions of the efficacy of the various software and digital learning objects that were piloted, implemented, or developed as learning innovations. The purpose of integrating these software and digital learning objects in PBL were reported variously as an aid/supplement [39] or a replacement [34] for traditional formats, such as lectures, dissection, and clinical practice; or for the development of innovative approaches, bridging the gap between theoretical knowledge and clinical practice [29]; or for facilitating collaboration outside of the classroom [42]. Additional implementation goals were the reinforcement of knowledge construction and supporting decision-making processes [33,41], as well as the advancement of teaching and learning [35].

Perceived positive educational impact was seen in providing a more authentic learner environment [33]; conveying and facilitating understanding of information and complex phenomena [28,41]; facilitating enhanced knowledge [35]; improving cognitive, metacognitive, affective, and overall learning processes or outcomes in PBL [37,38]; having a positive impact on active learning [29,34] and critical thinking [29]; a reflective aid to learning clinical skills [31]; providing a suitable environment for collaboration and communication [33,43]; permitting reduced laboratory time; and increasing small-group activity with less reliance on staff [43]. Although the majority were positively disposed toward learning technologies in PBL, 1 study was more critical [32] finding the PBL video scenario to be cumbersome and not imitating real life; therefore, it was seen to be of little educational value.

The key implications include the importance of the modality of the scenario presentation [32] and the need for guiding principles and a direct facilitator connected to the use of 3D visualizations [28]. Hege et al [39] indicated integration of a computer-based learning tool into the curriculum is as important as the optimization of the software itself and concluded that a few aspects or strategies needed to be considered in integration of software into curriculum (eg, the software should be easy-to-use, highly accessible, and should support user evaluation, the delivering of content, user support, and case maintenance). Jha and Duffy [26] proposed 10 “golden rules” from an evaluation of a CD-ROM program in continuing medical education.

Table 2 indicates that the use of IWBs is a new phenomenon in clinical education with 4 studies of IWBs in PBL curricula [3,11,46,47] arising in the past 7 years in addition to 1 study on the use of plasma screens [48] in 2005. Kerfoot [48] indicated that the introduction of computers and plasma screens had a positive impact on PBL tutorials. Bridges and her colleagues [11] adopted an interactional ethnographic methodology to analyze student engagement with digital materials through the use of an IWB and found that “the integration of face-to-face and virtual modalities through the single PBL group’s use of an IWB across tutorials and self-study was seamless and supported whole-group engagement in the process.” In another study in undergraduate dentistry, Bridges and her colleagues [3] noted that the use of different texts and tools in one problem cycle supported a discursive shift from stimulus for hypothesizing to evidence for final hypotheses. Lu’s 2 studies in medical education [46,47] compared a traditional whiteboard group with an IWB group . One study [47] described that nature of scaffolding of collaborative problem solving under the 2 conditions and concluded that educational technology such as IWB can help by expanding the scaffolding choices. The other study [46] identified relationships between learners’ collaborative decision making and communicative discourse when engaged in a simulated medical emergency. Group differences were found in that IWB group participants engaged in more adaptive decision-making behavior earlier than the traditional whiteboard group, which led to shared understandings and subsequently to more effective patient management [46]. They also found more productive argumentation in the type of collaborative discourse produced in IWB medical student groups [46]. There are limited studies to show the relative advantages or disadvantages of using IWBs in health sciences education. More studies, therefore, are encouraged to explore both the impact of using IWBs for large-screen visualization and collaboration.

Table 3 lists studies that adopted various LMSs to support problem-based approaches. These included the application of WebCT [49], VMS Portal [50] to manage inquiry-based materials and activities for PBL curricula, and iSUS for self-directed learning [51].

A few studies noted the positive effects of using LMS in PBL curricula [49-51]. In Dornan’s study [51], a Web-based LMS-ISUS helped to provide practical guidance about what to learn and how to learn, helped access appropriate experiences and manage time, gave feedback on students’ accumulated real patient learning, provided peer comparison, and helped self-management. The authors argued that ISUS can provide the motivational jigsaw to fill the gap between PBL and placement learning. McLean [49] found students perceived that LMS are the most useful means of communication and resource delivery for PBL in medicine. His study also highlighted limitations in that insufficient support, resources, and training might result in less successful implementation of educational technologies.

With regard to the question of scaffolding, some of the previous studies noted that monitoring, support, and development are important for efficient and positive implementation of an LMS in PBL curricula. In the VMS Portal project [50], medical students were involved in website development to help, consolidate, integrate, and develop Web resources for their peers. Critical to successful scaffolding in PBL are tutor facilitation strategies [10]. In online environments, tutor presence, ongoing engagement, and timely feedback become factors in facilitating students’ problem solving, self-directed learning, and collaboration in health sciences education.

The journal articles and book chapters examined in this systematic review indicate the generally positive effect of the thoughtful implementation of educational technologies in PBL. This is particularly the case where such technologies support scaffolding thereby reducing cognitive load and allowing students to learn in complex domains [19]. Firstly, when considering resource development, educational technologies enable provision of rich, authentic problems and/or case contexts accessible on demand in virtual spaces. Online virtual/simulated patients, video case(s), and online cases [33-41] convey complex phenomena in a more authentic learning environment. Secondly, educational technologies provide not only access to engagement in the problem-based inquiry, but also structure information by embedding expert knowledge and skills. This may be in the form of problem-relevant videos and simulations made available during self-directed learning [3] demonstrating case reports independent of time and place. Thirdly, educational technologies support students and their facilitators in making disciplinary thinking explicit. Dedicated software can help learners to construct explanations, structure tasks, and make them more manageable [19]. Adaptations of standard LMS as well as dedicated inquiry-based LMS can provide a platform to elicit articulation, collaboration, and reflection [52]. Technological resources, such as different software and 3D models, LMS, and IWB, can assist students engage in problem-solving processes.

Although generally positive, a limited number of studies have indicated the adverse effects of educational technologies or their methods of implementation. In this review paper, the less successful implementations may be attributable to the content or delivery of the video scenario [32]. Additional cognitive burden due to higher levels of complexity was seen as a possible limitation to their effective implementation; however, in highly positive cases, technologies were seen as providing an additional supportive scaffold for student learning. In terms of infrastructure and staff development, insufficient support, resources, and training [49,53] were seen as disabling. Adopting and/or adapting functionalities from software or LMS with limited staff-student and/or student-student interactivity and limited student feedback processes were seen as shortcomings in cases where this occurred. This would indicate that thoughtful instructional design approaches need to be applied when adapting more traditional systems or when designing new programs for inquiry-based learning [54]. Well-designed empirical research studies are needed to establish best practices for technological hardware and software in enhancing teaching and learning productivity and building stronger learning communities [55].

Although the major forms of educational technologies presented in these studies look very promising and are potentially fit for educational purpose in many problem-based health sciences education settings, there is a need for support and training in light of the ever-changing nature of both technical knowledge of teaching staff and the technological affordances themselves. The power of computers and Internet, merging with multimedia and interactive spaces, appear to allow a high degree of flexibility and accessibility to digital instructional content in health sciences education [56]. These affordances may enable integration of technologies into the curriculum and support staff development by encouraging interactive teaching and learning. However, the often-false assumption is the “sink or swim” approach in which faculty may assume students’ prior technological skills or knowledge. Where new affordances are introduced or even old affordances with new inquiry-based purposes, additional hands-on tutorials and training are helpful to facilitate student adaptation to the technology and ensure optimal benefit [49,57]. Successful multimedia teaching and learning relies not only on the proper use of information technology, but also on a clear implementation strategy [58]. Implications also arise in considering Technological Pedagogical Content Knowledge [59] with regard to disciplinary influences on the ways that educational technologies are incorporated into the curriculum. Guidance and support should be tailored to meet the needs of each user including learner, facilitators, and curriculum developers to be at its most effective.

In conclusion, this literature review indicates a generally positive effect from the adoption of various educational technologies in PBL. Positive outcomes for student learning included providing rich, authentic problems and/or case contexts for learning; supporting student development of medical expertise through the accessing and structuring of expert knowledge and skills; making disciplinary thinking and strategies explicit; providing a platform to elicit articulation, collaboration, and reflection; and reducing perceived cognitive load. Insufficient technical support, infrastructure, and resources were seen as impacting negatively on uptake and learning outcomes. Staff and student induction and ongoing training in the use of educational technologies for learning in inquiry-based contexts such as PBL is recommended.

Although educational technologies have been increasingly used in health sciences education, it has been questioned whether they can completely substitute traditional teaching methods [60]. The rise of Massive Online Open Courses in all fields, including health sciences, has been seen as positive, particularly for continuous medical education and public health literacy [15]. In considering undergraduate inquiry-based curricula, this review supports Hmelo-Silver [19] and Bridges et al’s [61] predictions that technology can play an important but synergistic role with other components of PBL. Further research into the various applications of educational technologies in PBL curricula is needed to fully realize their potential in enhancing inquiry-based approaches in health sciences education. In an increasingly digital, networked world, convergence of educational technologies is increasingly apparent. This has given rise to understandings that learners are positioned within digital ecosystems. Consequently, it is possible that a learner might engage with the merging of distinct educational technologies. The effects of learning in a digital ecosystem need to be identified and explored in further research.