Maintenance Notice

Due to necessary scheduled maintenance, the JMIR Publications website will be unavailable from Monday, March 11, 2019 at 4:00 PM to 4:30 PM EST. We apologize in advance for any inconvenience this may cause you.

Who will be affected?

Advertisement

Citing this Article

Right click to copy or hit: ctrl+c (cmd+c on mac)

Published on 22.01.20 in Vol 22, No 1 (2020): January

Preprints (earlier versions) of this paper are available at http://preprints.jmir.org/preprint/16808, first published Oct 27, 2019.

This paper is in the following e-collection/theme issue:

    Review

    A Web-Based Dementia Education Program and its Application to an Australian Web-Based Dementia Care Competency and Training Network: Integrative Systematic Review

    1Northern New South Wales Local Health District, New South Wales Ministry of Health, Lismore, Australia

    2Western Sydney University, Sydney, Australia

    3University Centre for Rural Health, University of Sydney, Sydney, Australia

    *all authors contributed equally

    Corresponding Author:

    Anne Moehead, MA

    Northern New South Wales Local Health District

    New South Wales Ministry of Health

    Uralba St

    Lismore, 2480

    Australia

    Phone: 61 429449591

    Fax:61 266202180

    Email: gmo11800@dodo.com.au


    ABSTRACT

    Background: Dementia education that meets quality and safety standards is paramount to ensure a highly skilled dementia care workforce. Web-based education provides a flexible and cost-effective medium. To be successful, Web-based education must contain features that promote learning and support knowledge translation into practice. The Dementia Care Competency and Training Network (DCC&TN) has developed an innovative Web-based program that promotes improvement of the attitudes, knowledge, skills, behavior, and practice of clinicians, regardless of their work setting, in order to improve the quality of life for people living with dementia.

    Objective: This review aims to (1) determine the key features that are associated with an effective and functional Web-based education program—an effective and functional Web-based program is defined as one that measures results, is accessible, is user friendly, and translates into clinical practice—and (2) determine how these features correlate with the DCC&TN.

    Methods: Six electronic databases—Medline, Embase, Cumulative Index to Nursing and Allied Health Literature (CINAHL), AusHealth, Nursing@Ovid, and Google Scholar—were searched for articles published between 2009 and 2018 using the following keywords: Education, Distance, Continuing, Learning, Online, Web-Based, Internet, Dementia, Program Evaluation, Validation Studies, Outcome and Process Assessment Healthcare, Nursing, Assisted Instruction, and Facilitated. The Critical Appraisal Skills Programme (CASP) and Kirkpatrick’s model for the evaluation of training were used to ensure quality and rigor of the analysis.

    Results: A total of 46 studies met the inclusion criteria. In total, 14 key features were associated with an effective Web-based learning environment, which enabled the environment to be as follows: self-directed, individualized, interactive, multimodal, flexible, accessible, consistent, cost-effective, measurable with respect to participant satisfaction, equitable, facilitated, nurturing of critical thinking and reflection, supportive of creating a learning community, and translated into practice. These features were further categorized into five subgroups: applicability, attractiveness, functionality, learner interaction, and implementation into practice. Literature frequently cites Kirkpatrick’s four-level model of evaluation and application in the review of education and training; however, few studies appeared to integrate all four levels of Kirkpatrick’s model. Features were then correlated against the DCC&TN, with an encouraging connection found between these features and their inclusion within the content and structure of the DCC&TN.

    Conclusions: A total of 14 key features were identified that support an effective and functional Web-based learning environment. Few studies incorporated Kirkpatrick’s salient elements of the model—reaction, learning, behavior, and results—in their evaluation and clinical application. It could, therefore, be considered prudent to include Kirkpatrick’s levels of training evaluation within studies of dementia training. There were few studies that evaluated Web-based dementia education programs, with even fewer reporting evidence that Web-based training could increase staff confidence, knowledge, skills, and attitudes toward people with dementia and be sustainable over time. The DCC&TN appeared to contain the majority of key features and is one of the few programs inclusive of hospital, community, and residential care settings. The 14 key features can potentially enhance and complement future development of online training programs for health sciences education and beyond. The DCC&TN model could potentially be used as a template for future developers and evaluators of Web-based dementia training.

    J Med Internet Res 2020;22(1):e16808

    doi:10.2196/16808

    KEYWORDS



    Introduction

    Background

    The global dementia epidemic demands a skilled and knowledgeable workforce ready to meet its related challenges [1]. For clinicians, there is an urgent need for access to education that is user friendly, affordable, and accessible with available peer support, supervision, and access to dementia champions, especially for those working in rural and remote regions. Web-based education has changed the face of learning and provides flexible, accessible, and cost-effective platforms for the delivery of education to a wide audience, regardless of their setting or location [2].

    In 2015, an estimated 46.8 million people were living with dementia globally. The number of people living with dementia is expected to reach 74.7 million by 2030 and 131.5 million by 2050. In high-income countries, this number will grow by 116% between 2015 and 2050 [3]. In 2018, the estimated cost of dementia to Australia was over Aus $15 billion [4]. People with dementia occupy up to one-quarter of Australian acute-care hospital beds [1,5] and make up 52% of all residents in residential aged-care facilities [4].

    The lack of professional knowledge around treatment and care options can result in delayed or hindered access to ongoing care, treatment, and support for people living with dementia [1,5]. Web-based education provides a platform for health professionals to access flexible education to improve awareness, knowledge, and skills in delivering dementia care. The high enrollment rate—almost 10,000 people from 65 countries—in the Understanding Dementia Massive Open Online Course (MOOC) [6] highlights the interest and need for quality dementia education. However, the 38% completion rate of the MOOC [6] was relatively low, which reflects the need for a more effective and responsive learning environment.

    In 2007, a dementia Web-based program—the Dementia Care Competency and Training Network (DCC&TN)—was developed based on recommendations from a report commissioned by New South Wales (NSW) Health by Wylie et al [7,8]. This program aims to advance the knowledge, skills, and practice of clinicians and is facilitated by dementia champions. Our definition of a dementia champion is a “clinician who has excellent knowledge and skills in the care of the person with dementia and has a commitment to provide information and support to those undertaking the online courses.” The DCC&TN delivers a Web-based learning platform that is interactive, multimodal, and facilitated. Additionally, the program includes dementia care competencies developed by Traynor et al [9,10]. These competencies are available to enhance the learner’s knowledge, skills, and attitudes in the delivery of person-centered care and are freely available. Prior to the development of these competencies, Traynor and coworkers reported that no dementia competency framework existed that was applicable across care settings or levels of practice [10].

    Since the inception of the DCC&TN, it has delivered high-quality dementia education and resources to over 10,000 clinicians across NSW, Australia, with an average completion rate of 78%. The program is a key training resource for NSW dementia clinical nurse consultants with a focus on a person-centered approach to delivering dementia care. The syllabus content of the DCC&TN aligns with the Australian Commission on Safety and Quality in Health Care Standards [5], with the goal of embedding these standards into clinical practice to improve the quality of life and outcomes for people living with dementia.

    The literature suggests that Web-based education is a flexible [6,11-14] and cost-effective [6,15-18] medium; however, to be effective and functional it must contain features that promote learning and support knowledge translation into practice [19-21]. To our knowledge, there is limited research that has evaluated Web-based dementia education programs. An effective and functional Web-based program is defined as one that measures results, is accessible, is user friendly, and translates to clinical practice.

    The DCC&TN provides a multifaceted education platform that assists clinicians in meeting the challenges of caring for the person living with dementia and in meeting professional obligations for lifelong learning. The training network is an online website comprising a content management system and a learning management system integrated into a single user experience that provides continuous membership, allowing ongoing access to resources, tools, clinical experts, and interactive forums. The library includes resources referenced within five online courses.

    Objectives

    This review aims to (1) determine the key features that are associated with an effective and functional Web-based education program, which is defined as one that measures results, is accessible, is user friendly, and translates into clinical practice, and (2) determine how these features correlate with the DCC&TN.


    Methods

    Overview

    Integrative review methodology [22] was chosen for several reasons. First, it allows for the inclusion of both experimental and nonexperimental research, so as to improve our understanding of a phenomenon. Second, integrative reviews can also combine empirical literature together with theoretical frameworks [22]. Lastly, integrative reviews address a diverse range of purposes, including defining concepts, reviewing theories or evidence, and analyzing methodological issues. The wide-ranging sampling frame, together with the diversity of purposes of integrative reviews, can assist in understanding complex concepts and theories, such as Web-based dementia care education. Hence, this review included both qualitative and quantitative studies as well as literature that looked at theories and empirical studies [22].

    Study Selection

    Six electronic databases—Medline, Embase, Cumulative Index to Nursing and Allied Health Literature (CINAHL), AusHealth, Nursing@Ovid, and Google Scholar—were searched for studies published between 2009 and 2018. Keywords used included the following: Education, Distance, Continuing, Learning, Online, Web-Based, Internet, Dementia, Program Evaluation, Validation Studies, Outcome and Process Assessment Healthcare, Nursing, Assisted Instruction, and Facilitated (see Multimedia Appendix 1). The Critical Appraisal Skills Programme (CASP) [23] and Kirkpatrick’s salient elements of the model for the evaluation of training were used to ensure quality and rigor of the analysis [18]. Search strategies were carried out using Medical Subject Headings (MeSH), relevant terms citations, and abbreviations (see Multimedia Appendix 1). Citation tracking and reference list inspections were undertaken in the search for relevant papers. Three clinical experts (KDS, AM, and KW) in dementia education conducted three stages of study selection: (1) an initial screening of titles, (2) a review of titles and abstracts, and (3) a review of the full text to identify suitable articles for inclusion. Consensus was used in the case of discrepancies.

    Eligibility Criteria

    Inclusion Criteria

    Studies had to adhere to the following inclusion criteria in order to be included in the review:

    1. Population: health personnel across different care settings or levels of practice.
    2. Concept: educational interventions that included (1) Web-based learning, online learning, Internet-based education, or computer‐assisted instruction, and (2) interactive facilitated education or tutor-supported education. Studies also had to measure learner satisfaction, knowledge, skills, or behavior.
    3. Context: articles from countries with similar health care provision to Australia were included.
    4. Types of studies: quantitative and qualitative research papers were included. Additionally, comparison studies, literature reviews, case studies, cohort studies, systematic reviews, and randomized controlled trials were included. Studies that contained experiential and correlational designs, quasi designs, flexible learning, and Web-based learning were also included.
    Exclusion Criteria

    Exclusion criteria were as follows: studies that excluded information technology (IT), studies published prior to 2009, studies that were unobtainable, non-English literature, studies with unrelated relevance to the review objectives, studies with nonhealth-related context, protocol descriptions, and studies with limited reporting on translation of learning or outcomes.

    Quality Appraisal, Abstraction, and Synthesis

    The CASP system of appraisal [23] was adopted to evaluate the final studies for rigor, methods, credibility, and relevance. CASP is a well-utilized tool to enhance the utility of evidence-based research by health professionals. Each article was critiqued for design, methods, and study detail including aims, ethical considerations, sample population and size, interventions, and outcome measures (see Textbox 1).

    Data Extraction

    A standardized data extraction process to perform data extraction was used. Any discrepancies were resolved by consensus. Data extracted from each eligible study was entered into a standardized form and included the following:

    1. General information: author, year of publication, and location.
    2. Study characteristics: aim, study design, and ethics.
    3. Sample population and size.
    4. Comparative interventions.
    5. Outcome measures and instruments.
    6. Main findings.

    A rating criteria framework was developed and agreed upon by the authors (see Textbox 1) to compare all studies so that recurring key features could be identified and subsequently applied during the review. These criteria also included questions on possible correlation of the studies with the components and characteristics of the DCC&TN.


    Textbox 1. Rating criteria applied across the 46 articles included in the review.
    View this box

    Association of Key Features With the Dementia Care Competency and Training Network

    Following the review, 14 key features were compared with the DCC&TN. Correlation with the features found in the review and the content and delivery of the DCC&TN was considered to be strong if eight or more features matched the DCC&TN, medium if between four and seven features matched the DCC&TN, and weak if three or fewer of the features matched the DCC&TN.

    Ethics Approval and Consent to Participate

    Ethics approval from a human research ethics committee was not required, as this was a systematic review of published literature.


    Results

    Search Outcome

    There were three phases of the search process. Initially, the systematic search produced 542 citations; 470 were excluded for the following reasons: duplication; non-English; full text unavailable; relevance to objectives; context not health related; not inclusive of, or not delivered via, the Internet or not Web based or online; protocol descriptions; and limited reporting on translation of learning or outcomes (see Figure 1). Secondly, 72 full-text papers were subsequently reviewed; of these, an additional 26 papers were excluded. The third phase included the review of the remaining 46 articles, which were matched to the rating criteria in Textbox 1. The review of articles included the following study designs: 6 randomized controlled trials, 30 cross-sectional studies, and 10 literature reviews.

    Figure 1. Flow diagram of the systematic review search.
    View this figure

    Setting and Participants

    Multimedia Appendix 2 displays a summary of the 46 included papers [6-56]. Most studies were conducted in the United States (13/46, 28%), followed by the United Kingdom (10/46, 22%) and Australia (8/46, 17%), with the remaining 33% (15/46) from a variety of countries. Web-based training programs were delivered to health professionals.

    Data Synthesis of Key Features

    The qualitative synthesis of the 46 articles was structured on what features were linked to an effective and functional Web-based learning experience and how this correlates with the DCC&TN. During data analysis, two initial observations were made. First, one seminal systematic review published in 2009 by Booth et al [14] identified five broad themes: peer communication, flexibility, support, knowledge validation, and course presentation and design. This was further supported by 11 subthemes, which provided a valuable framework for ongoing course development, as suggested by Booth et al and the literature.

    Second, 6 articles referred to Kirkpatrick’s salient elements of the four-level model, which are applied to determine “return on investment” and to “show [that] the business value and worth of training” was used to evaluate training and education provision [12,18,37,56]:

    1. Level 1: Examines the learner’s reaction to and satisfaction with the program.
    2. Level 2: Assesses the extent of learning and includes knowledge, skills, confidence, and attitudes.
    3. Level 3: Explores the extent to which completion of the training leads to staff behavior or practice change.
    4. Level 4: Assesses the results or outcomes of training, for example, in terms of quality of patient care.

    Surr and Gates [19,38], Ellis et al [12], and Scerri et al [39] refer to Kirkpatrick’s salient elements of the model of reaction, learning, behavior, results, and outcomes. Surr and Gates also reported that it is a weakness not to consider Kirkpatrick’s framework within dementia education [38].

    Upon further analysis of the 46 articles, 14 key features were identified and applied to what were considered effective and functional features for Web-based learning programs (see Table 1 and Multimedia Appendix 2). Some key features were used more frequently in studies, such as self-paced or self-directedlearning (n=29) and the program being interactive (n=28). Key features that were used less frequently included cost-effectiveness (n=10), provides equitable engagement (n=13), and individualized, based on learner’s profile and background (n=15).

    In total, 5 studies reported that training can increase staff confidence, knowledge, and attitudes toward those with dementia [20,21,33,39,40]. Gagnon et al [33] reported that the use of self-directed educational modules improved nurses’ knowledge and skills in relation to evidence-based practice. Du et al’s [21] systematic review identified learners as having a high satisfaction rate in regard to Web-based education as well as having improvement in knowledge and skills performance and enhanced self-efficacy in accomplishing nursing skills. Very few studies reported completion rates or whether learnings were translated into clinical practice. The themes that emerged from the review could be incorporated into future design elements of dementia-related Web-based learning programs. Some examples include the following:

    1. Delivery of an entire learning experience that goes beyond the module and syllabus design [17,19,24].
    2. Development of learning communities [13,21,31,41,45].
    3. Connecting learning to improved clinical practice [15,17,26,28,38,54].
    4. Learning that leads to the best possible provision of care for people living with dementia [17,21-49].

    A number of papers highlight barriers faced by online learners, including lack of time, competing interests, reliability of IT, organizational support during work hours, computer access, confidence with the computer, and ability to work at their own pace [19,38,44]. It is important that these factors are considered in the development of Web-based learning. For example, issues such as lack of time, competing interests, and organizational support may potentially be addressed by content design, which includes short, sharp training measures in the way Web-based education is delivered.

    It was frequently reported that learners who are engaged in Web-based studies require a level of commitment and willingness, the ability to develop self-direction, and a capacity for flexibility [14,19-37,42,43]. The authors’ evaluation of the learners’ feedback regarding the DCC&TN emphasized similar challenges.

    Application of Key Features to the Dementia Care Competency and Training Network Program

    Following the literature review, the 14 key features were cross-referenced with the syllabus content, the interactive elements of the learning platform, and the delivery environment of the DCC&TN. Table 1 shows that all features were found to correlate with the program; some features correlated strongly while others had only a weak correlation. Of the 46 studies reviewed, 57% (26/46) had a strong correlation, 28% (13/46) had a medium correlation, and 15% (7/46) had a weak correlation to the DCC&TN. Overall, the features identified in the literature, which are required for a functional and effective Web-based learning environment, are embedded in the content and structure within the learning management system of the DCC&TN.

    Table 1. Thematic analysis of the literature identifying key features for effective and functional online learning.
    View this table

    Discussion

    Principal Findings

    This literature review identified key features that could contribute to effective and functional Web-based learning programs. The literature frequently cited Kirkpatrick’s model of evaluation and application in the review of education and training, identifying four significant elements: reaction, learning, behavior, and results. These four elements align with the expanded 14 features identified in this literature review. Additionally, the review suggests that the structure and content of the DCC&TN nurtures critical thinking within a learning community, via support and facilitation by dementia champions, and ultimately encourages translation of learning into effective person-centered practice. This has been further substantiated by the evaluation of the participant’s response to feedback questionnaires during the learning experience. The DCC&TN achieves completion rates of 78% each year compared to, for example, 38% in the MOOC dementia education training program [6]. The DCC&TN has also aligned its syllabus content and resources with the Australian National Safety and Quality Health Service Standards of clinical care.

    Surr and Gates [38] conducted a systematic review on effective dementia education and training for the health and social care workforce. They concluded that none of the reviews examined each of the salient elements of Kirkpatrick’s four levels of evaluation. They also found that none of the reviews combined the elements to understand the full context of key features that lead to an efficacious dementia training program in the hospital setting. This was determined to be a weakness in the current dementia literature [19]. In total, 8 studies in this review highlighted the importance of including the evaluation of staff confidence, knowledge, and attitudes toward those living with dementia [19,21-49]. Additionally, the majority of studies did not report on completion rates or whether clinical practices could be sustained over time [20,54]. The authors suggest that these are important features for translating learning into practice and should be incorporated into future evaluations. This review indicates that there is still limited evidence on the effectiveness of training in changing staff confidence, knowledge, and attitudes [37,39].

    This analysis also identified a shortfall, in that much of the literature does not consider the importance of the following identified themes: training design, content or a delivery mode that is self-directed and individualized [14,38], interactivity [6,14], multimodality [6,14], flexibility [17,36,54], accessibility [17,36,54], consistency [6], and cost-effectiveness [6,17]. The findings indicate that greater depth and breadth of knowledge and education is needed to have an impact on clinicians’ feelings of caring efficacy, positive attitudes, and satisfaction toward people with dementia [13,19,39,40,47,49,54].

    Many of the 14 features were found across the literature in varying degrees, with Booth et al [14] identifying five similar themes and Kirkpatrick’s four salient themes, further validating the 14 key features linked to an effective Web-based education environment, as follows: self-directed, individualized, interactive, multimodal, flexible, accessible, consistent, cost-effective, measurable with respect to participant satisfaction, equitable, facilitated, nurturing of critical thinking, supportive of creating a learning community, and translated into practice. The DCC&TN is able to demonstrate correlation in varying degrees to each of these features. The DCC&TN has established a real-world application of what is fundamental to enable translation of dementia education to the clinical coalface.

    Barriers

    The literature reports that Web-based learners are challenged by the following: lack of time, busy workplace, reliability of IT, limited organizational support during work hours, computer access, confidence with computers, and the ability to work at their own pace. Learners require a strong commitment to external studies, requiring a level of discipline, a willingness to develop self-direction, and a capacity for resilience [14,19-37,42,43]. It is the authors’ opinion that a work-life balance needs to be addressed to encourage online learning among health professionals, their peers, and their managers.

    Strengths and Limitations

    Some of the excluded articles could have contributed further in emphasizing limitations for effective dementia Web-based training. The selection of papers reviewed were diverse and provided a good overview for comparison. Caution should be used when interpreting the results in order to consider outcomes that may have been reported elsewhere outside of this review.

    Recommendations and Future Research

    First, the DCC&TN can potentially provide an effective professional development platform and simultaneously advocate that improved outcomes for people living with dementia can be achieved with effective, functional, Web-based training programs.  The ideal outcome for any Web-based dementia program is to improve the quality of life and well-being of people with dementia through the delivery of person-centered care by a skilled and knowledgeable workforce. Therefore, future developers seeking to design and develop new and innovative Web-based learning programs for dementia clinicians could be well-inspired by the exemplary DCC&TN structure and delivery mode. Second, clinicians require better access to free learning opportunities to be informed and competent in knowledge translation [57]. Our DCC&TN is provided for free to clinicians to learn about dementia person-centered care and how to apply learned knowledge to practice. Third, there are many apps that offer functions that have resulted in reducing the burden and improving health outcomes of caregivers [58]. It will be interesting to conduct further research to identify whether the same 14 features are important for training family caregivers in an online environment. Fourth, currently there is an Aged Care Royal Commission in place in Australia. The Australian Government has responded to one of the key recommendations that has come out of the Commission and has announced that it will deliver Aus $10 million for additional dementia training and support for aged-care workers and providers [59]. The results from our study can potentially be used to assist in delivering this key recommendation. Fifth, the International Organization for Standardization aims for harmonization of products and services globally. Recently, a technical committee has been set up in the area of aging societies [60]. The results from this study can potentially be used to inform standardization processes in the area of online dementia training.

    Conclusions

    This review identified 14 key features that are linked to deliver a functional and effective online dementia learning environment. The DCC&TN demonstrates correlation in varying degrees to each of the identified key features required for an effective and functional Web-based learning environment in that it delivers a platform that is self-directed, individualized, interactive, multimodal, flexible, accessible, consistent, and cost-effective. It is suggested that critical thinking is nurtured within a learning community supported by dementia facilitators, while encouraging translation of learning into practice.

    Acknowledgments

    We would like to thank Katelyn Winkworth for her support in reviewing this paper.

    Conflicts of Interest

    None declared.

    Multimedia Appendix 1

    Example of a search strategy to identify papers for this review.

    DOC File , 50 KB

    Multimedia Appendix 2

    Summary of all included studies.

    DOC File , 280 KB

    References

    1. Alzheimer's Australia. 2014 Jun. Dementia Care in the Acute Hospital Setting: Issues and Strategies.   URL: https://www.dementia.org.au/files/Alzheimers_Australia_Numbered_Publication_40.PDF [accessed 2020-01-06]
    2. Cartwright J, Franklin D, Forman D, Freegard H. Promoting collaborative dementia care via online interprofessional education. Australas J Ageing 2015 Jun;34(2):88-94. [CrossRef] [Medline]
    3. Prince M, Wimo A, Guerchet M, Ali GC, Wu YT, Prina M. World Alzheimer Report 2015: The Global Impact of Dementia. An Analysis of Prevalence, Incidence, Cost and Trends. London, UK: Alzheimer’s Disease International (ADI); 2015 Aug.   URL: https://www.alz.co.uk/research/WorldAlzheimerReport2015.pdf [accessed 2020-01-06]
    4. Dementia Australia. 2019 Apr. Dementia statistics.   URL: https://www.dementia.org.au/statistics [accessed 2020-01-06]
    5. Australian Commission on Safety and Quality in Health Care. A Better Way to Care: Safe and High-Quality Care for Patients With Cognitive Impairment (Dementia and Delirium) in Hospital—Actions for Health Service Managers. Sydney, Australia: ACSQHC; 2014.   URL: https:/​/www.​safetyandquality.gov.au/​sites/​default/​files/​migrated/​A-better-way-to-care-Actions-for-health-service-managers.​pdf [accessed 2020-01-06]
    6. Goldberg LR, Bell E, King C, O'Mara C, McInerney F, Robinson A, et al. Relationship between participants' level of education and engagement in their completion of the Understanding Dementia Massive Open Online Course. BMC Med Educ 2015 Mar 26;15:60 [FREE Full text] [CrossRef] [Medline]
    7. Wylie K, Alle A, Hillier G, Higgins I, Penning C, Byles J. Models of Good Practice, Nationally and Internationally, on Dementia. Specific Education for Hospital Staff in Acute Care Settings: A Review of the Literature. Newcastle, Australia: Centre for Research and Education in Ageing (CREA), University of Newcastle, in partnership with NSW Centre for Rural & Remote Mental Health; 2005.
    8. Dementia Care Competency & Training Network.   URL: https://dementiacare.health.nsw.gov.au/ [accessed 2020-01-06]
    9. Traynor V, Cumming A, Britten N. Developing an Inter-Disciplinary Cross-Setting Dementia Care Competency Framework. Wollongong, Australia: University of Wollongong; 2015 Jan.   URL: http:/​/www.​adhere.org.au/​pdf/​Developing_an_Inter-disciplinary_Cross-Setting_Dementia_Care_Competency_Framework.​pdf [accessed 2020-01-06]
    10. Traynor V, Inoue K, Crookes P. Literature review: Understanding nursing competence in dementia care. J Clin Nurs 2011 Jul;20(13-14):1948-1960. [CrossRef] [Medline]
    11. Berger J, Topp R, Davis L, Jones J, Stewart L. Comparison of Web-based and face-to-face training concerning patient education within a hospital system. J Nurses Staff Dev 2009;25(3):127-132. [CrossRef]
    12. Ellis C, Roland D, Blair M. Professional educational interventions designed to improve knowledge and uptake of immunisation. Community Pract 2013 Jun;86(6):20-23. [Medline]
    13. Innes A, Kelly F, McCabe L. An evaluation of an online postgraduate dementia studies program. Gerontol Geriatr Educ 2012;33(4):364-382. [CrossRef] [Medline]
    14. Booth A, Carroll C, Papaioannou D, Sutton A, Wong R. Applying findings from a systematic review of workplace-based e-learning: Implications for health information professionals. Health Info Libr J 2009 Mar;26(1):4-21 [FREE Full text] [CrossRef] [Medline]
    15. Maloney S, Nicklen P, Rivers G, Foo J, Ooi YY, Reeves S, et al. A cost-effectiveness analysis of blended versus face-to-face delivery of evidence-based medicine to medical students. J Med Internet Res 2015 Jul 21;17(7):e182 [FREE Full text] [CrossRef] [Medline]
    16. Nguyen T. The effectiveness of online learning: Beyond no significant difference and future horizons. J Online Learn Teach 2015 Jun;11(2):309-319 [FREE Full text]
    17. Riley K, Schmidt D. Does online learning click with rural nurses? A qualitative study. Aust J Rural Health 2016 Aug;24(4):265-270. [CrossRef] [Medline]
    18. Kirkpatrick J, Kayser Kirkpatrick W. The Kirkpatrick Four Levels™: A Fresh Look After 50 Years, 1959 - 2009. Newnan, GA: Kirkpatrick Partners; 2009 Apr.   URL: http:/​/www.​kirkpatrickpartners.com/​Portals/​0/​Resources/​Kirkpatrick%20Four%20Levels%20white%20paper.​pdf [accessed 2020-01-06]
    19. Surr CC, Gates C, Irving D, Oyebode J, Smith SJ, Parveen S, et al. Effective dementia education and training for the health and social care workforce: A systematic review of the literature. Rev Educ Res 2017 Oct;87(5):966-1002 [FREE Full text] [CrossRef] [Medline]
    20. Carroll C, Booth A, Papaioannou D, Sutton A, Wong R. UK health-care professionals' experience of online learning techniques: A systematic review of qualitative data. J Contin Educ Health Prof 2009;29(4):235-241. [CrossRef] [Medline]
    21. Du S, Liu Z, Liu S, Yin H, Xu G, Zhang H, et al. Web-based distance learning for nurse education: A systematic review. Int Nurs Rev 2013 Jun;60(2):167-177. [CrossRef] [Medline]
    22. Whittemore R, Knafl K. The integrative review: Updated methodology. J Adv Nurs 2005 Dec;52(5):546-553 [FREE Full text] [CrossRef] [Medline]
    23. Critical Appraisal Skills Programme (CASP). 2018. CASP checklists.   URL: https://casp-uk.net/casp-tools-checklists/ [accessed 2020-01-06]
    24. Shatto B, Erwin K. Moving on from Millennials: Preparing for Generation Z. J Contin Educ Nurs 2016 Jun 01;47(6):253-254. [CrossRef] [Medline]
    25. McGowan BS, Balmer JT, Chappell K. Flipping the classroom: A data-driven model for nursing education. J Contin Educ Nurs 2014 Nov;45(11):477-478. [CrossRef] [Medline]
    26. Ballman K, Garritano N, Beery T. Broadening the reach of standardized patients in nurse practitioner education to include the distance learner. Nurse Educ 2016;41(5):230-233. [CrossRef]
    27. Hardy S, Mushore M, Goddard L. Supporting student mental health nurses in clinical placement through virtual in-practice support (VIPS): Innovation uptake and the 'VIPS' project. Nurse Educ Today 2016 Nov;46:133-138. [CrossRef] [Medline]
    28. Bourhy H, Troupin C, Faye O, Meslin F, Abela-Ridder B, Sall AA, et al. Customized online and onsite training for rabies-control officers. Bull World Health Organ 2015 Apr 27;93(7):503-506. [CrossRef]
    29. Peine A, Kabino K, Spreckelsen C. Self-directed learning can outperform direct instruction in the course of a modern German medical curriculum: Results of a mixed methods trial. BMC Med Educ 2016 Jun 03;16:158 [FREE Full text] [CrossRef] [Medline]
    30. Law M, Rapoport MJ, Seitz D, Davidson M, Madan R, Wiens A. Evaluation of a national online educational program in geriatric psychiatry. Acad Psychiatry 2016 Dec;40(6):923-927 [FREE Full text] [CrossRef] [Medline]
    31. Canniford LJ, Fox-Young S. Learning and assessing competence in reflective practice: Student evaluation of the relative value of aspects of an integrated, interactive reflective practice syllabus. Collegian 2015;22(3):291-297. [CrossRef] [Medline]
    32. Boyd MR, Baliko B, Polyakova-Norwood V. Using debates to teach evidence-based practice in large online courses. J Nurs Educ 2015 Oct;54(10):578-582. [CrossRef] [Medline]
    33. Gagnon J, Gagnon MP, Buteau RA, Azizah GM, Jetté S, Lampron A, et al. Adaptation and evaluation of online self-learning modules to teach critical appraisal and evidence-based practice in nursing: An international collaboration. Comput Inform Nurs 2015 Jul;33(7):285-294; quiz E1. [CrossRef] [Medline]
    34. Badiyepeymaie Jahromi Z, Mosalanejad L. Integrated method of teaching in Web Quest activity and its impact on undergraduate students' cognition and learning behaviors: A future trend in medical education. Glob J Health Sci 2015 Jan 14;7(4):249-259 [FREE Full text] [CrossRef] [Medline]
    35. Ramaswamy R, Leipzig RM, Howe CL, Sauvigne K, Usiak C, Soriano RP. The portal of geriatrics online education: A 21st-century resource for teaching geriatrics. J Am Geriatr Soc 2015 Feb;63(2):335-340. [CrossRef] [Medline]
    36. Phillips BN, Turnbull BJ, He FX. Assessing readiness for self-directed learning within a non-traditional nursing cohort. Nurse Educ Today 2015 Mar;35(3):e1-e7. [CrossRef] [Medline]
    37. Reviriego E, Cidoncha MA, Asua J, Gagnon MP, Mateos M, Gárate L, et al. Online training course on critical appraisal for nurses: Adaptation and assessment. BMC Med Educ 2014 Jul 05;14:136 [FREE Full text] [CrossRef] [Medline]
    38. Surr CA, Gates C. What works in delivering dementia education or training to hospital staff? A critical synthesis of the evidence. Int J Nurs Stud 2017 Oct;75:172-188. [CrossRef] [Medline]
    39. Scerri A, Innes A, Scerri C. Dementia training programmes for staff working in general hospital settings: A systematic review of the literature. Aging Ment Health 2017 Aug;21(8):783-796. [CrossRef] [Medline]
    40. Lewis KO, Cidon MJ, Seto TL, Chen H, Mahan JD. Leveraging e-learning in medical education. Curr Probl Pediatr Adolesc Health Care 2014 Jul;44(6):150-163. [CrossRef] [Medline]
    41. Seckman CA. Perceived sense of community, cognitive engagement, and learning outcomes among undergraduate nursing students enrolled in an Internet-based learning course. Comput Inform Nurs 2014 Oct;32(10):482-489; quiz 490. [CrossRef] [Medline]
    42. Arving C, Wadensten B, Johansson B. Registered nurses' thoughts on blended learning in a postgraduate course in cancer care: Content analyses of Web surveys and a focus group interview. J Cancer Educ 2014 Jun;29(2):278-283. [CrossRef] [Medline]
    43. Mgutshini T. Online or not? A comparison of students' experiences of an online and an on-campus class. Curationis 2013 Mar 18;36(1):E1-E7. [CrossRef] [Medline]
    44. Wearne SM, Greenhill J, Berryman C, Sweet L, Tietz L. An online course in clinical education: Experiences of Australian clinicians. Aust Fam Physician 2011 Dec;40(12):1001-1003 [FREE Full text]
    45. Moule P, Ward R, Lockyer L. Nursing and healthcare students' experiences and use of e-learning in higher education. J Adv Nurs 2010 Dec;66(12):2785-2795. [CrossRef] [Medline]
    46. Sowan AK, Jenkins LS. Designing, delivering and evaluating a distance learning nursing course responsive to students needs. Int J Med Inform 2013 Jun;82(6):553-564. [CrossRef] [Medline]
    47. Kontio R, Hätönen H, Joffe G, Pitkänen A, Lahti M, Välimäki M. Impact of eLearning course on nurses' professional competence in seclusion and restraint practices: 9-month follow-up results of a randomized controlled study (ISRCTN32869544). J Psychiatr Ment Health Nurs 2013 Apr;20(5):411-418. [CrossRef] [Medline]
    48. Sherman H, Comer L, Putnam L, Freeman H. Blended versus lecture learning. J Nurses Staff Dev 2012;28(4):186-190. [CrossRef]
    49. Surr C, Smith S, Crossland J, Robins J. Impact of a person-centred dementia care training programme on hospital staff attitudes, role efficacy and perceptions of caring for people with dementia: A repeated measures study. Int J Nurs Stud 2016 Jan;53:144-151. [CrossRef] [Medline]
    50. Chao S, Chang Y, Yang S, Clark M. Development, implementation, and effects of an integrated Web-based teaching model in a nursing ethics course. Nurse Educ Today 2017 Aug;55:31-37. [CrossRef] [Medline]
    51. López Soblechero MV, González Gaya C, Hernández Ramírez JJ. A comparative study of classroom and online distance modes of official vocational education and training. PLoS One 2014;9(5):e96052 [FREE Full text] [CrossRef] [Medline]
    52. Page J, Meehan-Andrews T, Weerakkody N, Hughes DL, Rathner JA. Student perceptions and learning outcomes of blended learning in a massive first-year core physiology for allied health subjects. Adv Physiol Educ 2017 Mar 01;41(1):44-55 [FREE Full text] [CrossRef] [Medline]
    53. Mayer B, Ring C, Muche R, Rothenbacher D, Schmidt-Strassburger U. Creating a blended learning module in an online master study programme in oncology. Educ Health (Abingdon) 2015;28(1):101-105 [FREE Full text] [CrossRef] [Medline]
    54. Ollerenshaw A, Wong Shee A, Yates M. Towards good dementia care: Awareness and uptake of an online Dementia Pathways tool for rural and regional primary health practitioners. Aust J Rural Health 2018 Apr;26(2):112-118. [CrossRef] [Medline]
    55. Collier E, Knifton C, Surr C. Dementia education in higher education institutions. Nurse Educ Today 2015 Jun;35(6):731-732. [CrossRef] [Medline]
    56. Hertfordshire County Council. Kirkpatrick’s Evaluation Model.   URL: https://www.hertfordshire.gov.uk/Media-library/Documents/Kirkpatricks-Evaluation-Model.pdf [accessed 2020-01-06]
    57. Damarell RA, Tieman JJ. How do clinicians learn about knowledge translation? An investigation of current Web-based learning opportunities. JMIR Med Educ 2017 Jul 13;3(2):e12 [FREE Full text] [CrossRef] [Medline]
    58. Grossman MR, Zak DK, Zelinski EM. Mobile apps for caregivers of older adults: Quantitative content analysis. JMIR Mhealth Uhealth 2018 Jul 30;6(7):e162 [FREE Full text] [CrossRef] [Medline]
    59. Prime Minister of Australia. 2019 Nov 25. Response to Aged Care Royal Commission Interim Report   URL: https://www.pm.gov.au/media/response-aged-care-royal-commission-interim-report [accessed 2020-01-06]
    60. International Organization for Standardization. 2017. Ageing societies.   URL: https://www.iso.org/committee/6810883.html [accessed 2020-01-06]


    Abbreviations

    CASP: Critical Appraisal Skills Programme
    CINAHL: Cumulative Index to Nursing and Allied Health Literature
    DCC&TN: Dementia Care Competency and Training Network
    IT: information technology
    MeSH: Medical Subject Headings
    MOOC: Massive Open Online Course
    NSW: New South Wales


    Edited by G Eysenbach; submitted 27.10.19; peer-reviewed by W Zhang, IV George, R Chan; comments to author 16.11.19; revised version received 28.11.19; accepted 15.12.19; published 22.01.20

    ©Anne Moehead, Kathryn DeSouza, Karen Walsh, Sabrina W Pit. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 22.01.2020.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.