Maintenance Note

On Friday, August 31, 2018 at 12:00 pm Eastern Time, JMIR will be completing a server migration to improve site stability and user experience. We expect to be back online Friday, August 31, 2018 at 5:00 pm Eastern Time. Should any problems arise our technical team will be using the weekend to resolve them, and users will be able to access our site by Sunday, September 2, 2018 at 1:00pm Eastern Time.

Who will be affected?

Advertisement

Citing this Article

Right click to copy or hit: ctrl+c (cmd+c on mac)

Published on 15.05.18 in Vol 20, No 5 (2018): May

Preprints (earlier versions) of this paper are available at http://preprints.jmir.org/preprint/10229, first published Feb 28, 2018.

This paper is in the following e-collection/theme issue:

    Original Paper

    An eHealth Capabilities Framework for Graduates and Health Professionals: Mixed-Methods Study

    1Research in Implementation Science and eHealth, Faculty of Health Sciences, The University of Sydney, Camperdown, Australia

    2Speech Pathology, Graduate School of Health, The University of Technology Sydney, Ultimo, Australia

    3Faculty of Health Sciences, The University of Sydney, Camperdown, Australia

    4School of Dentistry, The University of Sydney, Westmead, Australia

    5Sydney Local Health District, Camperdown, Australia

    6Faculty of Arts and Social Sciences, The University of Sydney, Camperdown, Australia

    7eHealth NSW, Chatswood, Australia

    8NSW Health Education Training Institute, Gladesville, Australia

    Corresponding Author:

    Melissa Brunner, BAppSc, MHlthSc

    Research in Implementation Science and eHealth

    Faculty of Health Sciences

    The University of Sydney

    Charles Perkins Centre

    Camperdown, 2006

    Australia

    Phone: 61 401190686

    Email:


    ABSTRACT

    Background: The demand for an eHealth-ready and adaptable workforce is placing increasing pressure on universities to deliver eHealth education. At present, eHealth education is largely focused on components of eHealth rather than considering a curriculum-wide approach.

    Objective: This study aimed to develop a framework that could be used to guide health curriculum design based on current evidence, and stakeholder perceptions of eHealth capabilities expected of tertiary health graduates.

    Methods: A 3-phase, mixed-methods approach incorporated the results of a literature review, focus groups, and a Delphi process to develop a framework of eHealth capability statements.

    Results: Participants (N=39) with expertise or experience in eHealth education, practice, or policy provided feedback on the proposed framework, and following the fourth iteration of this process, consensus was achieved. The final framework consisted of 4 higher-level capability statements that describe the learning outcomes expected of university graduates across the domains of (1) digital health technologies, systems, and policies; (2) clinical practice; (3) data analysis and knowledge creation; and (4) technology implementation and codesign. Across the capability statements are 40 performance cues that provide examples of how these capabilities might be demonstrated.

    Conclusions: The results of this study inform a cross-faculty eHealth curriculum that aligns with workforce expectations. There is a need for educational curriculum to reinforce existing eHealth capabilities, adapt existing capabilities to make them transferable to novel eHealth contexts, and introduce new learning opportunities for interactions with technologies within education and practice encounters. As such, the capability framework developed may assist in the application of eHealth by emerging and existing health care professionals. Future research needs to explore the potential for integration of findings into workforce development programs.

    J Med Internet Res 2018;20(5):e10229

    doi:10.2196/10229

    KEYWORDS



    Introduction

    Background

    Developing an eHealth-ready workforce [1,2] is becoming a key priority for addressing the complex challenges in health care globally [3,4]. Modern health care now incorporates a multitude of eHealth technologies that can be used for (1) monitoring, tracking, and informing health; (2) interacting for health such as using digital technologies to enable health communication among practitioners and between health professionals and clients or patients; and (3) data that enable health via collecting, managing, and using health data to improve outcomes [5]. eHealth has been recognized as pivotal in recent health reforms, with the potential to provide more efficient, cost-effective care with better outcomes [4,6,7]. As such, health services increasingly expect health professionals, new graduates, and experienced workforce alike, to be eHealth ready within practice across diverse digital health environments [4,8]. In turn, this is directing efforts to integrate eHealth education into clinical health degrees, specifically allied health, nursing, pharmacy, dentistry, and medical programs [1,2,9].

    Significant work has been conducted by Gray et al (2014) to identify the educational needs of health professionals necessary for an eHealth-capable workforce [9]. A key factor limiting the eHealth readiness of current and future health professionals is the lack of coordinated, formal education in the use of digital technologies in health [9]. The health practice context has become vastly more complex, mirroring changes in patients, the health system, and medical science, making it increasingly challenging for clinicians to safely and efficiently navigate health care [10]. As digital health becomes more widespread, so does the requirement for health professionals to be well versed in navigating and using such technologies [3]. eHealth education, therefore, needs to be coordinated to explain and explore eHealth in both current and future health care contexts, and to incorporate clear specifications as to the eHealth capabilities expected of the current and future health care workforce [9]. Research into, and the development of, coordinated approaches to eHealth curriculum design and implementation are necessary to effectively integrate eHealth capabilities among health graduates and to support the health care workforce in using eHealth technologies [3]. Consequently, specific eHealth education is required, even if students report to be competent and confident in using technology [11]. Such skills must not to be equated with technology proficiency, and most certainly not with information fluency [12]. Information literacy is defined in essence as “the ability to access, evaluate, and use information from a variety of sources” [13] and has become a critical skill for the present generation of students, and indeed for the 21st-century citizen [14,15]. For this to be achieved, a systematic approach to curriculum design and collaborative efforts from stakeholders are needed [9,16].

    Multiple issues need to be addressed when designing strategies for embedding eHealth into undergraduate curricula [4,9,16]. Drawing from research into teaching evidence-based medicine [17], eHealth would be more effectively taught (ie, demonstrate improved knowledge, skills, and attitudes) when it is integrated into clinical subjects rather than as an adjunct [16]. Education needs to improve awareness and understanding of the purpose of eHealth in practice contexts, as well as training in the use of technologies [4,9]. Competency frameworks in health informatics [18,19] provide valuable reference points in technical and informatics literacy for workforce development. eHealth-enabled health care, however, extends beyond informatics and requires professionals to integrate digital technologies into the health care and management processes [5]. As such, although health informatics competencies provide a strong foundation for the technical requirements of eHealth, the current and future health workforce must also receive education in the use of eHealth in practice to perform a range of professional functions, such as clinical decision making, patient empowerment, promotion of health and wellness, critical reflection and ethical decision making, and enabling new models of care [3,20]. Although such workforce eHealth competency frameworks are emerging for specific professions [21], limited understanding and agreement remain regarding the core eHealth competencies expected of tertiary graduates.

    Although the tertiary education of health professionals is traditionally framed around a set of specific competency standards decreed by professional associations that allow measurable behaviors to be observed and assessed [22], competency frameworks are most beneficial when the skills and practice are consistent across contexts, eg, in the assessment of a specific injury or illness [23]. eHealth practice, on the other hand, might look different in different contexts. For example, effective integration of mobile technologies in one context might involve using mobile devices to enhance the measurement and assessment of injury. In another context, clinicians could be using apps to remotely prescribe exercises and monitor patients’ physical activity. The fluidity in how eHealth skills are demonstrated across different contexts and the speed with which eHealth technologies evolve both clash with the rigidity and speed at which competencies and educational frameworks can be developed. As such, this suggests that competency standards may not be the most appropriate framing of eHealth skills and education [5,22,23].

    In contrast to competency, capability has been defined as “a holistic concept that describes how an individual or organization applies their ability in a confident manner to problems in new and unfamiliar circumstances as well as in familiar situations” [24]. As such, a capable workforce includes lifelong learners who are able to identify the need for change, adapt to familiar and novel situations and environments, and work collaboratively with other stakeholders to provide and potentially transform care [22,23]. Although capability has been described as being similar to competence, it in fact encompasses competence and extends beyond the technical skills implied by competence to emphasize the components of adaptability to change, lifelong learning, and self-efficacy [23-25]. As such, capability-informed frameworks address wider aspects of professionalism, focusing on supporting continuous development rather than assessment of a skill at a specific point in time [25].

    Objectives

    A capabilities approach is, therefore, able to inform and modify competency-based frameworks that better reflect the complexity of real-life environments as is found in the field of using eHealth for health care and wellness [22,25]. With this in mind, this study aimed to develop an eHealth capabilities framework based on current evidence and stakeholder perceptions of eHealth capabilities that are expected of workforce-ready tertiary health graduates. This study formed one component of a larger project, eHealthMap, which aimed to align the University’s approach to eHealth education with the best evidence and national expectations of workforce-ready graduates. A key focus of the study was to ensure alignment between the required capabilities of new graduates with workforce capability requirements. This led to a collaborative design approach between the University and the New South Wales Ministry of Health.


    Methods

    Triphasic Approach

    A 3-phase approach was used. This consisted of a literature review in the first phase, followed by qualitative studies to identify relevant themes, gain expert opinions, and raise consensus with key stakeholders in the second stage. In the third phase, data from the 2 previous stages were reviewed using a Delphi process to develop a framework of capability statements for guiding health curricula and measurement of workforce readiness in the use of eHealth technologies for health care. The project occurred between October 2016 and November 2017.

    Literature Review

    A systematic search for literature was undertaken during October 2016 to December 2016 to identify papers related to workforce readiness in the use of eHealth technologies. The databases and search engines used included CINAHL, Medline, ERIC, PsychINFO, Google Scholar, and Google. Databases and search engines were systematically searched for literature on eHealth competencies using combinations and variations of the following key search terms: eHealth, digital health, health professional, workforce readiness, graduate, student, higher education, capability statement, and competency. The fields of health, medicine, nursing, public health, allied health, pharmacy, psychology, physiotherapy, occupational therapy, speech language pathology, dentistry, paramedicine, social work, dietetics, nutrition, radiography, audiology, exercise and sports science, optometry, orthoptics, ophthalmology, and podiatry were included. Researchers also reached out to health and academic communities and reviewed Web pages from a number of professional societies and health organizations to identify the existence of other eHealth readiness competencies and capabilities.

    The search was limited to papers published after 2000 to maximize relevance to current clinical contexts and papers written in English. Research studies and reports were included if they (1) referred to readiness competencies or capabilities for students, graduates, or workforce; (2) referred to the use of digital technologies in health care; and (3) provided empirical support for the capability statements or framework. Included peer-reviewed and gray literature were then reviewed to identify existing core competencies or capability statements relating to eHealth readiness. Data were extracted by 1 author (author 2), analysis and synthesis [26] conducted independently by 2 authors (authors 1 and 2), with consensus provided by a third author as required (author 3).

    Focus Groups

    Focus groups with key informants from the health workforce and higher education were used to ascertain the perceived eHealth capabilities requirements of new graduates [27]. Purposeful sampling recruited 23 participants with significant expertise or vested interests in eHealth education, practice, or policy. Focus groups were conducted during a workshop held in February 2017 at the University of Sydney. Ethical approval was obtained from the University of Sydney Human Research Ethics Committee (Protocol No. 2016/811) before participant recruitment.

    The workshop included 2 focus group sessions: (1) a large focus group (n=23, 1 hour and 14 min) and (2) 4 small breakout focus groups (n=23, with 5-6 participants per group, 45 min). A semistructured script encouraged topic exploration and included the following questions: (1) What eHealth competencies do you expect health graduates to be able to demonstrate? and (2) Could you provide some examples of how these competencies are taught or applied in your organization? Focus group discussions were facilitated by the researcher team experienced in qualitative research methods. To stimulate discussions, participants were presented with evidence of core eHealth readiness competencies identified via the literature review. Of the focus group participants, 65% (15/23) were female, and 48% (11/23) were University of Sydney faculty representatives. Faculty representatives encompassed the fields of physiotherapy, speech pathology, psychology, nursing, dentistry, pharmacy, medicine, information technology and engineering, and mathematics and statistics. The balance of the participants (12/23, 52%) included broad representation from health services (9/23, 39%) and state and national government health agencies (3/23, 13%), including senior executives, clinicians, and senior health administrators. Participants also included recent health professional graduates (2/23, 9%) and 1 enrolled student from the University of Sydney. Interviews were transcribed verbatim and thematically analyzed [26] independently by 2 authors (authors 1 and 2). Codes were applied to the text of the transcripts, with themes systematically refined until saturation was achieved.

    Development of an eHealth Capability Framework

    The third phase involved identifying core capability statements by using the Delphi method [28] to refine and establish core eHealth capability statements. This was conducted over 4 iterations [29] during 2017, which allowed participants recruited from the focus groups (n=23) and via invitation to a wider group of key stakeholders (n=16) to systematically consider the capability statements as they evolved. Each round consisted of a meeting and follow-up email correspondence, promoting discussion and consensus between participants. A predetermined quorum of participants (n=12, ie, participation quorum of 50% based on the total number of participants recruited during the focus group phase, where N=23) was present for each round required to reach consensus, and each participant provided feedback throughout the process during meetings or via email. At the end of each round, feedback was incorporated into the capability framework and the revised statements presented for deliberation at the next meeting. Incorporation of feedback resulted in improved clarity and limited redundancy where similar capabilities were collapsed or condensed. Iterations continued until such time as consensus among the participants was achieved [29]. Following consensus agreement, endorsement from key academic and industry organizations was sought.


    Results

    Literature Review

    After duplicates were removed, the search revealed 92 relevant papers. Papers were excluded if they were opinion pieces, provided no empirical support, or reported no specific details regarding eHealth capabilities. This left a total of 30 papers included in this review. Emergent themes from the literature were identified (by authors 1 and 2) and refined through research team discussion at regular meetings.

    The health, activity, and participation issues identified within the literature were initially arranged into the following 6 categories: (1) health information management (n=24); (2) communication (n=19); (3) professionalism (n=24); (4) information systems and technologies (n=25); (5) patient focus (n=13); and (6) health analytics (n=14). Categorization of the 6 core domains as identified in the literature is presented in Table 1.

    Focus Groups

    Three overarching themes, each with multiple subthemes, emerged from the thematic analyses: (1) reinforce fundamental clinical capabilities, (2) acknowledge and adapt existing capabilities, and (3) introduce and provide opportunities for new learning. Themes and subthemes are presented in Table 2 and have been reported previously [20].

    Development of the eHealth Capability Framework

    At the completion of the fourth round, consensus was achieved. The final capability framework consisted of 4 overarching domains and 40 performance cues (Multimedia Appendix 1). The 4 upper-level eHealth capability statements describe what an eHealth-ready health graduate should be able to demonstrate (Table 3). The nature of these statements is such that more specific examples of what they might look like in different contexts are needed. These specific examples are provided in the form of performance cues in the framework. As these statements reflect expected capabilities among entry-level clinical positions, levels of mastery are not specified. This would be more appropriate at the curriculum level where educators consider how these capabilities might be developed and assessed in their specific disciplinary and educational context. Feedback from the focus groups indicated that, ideally, the eHealth capability framework would be integrated with relevant professional competency-based occupational standards. Throughout the development process, key stakeholders consistently recommended that the framework needed to incorporate the essential components of both safety and effective communication across all the 4 high-level capability statements.

    Table 1. Categorization of core domains of eHealth capability as identified in the literature review.
    View this table
    Table 2. Fundamental themes of eHealth capability identified from focus groups.
    View this table
    Table 3. Overarching domains of eHealth capabilities.
    View this table

    Discussion

    Principal Findings

    This study aimed to develop a framework of eHealth capabilities that could be used to inform development of higher education curricula for health students and professional development opportunities for the current health workforce. The framework resulting from our literature review, focus groups, and Delphi process describes 4 overarching domains of eHealth capabilities: (1) digital technologies, systems, and policies; (2) clinical practice and applications; (3) data analysis and knowledge creation; and (4) system and technology implementation. Each domain is supported by a capability statement that articulates what learners need to be able to know and do to demonstrate achievement of this capability. Examples of these knowledge and behaviors are provided in the form of performance cues. The key components of the eHealth capability framework are provided in Multimedia Appendix 1.

    The systematic process for developing the framework was necessary for the successful integration of eHealth and continuous quality improvement at all levels in the health system [9]. Critical to long-term implementation and adoption of the framework is the capability approach, which contrasts with a competency approach by recognizing the complexity and ever-changing nature of eHealth in practice. The idea of capability not only addresses complexity and evolving contexts but also incorporates the aspects of critical analysis and ethical practice in health care [25]. Furthermore, this research identified that eHealth capabilities should be integrated with clinical competencies and be sufficiently flexible so that they can be adapted to specific contexts.

    Given the inherent nature of eHealth as being innovative and transformative, it is critical that we enable our health workforce to be suitably prepared and adaptable. Tertiary education of future health graduates needs to move from a focus on technical skills to encompass broader eHealth capabilities, such as professional competencies and attributes of an adaptable, improvement-minded, and innovative workforce. The eHealth capabilities framework extends beyond this technical proficiency to include the integration of technology into current practice and demonstrating a strong ethos around lifelong learning and transforming care. By re-evaluating traditional resources used in education and incorporating knowledge around eHealth capabilities, educational curriculum will provide greater opportunities to build capability in working with systems undergoing digital transformation [3]. Consistent with previous research [4,9], we identified the need for educational curriculum to reinforce existing eHealth capabilities, adapt existing capabilities to make them transferable to novel eHealth contexts, and introduce new learning opportunities for interactions with technologies within education and practice encounters. In essence, the capability framework developed assists in bridging the gap between academia and the application of digital health by emerging and existing health care professionals.

    This framework is not a curriculum; it is a starting point to guide curriculum development and redesign. Furthermore, the framework is intended to stimulate discussion with industry stakeholders regarding workforce capability with regard to eHealth in practice. As a guide, it will enable further development of both curriculum and competency evaluation [48]. The eHealth capabilities framework is not intended to set a rigid curriculum for eHealth education, but rather to provide a key resource and common standards for the review, development, and alignment of profession-specific curricula to ensure high-quality and consistent student learning experiences. Attaining capability will ideally involve embedding eHealth within problem-based, case-based, and practice-based learning experiences and incorporate digital simulations and codesign projects. This work emphasizes core professional practice principles that underpin all activities involving eHealth, including quality and safety, consumer-centeredness, critical thinking, and evidence-based practice. As technologies evolve and practices involving eHealth grow, it is critical to maintain attention to these core principles. The inclusion of performance cues within the framework provides a starting point to describe how successful learning might be demonstrated. These are intended to guide the development of assessments that support the achievement of specific learning outcomes and activities of the discipline-specific curriculum.

    It was clear that there is a need to create an eHealth capability framework that accurately reflected the roles and work contexts in the digital age. In addition to moving beyond a focus on the technical skills, this included offering theoretical and practical opportunities for health graduates to integrate eHealth into practice, eg, using technologies to provide new models of care and facilitate consumer empowerment, or to use routinely collected digital health data to inform practice. One of the key challenges for education in this space is the ability for curriculum to adequately address the divide between digitally capable individuals and their capacity to apply digital skills to clinical situations that often use archaic systems [49]. Although current students may now use digital technologies in their everyday lives, they might have limited knowledge of the opportunities and issues that technology can bring to the health care landscape [49]. As a result, students require tailored opportunities to ensure that they develop or translate skills and knowledge to effectively practice in evolving digital workplaces.

    Limitations

    In this study, we did not aim to achieve data saturation as this was beyond the scope of this initial work. The participants in this study brought with them a breadth of experience and representation of multiple health professions, but the lack of consumer representation was a study limitation. It is likely that future research incorporating additional participants working across a wider range of contexts, and is inclusive of diverse health care consumers, may provide additional insights into eHealth-enabled interdisciplinary practice. Further research exploring the implications for the existing health care workforce is also warranted, with a focus on identifying the potential relevance and impact of the capability statements on policy and practice, including recruitment, professional development, performance management, and systems improvement activities. Building on the results of this study, the University is undertaking a curriculum mapping process with a commitment to the development of high-quality teaching and learning activities and resources. This will include an analysis of clinical placement experiences, work-based training, and workplace orientation programs to further identify and address gaps in the preparation of our health graduates and workforce for eHealth contexts. We further excluded from the discussion implications of automation and computerization on the actual workforce and their predicted displacement [50]. Thus, the impact artificial intelligence might have on the health care delivery system was not in the scope of this research.

    Practical Implications

    This framework has direct implications for curriculum redevelopment in health education and professional development opportunities for the current health workforce. The framework could be used to assess which eHealth capabilities are currently being taught in health profession degrees, and how. Exemplars of effective eHealth education could be collated to form resources and professional development for health educators across the sector. Importantly, in using the framework to map current curricular, educators and course coordinators can identify the capabilities not addressed in their programs at present. This provides opportunity for revising the curriculum to better prepare graduates.

    For the current health workforce, the framework requires further development. At present, the capabilities reflect the knowledge and skills required of a graduate. Further research and development are required to articulate intermediate and advanced levels of capability across the 4 domains of the framework. Once established, this more comprehensive framework could guide professional development opportunities and self-reflection or self-assessment for practicing health professionals.

    Conclusions

    This paper describes the foundational level of eHealth capability expected of tertiary health students at graduation and as they enter the health workforce. Ideally, the eHealth capability framework will inform how tertiary health programs deliver and assess essential eHealth education. The results of this study will inform a cross-faculty eHealth curriculum that aligns with workforce expectations and will be of interest to professional associations, health services, and organizations. Future research needs to explore the potential for integration of findings into workforce development programs, particularly with consideration of intermediate and advanced levels of capability in collaboration with workforce and industry stakeholders.

    Acknowledgments

    All authors listed in this publication conducted the research as part of the salaried position supported by their hiring organization. This research was supported by a University of Sydney Strategic Education grant.

    Conflicts of Interest

    None declared.

    Multimedia Appendix 1

    eHealth capability framework.

    PDF File (Adobe PDF File), 724KB

    References

    1. Deaprtment of Health and Aging, Australian Government. 2011. The eHealth Readiness of Australia's Allied Health Sector   URL: http:/​/www.​health.gov.au/​internet/​publications/​publishing.nsf/​Content/​ehealth-readiness-allied-toc/​$FILE/​Allied%20Health%20ehealth%20readiness%20survey%20report.​pdf [accessed 2018-02-07] [WebCite Cache]
    2. Health Workforce Australia. 2013 Oct. Health Information Workforce Report   URL: https://www.aims.org.au/documents/item/401 [accessed 2018-02-07] [WebCite Cache]
    3. Gray K. Public Health Platforms: An Emerging Informatics Approach to Health Professional Learning and Development. J Public Health Res 2016 Apr 26;5(1):665 [FREE Full text] [CrossRef] [Medline]
    4. Li S, Bamidis PD, Konstantinidis ST, Traver V, Car J, Zary N. Setting priorities for EU healthcare workforce IT skills competence improvement. Health Informatics J 2017 Apr 01:1460458217704257. [CrossRef] [Medline]
    5. Shaw T, McGregor D, Brunner M, Keep M, Janssen A, Barnet S. What is eHealth (6)? Development of a Conceptual Model for eHealth: Qualitative Study with Key Informants. J Med Internet Res 2017 Oct 24;19(10):e324 [FREE Full text] [CrossRef] [Medline]
    6. Jolly R. Parliament of Australia. 2011. The e health revolution - easier said than done   URL: https:/​/www.​aph.gov.au/​about_parliament/​parliamentary_departments/​parliamentary_library/​pubs/​rp/​rp1112/​12rp03 [accessed 2018-02-07] [WebCite Cache]
    7. New South Wales Health. 2016. eHealth Strategy for NSW Health 2016-2026   URL: http://www.health.nsw.gov.au/eHealth/Documents/eHealth-Strategy-for-NSW-Health-2016-2026.pdf [accessed 2018-02-07] [WebCite Cache]
    8. Queensland Government Metro South Health. 2016. Digital Hospital   URL: https://metrosouth.health.qld.gov.au/princess-alexandra-hospital/digital-hospital [accessed 2018-02-07] [WebCite Cache]
    9. Gray K, Dattakumar A, Maeder A, Butler-Henderson K, Chenery H. clinicalinformaticseducation.pbworks. 2014. Advancing Ehealth education for the clinical health professions   URL: http://clinicalinformaticseducation.pbworks.com/w/file/fetch/74500403/PP10_1806_Gray_report_2014.pdf [accessed 2018-02-07] [WebCite Cache]
    10. Obermeyer Z, Lee TH. Lost in Thought - The Limits of the Human Mind and the Future of Medicine. N Engl J Med 2017 Sep 28;377(13):1209-1211 [FREE Full text] [CrossRef] [Medline]
    11. Lam M, Hines M, Lowe R, Nagarajan S, Keep M, Penman M, et al. Preparedness for eHealth: Health Sciences Students’ Knowledge, Skills, and Confidence. JITE:Research 2016;15:305-334. [CrossRef]
    12. Oblinger DG. Growing up with Google: What it means to education. Emerging technologies for learning 2008 Mar:11-29 [FREE Full text]
    13. Doyle CS. US Department of Education. 1992. Outcome Measures for Information Literacy within the National Education Goals of 1990. Final Report to National Forum on Information Literacy   URL: https://files.eric.ed.gov/fulltext/ED351033.pdf [accessed 2018-02-07] [WebCite Cache]
    14. Hancock VE. US Department of Education. 1993. Information Literacy for Lifelong Learning   URL: https://files.eric.ed.gov/fulltext/ED358870.pdf [accessed 2018-02-07] [WebCite Cache]
    15. Duggan F. The Guardian. 2013. Why universities should acquire - and teach - digital literacy   URL: https://www.theguardian.com/education/2013/apr/23/universities-should-teach-digital-literacy [accessed 2018-02-07] [WebCite Cache]
    16. Clark J, Baker B, Baker D. Getting eHealth into basic nursing education: report of the RCN information in nursing project. Stud Health Technol Inform 2009;146:534-539. [Medline]
    17. Khan KS, Coomarasamy A. A hierarchy of effective teaching and learning to acquire competence in evidenced-based medicine. BMC Med Educ 2006 Dec 15;6:59 [FREE Full text] [CrossRef] [Medline]
    18. Australian Nursing and Midwifery Federation. 2015. National informatics standards for nurses and midwives   URL: http://anmf.org.au/documents/National_Informatics_Standards_For_Nurses_And_Midwives.pdf [accessed 2018-02-07] [WebCite Cache]
    19. Health Informatics Society of Australia. 2013 Dec. Certified Health Informatician Australasia (CHIA) Health Informatics Competencies Framework Edition 1   URL: http:/​/www.​healthinformaticscertification.com/​wp-content/​uploads/​2016/​02/​CHIA-competencies-Framework_FINAL.​pdf [accessed 2018-02-07] [WebCite Cache]
    20. McGregor D, Keep M, Brunner M, Janssen A, Quinn D, Avery J, et al. Preparing E-Health Ready Graduates: A Qualitative Focus Group Study. Stud Health Technol Inform 2017;239:91-96. [Medline]
    21. Academy of Medical Royal Colleges. 2011 Jun. eHealth Competency Framework: Defining the role of the expert clinician   URL: http://www.aomrc.org.uk/wp-content/uploads/2016/05/EHealth_Competency_Framework_0611.pdf [accessed 2018-02-07] [WebCite Cache]
    22. O'Connell J, Gardner G, Coyer F. Beyond competencies: using a capability framework in developing practice standards for advanced practice nursing. J Adv Nurs 2014 Dec;70(12):2728-2735. [CrossRef] [Medline]
    23. Phelps R, Hase S, Ellis A. Competency, capability, complexity and computers: exploring a new model for conceptualising end-user computer education. Br J Educ Technol 2005 Jan;36(1):67-84. [CrossRef]
    24. Townsend P, Cairns L. Developing the Global Manager Using a Capability Framework. Management Learning 2016 Aug 19;34(3):313-327. [CrossRef]
    25. Lester S. Professional standards, competence and capability. HESWBL 2014 Feb 11;4(1):31-43. [CrossRef]
    26. Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol 2006 Jan;3(2):77-101. [CrossRef]
    27. Creswell JW. Research design: Qualitative, quantitative, and mixed methods approaches. Thousand Oaks, California: Sage Publications; 2013.
    28. Dalkey N, Helmer O. An Experimental Application of the DELPHI Method to the Use of Experts. Manag Sci 1963 Apr;9(3):458-467. [CrossRef]
    29. Skulmoski GJ, Hartman FT, Krahn J. The Delphi method for graduate research. JITE 2007;6:1-21 [FREE Full text]
    30. Archer N. Project Management Characteristics and Education for eHealth Students. EJHI 2011;6(4):e30 [FREE Full text]
    31. Barakat A, Woolrych RD, Sixsmith A, Kearns WD, Kort HS. eHealth Technology Competencies for Health Professionals Working in Home Care to Support Older Adults to Age in Place: Outcomes of a Two-Day Collaborative Workshop. Med 2 0 2013 Sep 05;2(2):e10 [FREE Full text] [CrossRef] [Medline]
    32. Blake A, Kaye R. A Digital Solution to the EU/US Skilled eHealth/Health IT Worker Shortage: The HITCOMP Tool and Repository. : EU-US Workforce Development Workgroup; 2015 Presented at: eHealth Week; May 11-13, 2015; Riga, Latvia   URL: https:/​/www.​eiseverywhere.com/​file_uploads/​ace6e27c257b052431e3e7d7b16a23a3_BLAKE_ADigitalSolutiontotheEU_USSkilledeHealthHealthITWorkerShortage.​pdf
    33. Blak R, Shaw T, Blake A, Hübner U, Kaye R, Schug S, et al. Healthcare Information and Management Systems Society Foundation. 2016. EU-US eHealth Works to Improve Global Workforce Development   URL: http://www.himss.org/sites/himssorg/files/eu-us-ehealth-workforce-development.pdf [accessed 2018-02-07] [WebCite Cache]
    34. Britt RK, Hatten KN. The Development and Validation of the eHealth Competency Scale: A Measurement of Self-Efficacy, Knowledge, Usage, and Motivation. Tech Commun Q 2016 Apr 07;25(2):137-150 [FREE Full text] [CrossRef]
    35. Connoly R. Ethical leadership: Training, education, and eLearning to promote ethics as a core competency within the eHealth Program. Ottawa, Canada: Royal Roads University; 2008.
    36. Crawford A, Sunderji N, López J, Soklaridis S. Defining competencies for the practice of telepsychiatry through an assessment of resident learning needs. BMC Med Educ 2016 Jan 26;16:28 [FREE Full text] [CrossRef] [Medline]
    37. Department of Health and Human Services, State of Victoria. 2016. Allied health: credentialling, competency and capability framework (revised edition): Driving effective workforce practice in a changing health environment   URL: https://www2.health.vic.gov.au/Api/downloadmedia/%7B426782F8-57EB-4DD9-B9CF-25CB92256A02%7D [accessed 2018-02-07] [WebCite Cache]
    38. Gilstad H. Toward a comprehensive model of eHealth literacy. 2014 Presented at: Practical Aspects of Health Informatics 2014; May 19, 2014; Trondheim, Norway   URL: http://ceur-ws.org/
    39. Gray K, Choo D, Butler-Henderson K, Whetton S, Maeder A. Clinical Informatics Education. 2015 Aug. An online health informatics subject for clinical health profession students in Australian universities: an extension of the OLT Priority Project Coordinated interprofessional curriculum renewal for ehealth capability in clinical health profession degrees. Final report   URL: http:/​/clinicalinformaticseducation.​pbworks.com/​w/​file/​fetch/​103478171/​Clinical%20Informatics%20and%20eHealth%20OLT%20Final%20Report%202015.​pdf [accessed 2018-02-07] [WebCite Cache]
    40. Gray K, Choo D, Butler-Henderson K, Whetton S, Maeder A. Clinical Informatics Education. 2015. Clinical Informatics and eHealth: Open Curriculum   URL: http:/​/clinicalinformaticseducation.​pbworks.com/​w/​file/​fetch/​97197057/​Clinical%20Informatics%20and%20eHealth%20OLT%20Curriculum%20V1.​1.​pdf [accessed 2018-02-07] [WebCite Cache]
    41. Honey ML, Skiba DJ, Procter P, Foster J, Kouri P, Nagle LM. Nursing Informatics Competencies for Entry to Practice: The Perspective of Six Countries. Stud Health Technol Inform 2017;232:51-61. [Medline]
    42. Klecun E, Lichtner V, Cornford T. e-Literacy in health care. Stud Health Technol Inform 2014;205:843-847. [Medline]
    43. Procter PM. Ubiquitous Adoption of Innovative and Supportive Information and Communications Technology Across Health and Social Care Needs Education for Clinicians. Stud Health Technol Inform 2017;235:358-362. [Medline]
    44. Röhrig R, Stausberg J, Dugas M, GMDS project group, Medical Informatics Education in Medicine. Development of national competency-based learning objectives “Medical Informatics” for undergraduate medical education. Methods Inf Med 2013;52(3):184-188. [CrossRef] [Medline]
    45. van Houwelingen CT, Moerman AH, Ettema RG, Kort HS, Ten Cate O. Competencies required for nursing telehealth activities: A Delphi-study. Nurse Educ Today 2016 Apr;39:50-62 [FREE Full text] [CrossRef] [Medline]
    46. Värri A, Blake R, Roberts J, Fenton S, Cleary M, Zacks S, et al. Transatlantic collection of health informatics competencies. FinJeHeW 2016 May 23;8(2-3):127-136 [FREE Full text]
    47. Friedman CP, Donaldson KM, Vantsevich AV. Educating medical students in the era of ubiquitous information. Med Teach 2016 May;38(5):504-509 [FREE Full text] [CrossRef] [Medline]
    48. Jogerst K, Callender B, Adams V, Evert J, Fields E, Hall T, et al. Identifying interprofessional global health competencies for 21st-century health professionals. Ann Glob Health 2015;81(2):239-247 [FREE Full text] [CrossRef] [Medline]
    49. Biggins D, Holley D, Zezulkova M. Digital Competence and Capability Frameworks in Higher Education: Importance of Life-long Learning, Self-Development and Well-being. EAI Endorsed Transactions on e-Learning 2017;4(13):e1 [FREE Full text]
    50. Susskind R, Susskind D. Harvard Business Review. 2016. Technology Will Replace Many Doctors, Lawyers, and Other Professionals   URL: https://hbr.org/2016/10/robots-will-replace-doctors-lawyers-and-other-professionals [accessed 2018-02-07] [WebCite Cache]


    Abbreviations

    eHealth: electronic health


    Edited by G Eysenbach; submitted 28.02.18; peer-reviewed by PRG Cunningham, P Jimenez; comments to author 28.03.18; revised version received 09.04.18; accepted 09.04.18; published 15.05.18

    ©Melissa Brunner, Deborah McGregor, Melanie Keep, Anna Janssen, Heiko Spallek, Deleana Quinn, Aaron Jones, Emma Tseris, Wilson Yeung, Leanne Togher, Annette Solman, Tim Shaw. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 15.05.2018.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.