Published on in Vol 23, No 1 (2021): January

Preprints (earlier versions) of this paper are available at, first published .
Asynchronous Distance Learning of the National Institutes of Health Stroke Scale During the COVID-19 Pandemic (E-Learning vs Video): Randomized Controlled Trial

Asynchronous Distance Learning of the National Institutes of Health Stroke Scale During the COVID-19 Pandemic (E-Learning vs Video): Randomized Controlled Trial

Asynchronous Distance Learning of the National Institutes of Health Stroke Scale During the COVID-19 Pandemic (E-Learning vs Video): Randomized Controlled Trial

Original Paper

1Division of Anaesthesiology, Department of Anaesthesiology, Clinical Pharmacology, Intensive Care and Emergency Medicine, Geneva University Hospitals and Faculty of Medicine, Geneva, Switzerland

2Genève TEAM Ambulances, Geneva, Switzerland

3Stroke Center, Department of Neurology, Geneva University Hospitals and Faculty of Medicine University of Geneva, Geneva, Switzerland

4Division of Emergency Medicine, Department of Anaesthesiology, Clinical Pharmacology, Intensive Care and Emergency Medicine, Geneva University Hospitals and Faculty of Medicine, Geneva, Switzerland

5Cognitive Neurology Unit, Neurology, Department of Clinical Neuroscience, Geneva University Hospital and Faculty of Medicine, Geneva, Switzerland

6Unit of Development and Research in Medical Education, Faculty of Medicine, University of Geneva, Geneva, Switzerland

Corresponding Author:

Laurent Suppan, MD

Division of Emergency Medicine

Department of Anaesthesiology, Clinical Pharmacology, Intensive Care and Emergency Medicine

Geneva University Hospitals and Faculty of Medicine

Rue Gabrielle-Perret-Gentil 4

Geneva, 1211


Phone: 41 795532579


Background: The COVID-19 pandemic has considerably altered the regular medical education curriculum while increasing the need for health care professionals. Senior medical students are being incrementally deployed to the front line to address the shortage of certified physicians. These students, some of whom will be fast-tracked as physicians, may lack knowledge regarding the initial management of time-critical emergencies such as stroke.

Objective: Our aim was to determine whether an e-learning module could improve asynchronous distance knowledge acquisition of the National Institutes of Health Stroke Scale (NIHSS) in senior medical students compared to the traditional didactic video.

Methods: A randomized, data analyst–blinded web-based trial was conducted at the University of Geneva Faculty of Medicine between April and June 2020. Fifth year medical students followed a distance learning path designed to teach the NIHSS. The control group followed the traditional didactic video created by Patrick Lyden, while the e-learning group followed the updated version of a previously tested, highly interactive e-learning module. The main outcome was the score on a 50-question quiz displayed upon completion of the learning material. The difference in the proportion of correct answers for each specific NIHSS item was also assessed.

Results: Out of 158 potential participants, 88 started their allocated learning path and 75 completed the trial. Participants who followed the e-learning module performed better than those who followed the video (38 correct answers, 95% CI 37-39, vs 35 correct answers, 95% CI 34-36, P<.001). Participants in the e-learning group scored better on five elements than the video group: key NIHSS concepts (P=.02), the consciousness – global item (P<.001), the facial palsy item (P=.04), the ataxia item (P=.03), and the sensory item (P=.04).

Conclusions: Compared to the traditional didactic video, a highly interactive e-learning module enhances asynchronous distance learning and NIHSS knowledge acquisition in senior medical students.

J Med Internet Res 2021;23(1):e23594



The swift strengthening of public health policies in the context of the COVID-19 crisis has wrought deep changes in the regular medical education curricula of many countries [1-4] while also increasing the need for health care professionals, including physicians. Senior medical students are being incrementally used on the front lines to address the shortage of these professionals [5,6], and other students may soon be required to follow suit [7]. Accelerated graduation procedures have also been described in some regions [8]. Senior medical students as well as some of these fast-tracked physicians may lack knowledge regarding the initial management of specific emergencies such as stroke. Stroke is a time-critical emergency that must be treated swiftly to improve functional and vital prognoses [9]; however, disruptions in acute stroke pathways have been described in the wake of the COVID-19 pandemic [10]. The National Institutes of Health Stroke Scale (NIHSS) is widely used to assess stroke victims [11], and senior medical students as well as junior residents should be familiar with its application.

Traditional classroom or bedside teaching can be difficult to conduct in certain situations, such as a pandemic [12-14]. Many universities have strived to increase distance learning capabilities, thereby highlighting the potential benefits of electronic learning (e-learning) [15-17]. E-learning is a generic term that includes many types of technologically enhanced learning materials [18-20]. Asynchronous distance learning using these methods has yielded mixed results, probably due to differences in the quality of the content and the mode of delivery [21].

Since the release of Patrick Lyden’s didactic video in 1994 [22], the development of NIHSS teaching material has been rather limited. We have recently shown that compared to this didactic video, a highly interactive e-learning module improved NIHSS knowledge acquisition in paramedics [23]. We defined this module as “highly interactive” because it uses multiple learning mechanics to promote interaction and engagement. Among these mechanics, preventing content skipping [24] and providing feedback tailored to the user's answer were the most prominent [25]. Branching logic was extensively used to create this feedback.

This first study was performed with the participants present at the study site; therefore, they could immediately access technical support if needed. Moreover, although most results favored the use of the e-learning module, the control group was better at scoring the ataxia element than the e-learning group. Although video extracts were used within the e-learning module to demonstrate the assessment of almost all NIHSS items, the chapter regarding the ataxia element did not contain any video extracts. We therefore hypothesized that systematically embedding videos could improve NIHSS learning acquisition, and we updated the module accordingly.

Given the need for social distancing during the COVID-19 pandemic, our goal was to compare medical students’ asynchronous distance learning of the NIHSS using two different teaching tools: the gold standard didactic video and the updated version of our e-learning module.

Study Design and Setting

We performed a randomized, controlled, data analyst–blinded, web-based trial following the CONSORT-EHEALTH guidelines and incorporating relevant elements from the Checklist for Reporting Results of Internet E-Surveys (CHERRIES) checklist [26,27]. The study took place between April 28 and June 8, 2020, in Geneva, Switzerland. Fifth year medical students at the University of Geneva Faculty of Medicine (UGFM) were invited to take part in this trial on a voluntary basis.

Standard Protocol Approvals, Registrations, and Participant Consent

Because the study included no patients and as no health outcomes were recorded, trial registration was not required according to the International Committee of Medical Journal Editors guidelines. Although the participants were not part of a vulnerable group according to Swiss federal law on human research [28], we filed a jurisdictional enquiry, and the regional ethics committee issued a “Declaration of no objection” (Req 2020-00474). The study was also approved by the Board of the Teaching Committee of the UGFM. Informed consent was gathered electronically.


After gathering the necessary authorizations, the UGFM students’ secretary transmitted the exact number of fifth year medical students to MS, who performed a 1:1 computer assisted randomization without having access to any other data regarding the students. MS then created specific identifiers that were transmitted back to the UGFM along with a mailing template Multimedia Appendix 1. The UGFM staff were therefore unable to determine the students’ allocations or results. In addition, we were prevented from determining the students’ identities.

The students were informed of the goals of the investigation, were given information regarding data security and anonymization procedures, and were supplied with the email addresses of three investigators to allow them to ask further questions. Students who elected to browse the website were provided with additional information as well as with a link to a full 4-page consent form in PDF format that they could either print or save. Using their identifiers to log into the site was considered as acceptance to participate in the study. All participants were free to withdraw at any time. No financial incentive was provided.

Web-Based Platform and Learning Material

We created a specific web-based platform under the Joomla 3.9 content management system (Open Source Matters) [29]. The control was Patrick Lyden’s original video, which was subtitled in French [22]. The experiment used version 21 of our e-learning module, which was developed using Articulate Storyline 3 (Articulate Global). This software enables the creation of many types of interactive content, including gamified modules and serious games [30,31], which can be accessed on regular computers as well as on smartphones and tablets.

The e-learning module contains 16 independent chapters. The first chapter is the introduction, which is automatically displayed when the module is launched. Prevention of content skipping is the first learning mechanic used in the module, and it already appears in the introductory slides (Figures 1-3) [24].

A table of contents is displayed as soon as the user has completed the introduction. The user can then choose to review the introduction or to access any other chapter apart from the summary (Figure 4).

Figure 1. Prevention of content skipping. The user cannot click on the Cliquez ici pour continuer (Click here to continue) button until both blue buttons have been clicked.
View this figure
Figure 2. The user has clicked on one of the two buttons, and the learning content is now displayed in a lightbox. The Cliquez ici pour continuer (Click here to continue) button, which is slightly visible in the background, is still grey; therefore, it is inactive.
View this figure
Figure 3. Both buttons have been clicked, and the user has seen both lightboxed slides. The Cliquez ici pour continuer (Click here to continue) button has thus been activated and is now colored blue.
View this figure
Figure 4. Table of contents of the e-learning module. The Résumé (Summary) can only be accessed once all the other chapters have been completed.
View this figure

Thirteen chapters are used to explain each specific NIHSS item (3 chapters are used to cover the first item, “consciousness,” which is divided into 3 elements). Although the chapters are numbered consecutively according to the NIHSS scoring logic, the user can freely elect the order in which to follow the chapters.

All chapters include at least two learning mechanics. First, each chapter begins by displaying the NIHSS score specific to the scoring item, and users are once again prevented from skipping content, as they must click on each numbered button to discover the score (Figure 5).

The second learning mechanic is linked to the use of subtitled videos. Video extracts are shown to the user, who must correctly score the NIHSS item (Figure 6). This version of the module contains video extracts in all chapters, including for the items related to dysarthria, level of consciousness – global, and level of consciousness – questions. In the previously studied version of our module (version 20), there were no video extracts for these items [23].

Feedback is provided for each question [25]: if the answer is incorrect, a clue is given (Figure 7), and the user has the opportunity to review the NIHSS item scoring.

If the answer was correct, feedback is also given to reinforce the message (Figure 8).

Specific interactions were designed to further illustrate particular elements, such as visual field deficits (Figure 9) or extinction and inattention (Figure 10).

Figure 5. Prevention of content skipping. The user cannot click on the Continuer (Continue) button until all the blue buttons have been clicked.
View this figure
Figure 6. Video-based question. The user must choose the correct score for the patient displayed in the video before clicking on Valider la réponse (Validate the answer).
View this figure
Figure 7. Wrong answer feedback and clue. The user can choose either to try again (Réessayer) or to review the scoring specific to this item by clicking Cliquez ici pour afficher l’échelle (Click here to display the scale).
View this figure
Figure 8. Positive feedback after a correct answer.
View this figure
Figure 9. Interactive explanation of visual field deficit.
View this figure
Figure 10. Animation used to explain extinction and inattention.
View this figure

The user can choose to exit the module at any time, as a prompt will be shown to allow the user to either resume the module or reset it. Before the summary chapter can be activated, users must complete a 14th chapter, which details the “coma score.”

This e-learning module, along with its previously studied iteration, can be accessed freely on the internet [32].

Study Sequence

Immediately after login, the medical students learned which group they had been allocated to and were asked a first set of 6 questions displayed over a single page. Upon answering these questions, they could then access the learning material. No time limit was applied apart from the study end date (June 8, 2020). Once the learning material was completed, students were allowed to proceed to a 50-question quiz. This quiz was identical for all participants and contained five questions related to basic NIHSS concepts, followed by the clinical evaluation of 3 patients taken from Patrick Lyden’s certification videos. The NIHSS elements were displayed and scored in sequence according to the NIHSS scoring logic. After finalizing the quiz, participants were given their overall score as well as the possibility to review all questions at will and were shown their answers along with the correct answers. Then, 4 questions, based on a 5-point Likert scale, were asked to assess secondary outcomes, such as satisfaction. Students were finally given access to both the video and the e-learning module to discover the other teaching modality and/or to review the one they had just followed.


The primary outcome of the study was the proportion of correct answers to the 50-question quiz. Secondary outcomes were the proportion of correct answers for each specific NIHSS item, user satisfaction, perceived adequacy of the time needed to complete the course, perceived difficulty of the course, probability that the participant would recommend the course, and whether the learning path had been completed over multiple days.

Data Collection and Curation

Data were securely stored on an encrypted MariaDB 5.5.5 database (MariaDB Foundation) located on a Swiss server before being extracted in comma-separated values (CSV) format. We used STATA (StataCorp) for data curation and anonymization.

Statistical Analysis

STATA 15.1 was used by L Stuby for statistical analysis. Incomplete answers to the 50-question quiz were not analyzed.

Normality was assessed by graphical evaluation and, if in doubt, we used the Shapiro-Wilk test. We applied the Fisher exact test to categorical variables and the Student t test or the Mann-Whitney U test to continuous variables according to normality. We considered a 2-sided P value <.05 as significant.

We used a convenience sample and calculated the power post hoc. We defined 4 sensitivity analyses a priori according to whether the participant had prior knowledge of the NIHSS, had already followed a specific NIHSS course, had worked in either an intensive care unit or in a neurology or neurosurgery ward for more than 3 months, or had completed the learning path over multiple days. This was defined as more than 12 hours elapsed between initiation and completion of the course.

Finally, we performed univariate followed by multivariable linear regression to look for possible confounding factors.

Data Availability

Our curated data file is available on Mendeley Data [33].

Out of 158 potential participants, 75 (47.5%) completed the trial (Figure 11). Table 1 details their characteristics.

After the first mailing (April 28, 2020), 21 students completed the trial. The first reminder (May 11, 2020) led 29 more students to complete the course, while another 25 participated after the second and last reminder (May 18, 2020).

Figure 11. Study flowchart.
View this figure
Table 1. Participant data (N=75).

Video group (n=34)E-Learning group (n=41)
Age, median (Q1-Q3)24 (23-25)24 (23-24)
Prior knowledge of NIHSSa application, n (%)3 (9)3 (7)
Specific NIHSS course followed, n (%)

4 (12)

2 (5)

E-learning NIHSS course followed, n (%)0 (0)0 (0)
Had worked in intensive care unit or neurology ward, n (%)0 (0)1 (2)
Missing data, n (%)2 (6)2 (5)

aNIHSS: National Institutes of Health Stroke Scale.

Participants who followed the e-learning module performed better than those who followed the video (38 correct answers, 95% CI 37-39, vs 35, 95% CI 34-36, P<.001). Participants in the e-learning group scored better on five elements than the video group: key NIHSS concepts (P=.02), the consciousness – global item (P<.001), the facial palsy item (P=.04), the ataxia item (P=.03) and the sensory item (P=.04). There was no such effect in the video group. Detailed results are shown in Table 2.

The rate of “very satisfied” participants was higher in the e-learning group (14/35, 40%; 95% CI 24%-56%) versus the video group (4/26, 15%; 95% CI 5%-25%, P=.02) (Figure 12).

Table 2. Quiz results.
ItemVideo group (n=34)E-Learning group (n=41)P value
Overall score, mean (SD)35 (3)38 (3)<.001
Overall score, 95% CI34-3637-39N/Aa
Detailed results by item, median (Q1-Q3)

Key NIHSSb concepts5 (4-5)5 (5-5)


Consciousness – Global2 (2-2)3 (2-3)<.001

Consciousness – Questions3 (2-3)3 (3-3).70

Consciousness – Commands2 (2-3)3 (2-3).06

Gaze2 (2-3)3 (2-3).34

Visual2 (2-2)2 (2-2).23

Facial Palsy1 (0-2)2 (1-2).04

Motor arm4 (4-5)5 (4-5).17

Motor leg5 (4-6)5 (4-5).23

Ataxia1 (1-1)1 (1-2).03

Sensory3 (2-3)3 (3-3).04

Language1 (1-2)1 (1-1).63

Dysarthria2 (2-2)2 (2-2).07

Extinction and inattention2 (2-3)2 (2-3).14

aN/A: not applicable.

bNIHSS: National Institutes of Health Stroke Scale.

Figure 12. Satisfaction of the participants in the e-learning and video groups regarding the learning method.
View this figure

Although the precise total learning time dedicated to either method could not be assessed due to the study design, no statistical difference regarding the perceived duration of the course was identified (80% (28/35) adequate in the e-learning group, 95% CI 67%-93%, vs 65% (17/26) 95% CI 47%-83%, P=.25). There was no significant difference regarding the perceived difficulty of the course, as 84% (16/19) (95% CI 68%-100%) found it “easy or very easy” in the e-learning group versus 53% (8/15) (95% CI 28%-78%) in the video group (P=.07). Participants who followed the e-learning method were more likely to recommend it to a colleague; 23/35 participants (66%) answered “Yes, most certainly” (95% CI 50%-82%), versus 8/26 (31%; 95% CI 13%-49%) in the video group, P=.007 (Figure 13). The proportions of participants in the two groups who followed the course over less than 12 hours were similar (58% (14/24) in the e-learning group, 95% CI 38%-78%, versus 52% (17/33) in the video group, 95% CI 35 to 69, P=.79).

Figure 13. Likelihood that the participants in the e-learning and video groups would recommend the course.
View this figure

The post hoc calculation showed a power of 97%. None of the 4 preplanned sensitivity analyses showed any major changes in the direction of the effect. The multivariable linear regression only showed a minor change in the coefficient (<15%), confirming these results.

Principal Findings

In this study, asynchronous distance learning using a highly interactive e-learning module yielded better results than following the traditional didactic video on the web. The superiority of a previous version of this module has already been established in Swiss paramedics following an onsite computer-based course [23]. The present study confirms the generalizability of these findings when using this method for asynchronous distance learning in a different population of learners, namely, fifth year medical students. Indeed, although paramedics follow a 3-year curriculum focusing on critical emergencies, baseline knowledge and understanding of neurosciences should be higher in students on the verge of obtaining their master of medicine degree [34]. This assumption is supported by the median score of the control group, which was higher by two points in this study than it was in paramedics [23].

The shortcomings we had identified in the previous iteration of the e-learning module seem to have been addressed, as embedding cutscenes from the original video into every chapter of the module improved the impact of the module on knowledge acquisition. The use of short videos associated with active learning activities such as guiding questions or interactive elements has been shown to enhance knowledge acquisition and retention [35]. Interactivity itself is also known to improve both engagement and performance in medical students [36,37].

Slightly less than half of all potential participants completed their allocated learning path. Considering that the learning material was optional and that students’ summative assessments of this semester were replaced by formative assessments, the participation rate is rather encouraging given the global lack of incentive. More encouraging still is the proportion of students who would recommend the course to their peers, as such mechanisms may increase students' involvement [38]. As many medical students actively helped on the front lines during the crisis, some of them may have been prevented from participating in this study owing to their high workload [39].

The quiz shown to the participants upon completion of the learning material included not only the full evaluation of 3 different stroke patients, but also 5 general questions we had designed Multimedia Appendix 2. While this could be considered as a potential bias in the study design, these questions were solely linked to key elements and basic principles of the NIHSS, and their understanding is essential to the correct application of the scale Multimedia Appendix 2. Our aim was indeed to evaluate whether knowledge acquisition was different when presenting similar content in different learning formats. As the overall score regarding these questions was high in both groups, and as other significant results favored the e-learning method, there is little probability that these 5 initial questions induced a bias.

In many hospitals, the NIHSS is commonly used to triage stroke victims and help reduce both door-to-CT (CT: computed tomography) and door-to-needle times [9]. Decreasing these times is associated with better neurocognitive and functional outcomes [40]. Moreover, the adoption of a common scale between different specialists seems necessary to improve reproducibility and avoid the misinterpretation that can result from the use of different scores [41]. Swift acquisition and mastery of the NIHSS is therefore an essential skill for medical students, as most will be required to take care of stroke patients during their residency while working in the emergency department or in the neurology department. This is further strengthened by the fact that medical students often perceive neurology as the most difficult medical discipline, and the development of negative perceptions toward this specialty could lead to avoidance mechanisms when considering a career or treating a patient [42]. We might therefore assume that any kind of stroke-directed educational program could help raise awareness in non-neurologist physicians and thus increase the rate of correct treatment while decreasing door-to-needle time.

Limitations and Strengths

This study has limitations that must be acknowledged. The main limitation is that we only measured immediate knowledge acquisition; we were unable to assess knowledge retention due to the study design and the limited timeframe. As this latter parameter is critical to the clinical application of the NIHSS, further studies will be needed to assess whether the e-learning method improves retention and leads to more accurate application of the scale. Moreover, the precise time taken to complete either learning method was not evaluated in this study. While it can be argued that dedicating more time to learning given content should yield better results, studies have shown that engagement is the most important factor regarding knowledge acquisition [43]. Although time to learning material completion is an interesting outcome, we chose not to record these data for two main reasons: risk of unblinding and technical limitations. As the time required to watch the video is fixed unless the participants elect to use the video commands, and as most of them chose not to use this option in a previous study [23], we thought it better not to risk unnecessarily unblinding the data analyst. The technical aspect was linked to the web-based learning management system and to the mode of delivery of the teaching material. As access to the university premises was barred during the study period, participants followed the learning material from many different locations. Interruptions in the learning process may therefore have occurred; however, we had no means of recording recurrent short breaks in the group that followed the e-learning module, as pauses may also have resulted from taking notes, mulling over the content, or simply rereading some of the written paragraphs to better understand them. To mitigate this limitation, a sensitivity analysis comparing participants who completed the study path in less and more than 12 hours was performed. Reassuringly, no difference was noted.

Despite these limitations, this study also has several strengths, including the randomization, the blinding mechanisms, the electronic data acquisition, the originality of the learning method and its mode of delivery in the context of the COVID-19 pandemic.


Compared to the traditional didactic video, a highly interactive e-learning module enhances distant NIHSS knowledge acquisition in medical students.


The authors would like to thank Mr Daniel Scherly from the UGFM for his technical assistance as well as for dispatching the emails and reminders on our behalf.

Conflicts of Interest

None declared.

Editorial Notice

This randomized study was not registered. The authors explained that their study included no patients and no health outcomes were recorded. According to the ICMJE rules, if the purpose is to examine the effect only on the providers rather than patients, then registration is not necessary. However, readers are advised to carefully assess the validity of any potential explicit or implicit claims related to primary outcomes or effectiveness.

Multimedia Appendix 1

Mailing template.

PDF File (Adobe PDF File), 224 KB

Multimedia Appendix 2

Original questions used in the 50-question quiz.

PDF File (Adobe PDF File), 276 KB

Multimedia Appendix 3

CONSORT-eHEALTH checklist (V 1.6.1).

PDF File (Adobe PDF File), 2197 KB

  1. Ahmed H, Allaf M, Elghazaly H. COVID-19 and medical education. Lancet Infect Dis 2020 Jul;20(7):777-778. [CrossRef] [Medline]
  2. Hall AK, Nousiainen MT, Campisi P, Dagnone JD, Frank JR, Kroeker KI, et al. Training disrupted: Practical tips for supporting competency-based medical education during the COVID-19 pandemic. Med Teach 2020 May 25;42(7):756-761. [CrossRef] [Medline]
  3. Chen CH, Mullen AJ. COVID-19 Can Catalyze the Modernization of Medical Education. JMIR Med Educ 2020 Jun 12;6(1):e19725 [FREE Full text] [CrossRef] [Medline]
  4. Naidoo N, Akhras A, Banerjee Y. Confronting the Challenges of Anatomy Education in a Competency-Based Medical Curriculum During Normal and Unprecedented Times (COVID-19 Pandemic): Pedagogical Framework Development and Implementation. JMIR Med Educ 2020 Oct 07;6(2):e21701 [FREE Full text] [CrossRef] [Medline]
  5. Klasen JM, Meienberg A, Nickel C, Bingisser R. SWAB team instead of SWAT team: Medical students as a frontline force during the COVID-19 pandemic. Med Educ 2020 Sep 17;54(9):860-860 [FREE Full text] [CrossRef] [Medline]
  6. Harrington RA, Elkind MS, Benjamin IJ. Protecting Medical Trainees on the COVID-19 Frontlines Saves Us All. Circulation 2020 May 05;141(18):e775-e777. [CrossRef] [Medline]
  7. Olum R, Kajjimu J, Kanyike AM, Chekwech G, Wekha G, Nassozi DR, et al. Perspective of Medical Students on the COVID-19 Pandemic: Survey of Nine Medical Schools in Uganda. JMIR Public Health Surveill 2020 Jun 19;6(2):e19847 [FREE Full text] [CrossRef] [Medline]
  8. Flotte T, Larkin A, Fischer M, Chimienti S, DeMarco D, Fan P, et al. Accelerated Graduation and the Deployment of New Physicians During the COVID-19 Pandemic. Acad Med 2020 Oct 09;95(10):1492-1494 [FREE Full text] [CrossRef] [Medline]
  9. Powers WJ, Rabinstein AA, Ackerson T, Adeoye OM, Bambakidis NC, Becker K, American Heart Association Stroke Council. 2018 Guidelines for the Early Management of Patients With Acute Ischemic Stroke: A Guideline for Healthcare Professionals From the American Heart Association/American Stroke Association. Stroke 2018 Mar;49(3):e46-e110. [CrossRef] [Medline]
  10. Montaner J, Barragán-Prieto A, Pérez-Sánchez S, Escudero-Martínez I, Moniche F, Sánchez-Miura JA, et al. Break in the Stroke Chain of Survival due to COVID-19. Stroke 2020 Aug;51(8):2307-2314. [CrossRef] [Medline]
  11. Mowla A, Doyle J, Lail N, Rajabzadeh-Oghaz H, Deline C, Shirani P, et al. Delays in door-to-needle time for acute ischemic stroke in the emergency department: A comprehensive stroke center experience. J Neurol Sci 2017 May 15;376:102-105. [CrossRef] [Medline]
  12. Ross D, National Neuroscience Curriculum Initiative “Quarantine Curriculum” Committee. Creating a "Quarantine Curriculum" to Enhance Teaching and Learning During the COVID-19 Pandemic. Acad Med 2020 Aug;95(8):1125-1126 [FREE Full text] [CrossRef] [Medline]
  13. Ashokka B, Ong SY, Tay KH, Loh NHW, Gee CF, Samarasekera DD. Coordinated responses of academic medical centres to pandemics: Sustaining medical education during COVID-19. Med Teach 2020 Jul 13;42(7):762-771. [CrossRef] [Medline]
  14. Iancu AM, Kemp MT, Alam HB. Unmuting Medical Students' Education: Utilizing Telemedicine During the COVID-19 Pandemic and Beyond. J Med Internet Res 2020 Jul 20;22(7):e19667 [FREE Full text] [CrossRef] [Medline]
  15. Qandil A, Abdel-Halim H. Distance e-Learning is Closer than Everybody Thought: A Pharmacy Education Perspective. Health Prof Educ 2020 Sep;6(3):301-303. [CrossRef]
  16. Longhurst GJ, Stone DM, Dulohery K, Scully D, Campbell T, Smith CF. Strength, Weakness, Opportunity, Threat (SWOT) Analysis of the Adaptations to Anatomical Education in the United Kingdom and Republic of Ireland in Response to the Covid-19 Pandemic. Anat Sci Educ 2020 May;13(3):301-311 [FREE Full text] [CrossRef] [Medline]
  17. Remtulla R. The Present and Future Applications of Technology in Adapting Medical Education Amidst the COVID-19 Pandemic. JMIR Med Educ 2020 Jul 17;6(2):e20190 [FREE Full text] [CrossRef] [Medline]
  18. McGowan BS. The Rise and Stall of eLearning: Best Practices for Technology-Supported Education. J Contin Educ Nurs 2015 Jul;46(7):292-294. [CrossRef] [Medline]
  19. Ellaway R, Masters K. AMEE Guide 32: e-Learning in medical education Part 1: Learning, teaching and assessment. Med Teach 2008 Jun;30(5):455-473. [CrossRef] [Medline]
  20. Lee L, Chao Y, Huang C, Fang J, Wang S, Chuang C, et al. Cognitive Style and Mobile E-Learning in Emergent Otorhinolaryngology-Head and Neck Surgery Disorders for Millennial Undergraduate Medical Students: Randomized Controlled Trial. J Med Internet Res 2018 Feb 13;20(2):e56 [FREE Full text] [CrossRef] [Medline]
  21. Vaona A, Banzi R, Kwag KH, Rigon G, Cereda D, Pecoraro V, et al. E-learning for health professionals. Cochrane Database Syst Rev 2018 Jan 21;1:CD011736 [FREE Full text] [CrossRef] [Medline]
  22. Lyden P, Brott T, Tilley B, Welch KM, Mascha EJ, Levine S, et al. Improved reliability of the NIH Stroke Scale using video training. NINDS TPA Stroke Study Group. Stroke 1994 Nov;25(11):2220-2226. [CrossRef] [Medline]
  23. Koka A, Suppan L, Cottet P, Carrera E, Stuby L, Suppan M. Teaching the National Institutes of Health Stroke Scale to Paramedics (E-Learning vs Video): Randomized Controlled Trial. J Med Internet Res 2020 Jun 09;22(6):e18358 [FREE Full text] [CrossRef] [Medline]
  24. Fouh E, Breakiron DA, Hamouda S, Farghally MF, Shaffer CA. Exploring students learning behavior with an interactive etextbook in computer science courses. Computers in Human Behavior 2014 Dec;41:478-485. [CrossRef]
  25. Kopp V, Stark R, Fischer M. Fostering diagnostic knowledge through computer-supported, case-based worked examples: effects of erroneous examples and feedback. Med Educ 2008 Aug;42(8):823-829. [CrossRef] [Medline]
  26. Eysenbach G, CONSORT-EHEALTH Group. CONSORT-EHEALTH: improving and standardizing evaluation reports of Web-based and mobile health interventions. J Med Internet Res 2011 Dec 31;13(4):e126 [FREE Full text] [CrossRef] [Medline]
  27. Eysenbach G. Improving the quality of Web surveys: the Checklist for Reporting Results of Internet E-Surveys (CHERRIES). J Med Internet Res 2004 Sep 29;6(3):e34 [FREE Full text] [CrossRef] [Medline]
  28. 810.30 Federal Act on Research involving Human Beings (Human Research Act, HRA). The Federal Council of Switzerland. 2011 Sep 30.   URL: [accessed 2020-05-12]
  29. Suppan L, Suppan M. NIHSS EtuMed Study Platform.   URL: [accessed 2020-05-18]
  30. Verschueren S, van Aalst J, Bangels A, Toelen J, Allegaert K, Buffel C, et al. Development of CliniPup, a Serious Game Aimed at Reducing Perioperative Anxiety and Pain in Children: Mixed Methods Study. JMIR Serious Games 2019 Jun 01;7(2):e12429 [FREE Full text] [CrossRef] [Medline]
  31. Suppan M, Gartner B, Golay E, Stuby L, White M, Cottet P, et al. Teaching Adequate Prehospital Use of Personal Protective Equipment During the COVID-19 Pandemic: Development of a Gamified e-Learning Module. JMIR Serious Games 2020 Jun 12;8(2):e20173 [FREE Full text] [CrossRef] [Medline]
  32. NIHSS E-Learning Modules.   URL: [accessed 2020-08-16]
  33. Suppan L, Suppan M, Stuby L, Carrera E, Savoldelli G. NIHSS - Medical Students Dataset. Mendeley Data.   URL: [accessed 2020-01-11]
  34. Safdieh JE, Govindarajan R, Gelb DJ, Odia Y, Soni M. Core curriculum guidelines for a required clinical neurology experience. Neurology 2019 Feb 22;92(13):619-626. [CrossRef] [Medline]
  35. Brame CJ. Effective Educational Videos: Principles and Guidelines for Maximizing Student Learning from Video Content. CBE Life Sci Educ 2016;15(4) [FREE Full text] [CrossRef] [Medline]
  36. Stevens NT, Holmes K, Grainger RJ, Connolly R, Prior A, Fitzpatrick F, et al. Can e-learning improve the performance of undergraduate medical students in Clinical Microbiology examinations? BMC Med Educ 2019 Nov 07;19(1):408 [FREE Full text] [CrossRef] [Medline]
  37. Wade SWT, Moscova M, Tedla N, Moses DA, Young N, Kyaw M, et al. Adaptive Tutorials Versus Web-Based Resources in Radiology: A Mixed Methods Analysis of Efficacy and Engagement in Senior Medical Students. Acad Radiol 2019 Oct;26(10):1421-1431. [CrossRef] [Medline]
  38. Ang L, Breyer Y, Pitt J. Course recommendation as a construct in student evaluations: will students recommend your course? Stud High Educ 2016 Jul 05;43(6):944-959. [CrossRef]
  39. Soled D, Goel S, Barry D, Erfani P, Joseph N, Kochis M, et al. Medical Student Mobilization During a Crisis: Lessons From a COVID-19 Medical Student Response Team. Acad Med 2020 Sep;95(9):1384-1387 [FREE Full text] [CrossRef] [Medline]
  40. Sauser K, Levine DA, Nickles AV, Reeves MJ. Hospital variation in thrombolysis times among patients with acute ischemic stroke: the contributions of door-to-imaging time and imaging-to-needle time. JAMA Neurol 2014 Sep 01;71(9):1155-1161. [CrossRef] [Medline]
  41. Purrucker JC, Hametner C, Engelbrecht A, Bruckner T, Popp E, Poli S. Comparison of stroke recognition and stroke severity scores for stroke detection in a single cohort. J Neurol Neurosurg Psychiatry 2015 Sep;86(9):1021-1028. [CrossRef] [Medline]
  42. Abushouk AI, Duc NM. Curing neurophobia in medical schools: evidence-based strategies. Med Educ Online 2016;21:32476 [FREE Full text] [CrossRef] [Medline]
  43. Deater-Deckard K, Chang M, Evans ME. Engagement states and learning from educational games. New Dir Child Adolesc Dev 2013 Mar 08;2013(139):21-30. [CrossRef] [Medline]

CHERRIES: Checklist for Reporting Results of Internet E-Surveys
CSV: comma-separated values
CT: computed tomography
NIHSS: National Institutes of Health Stroke Scale
UGFM: University of Geneva Faculty of Medicine

Edited by R Kukafka; submitted 17.08.20; peer-reviewed by KH Miller; comments to author 09.09.20; revised version received 09.09.20; accepted 10.01.21; published 15.01.21


©Mélanie Suppan, Loric Stuby, Emmanuel Carrera, Philippe Cottet, Avinash Koka, Frédéric Assal, Georges Louis Savoldelli, Laurent Suppan. Originally published in the Journal of Medical Internet Research (, 15.01.2021.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on, as well as this copyright and license information must be included.