Launch and global reach of an online COVID-19 course in multiple languages on OpenWHO in the first quarter of 2020

Background: At the onset of the coronavirus outbreak, the World Health Organization’s (WHO) Health Emergencies Learning and Capacity Development Unit, together with WHO’s health technical lead on coronaviruses, developed a massive open online course (MOOC) within just three weeks as part of the global response to the emergency. The introductory COVID?19 course was launched on 26 January 2020 on the health emergencies learning platform OpenWHO.org. Objective: This article investigates geographic reach of different language courses accessed by a global audience seeking information on COVID-19. Users’ professional identities and background are explored to inform course owners on the use case. The course was developed and delivered via the open access learning platform OpenWHO.org. The self-paced resources available in a total of 13 languages were produced between 26 January and 25 March 2020. Methods: Data was collected from the online courses’ statistical data and metrics reporting system on the OpenWHO platform. User patterns and locations were analysed based on Google Analytics and the platform's own statistics capabilities, with data sets overlaid. This analysis was conducted based on user location, with the data disaggregated according to the six WHO regions, their top 10 countries and the proportion of use for each language version. Data includes affiliation, gender, age and other parameters for approximately 30% of users who indicated their background. Results: As of 25 March 2020, the introductory COVID-19 course totals 232 890 enrolments across all languages. The Spanish language course comprises more than half (51%) of all course enrolments, while the English language course comprises 37% of enrolments. WHO’s Region of the Americas accounts for most of the course enrolments, with more than 72% across all languages. Other regions are more evenly distributed with less than 10% each. 32.4% of users specified professional affiliation by choosing from the 12 most common backgrounds in OpenWHO’s user profiles. Before the COVID-19 pandemic, users were spread over the 11 distinct affiliations with small fractions of users identifying themselves as “Other”. With the COVID-19 introductory course, the largest number of users selected “Other” (31.7%) suggesting a large non-health professional and academic user load. The top 10 countries of users across all languages are Argentina, Chile, Colombia, Ecuador, India, Mexico, Peru, Spain, the United Kingdom and the United States of America. Conclusions: The online course has addressed a global learning need by providing WHO technical guidance packaged in simple formats for access and use. The learning material development was expedited to meet the onset of the epidemic. Initial data suggests that the various language versions of the course, in particular Spanish, have reached new user groups, fulfilling the platform’s aim of providing learning everywhere to anyone interested. User surveys will be carried out to measure real impact. (JMIR Preprints 03/04/2020:19076) DOI: https://doi.org/10.2196/preprints.19076


Yes, please make my accepted manuscript PDF available to anyone at any time (Recommended).
Yes, but please make my accepted manuscript PDF available only to logged-in users; I understand that the title and abstract will remain v Yes, but only make the title and abstract visible (see Important note, above). I understand that if I later pay to participate in <a href="http

Original Manuscript
Methods: Data was collected from the online courses' statistical data and metrics reporting system on the OpenWHO platform. User patterns and locations were analysed based on Google Analytics and the platform's own statistics capabilities, with data sets overlaid. This analysis was conducted based on user location, with the data disaggregated according to the six WHO regions, their top 10 countries and the proportion of use for each language version. Data includes affiliation, gender, age and other parameters for approximately 30% of users who indicated their background.
Results: As of 25 March 2020, the introductory COVID-19 course totals 232 890 enrolments across all languages. The Spanish language course comprises more than half (51%) of all course enrolments, while the English language course comprises 37% of enrolments. WHO's Region of the Americas accounts for most of the course enrolments, with more than 72% across all languages. Other regions are more evenly distributed with less than 10% each. 32.4% of users specified professional affiliation by choosing from the 12 most common backgrounds in OpenWHO's user profiles. Before the COVID-19 pandemic, users were spread over the 11 distinct affiliations with small fractions of users identifying themselves as "Other". With the COVID-19 introductory course, the largest number of users selected "Other" (31.7%) suggesting a large non-health professional and academic user load. The top 10 countries of users across all languages are Argentina, Chile, Colombia, Ecuador, India, Mexico, Peru, Spain, the United Kingdom and the United States of America.

Conclusions:
The online course has addressed a global learning need by providing WHO technical guidance packaged in simple formats for access and use. The learning material development was expedited to meet the onset of the epidemic. Initial data suggests that the various language versions of the course, in particular Spanish, have reached new user groups, fulfilling the platform's aim of providing learning everywhere to anyone interested. User surveys will be carried out to measure real impact.

Introduction
The research material is based on WHO's health emergencies platform OpenWHO.org that hosts online learning resources for outbreaks and epidemics. OpenWHO is an open source online platform adjusted for low bandwidths, with mobile and download capabilities. This free, web-based, knowledge-transfer platform was designed for massive real-time use during a pandemic and has been in a real test during the early part of 2020 in offering free online courses to improve COVID-19 preparedness and response. The COVID-19 resources are hosted on two learning channels on the platform: one for courses in official UN languages and a second for courses in additional national languages. The first COVID-19 related course "Introduction to emerging respiratory viruses, including COVID-19: methods for detection, prevention, response and control" was launched on OpenWHO.org on 26 January 2020 following the first WHO Emergency Committee meeting on 22-23 January 2020. The course has four modules:  1. Introduction to emerging respiratory viruses, including COVID-19  2. Detecting emerging respiratory viruses, including COVID-19: surveillance and laboratory investigation  3. Risk communication and community engagement  4. Preventing and responding to an emerging respiratory virus, including COVID-19 This first version of the course includes introductory information on novel coronavirus, other coronaviruses and basic information for anyone wanting to understand the new epidemic. The course was packaged and presented through PowerPoint slide decks, and video recordings by the different WHO health experts leading respective areas of work were added during the following days and weeks. This simple packaging allows for material to be used on multiple devices, in low bandwidth settings and through an offline function in the platform application. The packaging of the online learning using videos and slide decks also supports the ability to update the material to reflect the frequently changing and updated WHO technical guidance. Due to the changing content during the first months of the Public Health Emergency of International Concern and first weeks of the pandemic, the OpenWHO team has not made available any quizzes as is usual for other courses on the platform, and thus is not providing a certificate of completion. These features will be added once the technical content can be considered more final and established. The timeline (Figure 1) below shows the launch of the course in different languages and the total enrolments as of 27 March 2020, the day the OpenWHO platform reached 1 million enrolments. The course was published on 26 January 2020 in English and gradually in all other UN languages. Another seven national languages were produced courtesy of dozens of volunteers providing spontaneous translation offers from WHO country and regional offices and headquarters, as well as volunteers from public health institutes and educational units, all seeking to provide voluntary support to maximize at local level the uptake of courses for an effective response. Among other efforts, OpenWHO released an introductory video to COVID-19 in Indian sign language, the first sign language resource on the platform.

Relevant literature
Research has found that WHO is a respected source of accurate health information during epidemics, suggesting that the organization has a platform to shape health behaviour that less-trusted sources such as governments may lack. [1] The OpenWHO team has focused on designing knowledgetransfer resources for health emergencies in formats and languages that are suitable for frontline responders and affected communities. This approach has prioritized multi-language production, recognizing that language can be a key obstacle to health literacy. [2] According to WHO (1998), "health literacy represents the cognitive and social skills which determine the motivation and ability of individuals to gain access to, understand and use information in ways which promote and maintain good health". [3] A 2015 study in Kenya by Translators Without Borders found that providing health information in Swahili -the lingua franca throughout the country -produced a very significant increase in comprehension compared to providing the same information in English. [4] A 2019 study by the same organization, which partners with the WHO team on many translations, found that the local form of Swahili was the most effective language for risk communication and community engagement for the Ebola response in Goma, Democratic Republic of the Congo, compared to French and standard Swahili. [5] Providing health information in individuals' native language has also shown to improve knowledge of illness and medication in a patient population in Sri Lanka, as well as the understanding of oral health information among Vietnamese-speaking mothers in Australia. [6,7] Materials produced for OpenWHO are designed with additional accessibility considerations in mind. The resources are offered as downloadable slides that combine images and short texts that can be read on a mobile device. Video and audio formats are also being integrated for those with strong oral cultures. In addition, the open access nature of the platform can empower individuals who are more health literate to strengthen the health literacy of their communities, particularly in group-oriented societies. In a cholera-endemic neighbourhood in Ghana, researchers found that household units impact individual health literacy: nearly three-quarters of households surveyed followed suggestions from household members on how to prevent cholera. [8]

Study objective and methods
This article investigates the geographic reach of different language courses accessed by the global audience population while seeking health information on COVID-19. The users' professional identities are explored to inform the course owners on the use case. The course was developed and delivered via the learning platform OpenWHO.org. The self-paced introductory course has been provided in a total of 13 languages between 26 January and 25 March 2020. The 13 languages are: Arabic (AR), Chinese (ZH), English (EN), French (FR), Hindi (HI), Indian sign, Macedonian (MK), Persian (FA), Portuguese (PT), Russian (RU), Serbian (SR), Spanish (ES) and Turkish (TR). This study's preliminary objective was to demonstrate the rapid surge to the digitized learning materials as the COVID-19 epidemic grew into a pandemic during the early part of 2020. The aim of this study is to obtain a better understanding of the origin and type of people who sought access to online learning related to the emerging health emergency. Statistical data of the identical courses in 13 languages was generated. More in-depth analysis was carried out on the English and Spanish language courses given the large use case in these two languages -90% of all learners. The data was collected from the online courses' statistical data and metrics reporting system on the OpenWHO platform. User patterns and locations were analysed based on Google Analytics and the OpenWHO platform's own statistical capabilities, and data sets were overlaid. This snapshot analysis was conducted based on user location, with the data disaggregated according to the six WHO regions, the top 10 countries and the proportion of use for each language version. Data includes affiliation, gender, primary language, age and other parameters for approximately 30% of users who indicated their background.  The two most popular languages are the Spanish language course, which comprises more than half (51%) of all introductory COVID-19 course enrolments (119 000 on 25 March 2020), and the English language course, which comprises 37% of enrolments (64 780). Despite being launched two weeks later, the Spanish version rapidly surpassed the original English course on 19 March 2020, becoming the main language driving the enrolment increase. English and Spanish course users jointly amount to 210 000 users, representing 90% of all users across all language versions.

Introductory COVID-19 course user metrics
As expected, the number of accumulated enrolments across all language versions of the course has risen steadily with each new language version launched ( Figure 3).

Geographical distribution of users
The global distribution of users is displayed in Figure 5 according to the WHO regions. The AMRO/ PAHO region accounts for most of the enrolments in the course, with more than 72% of total enrolments across all languages. Other regions are more evenly distributed: AFRO 4%, EMRO 6.7%, EURO 9.5%, SEARO 3.8%, WPRO 2.7% and territories 1%.

Figure 5. Course use percentages by WHO region
Users originating from the AMRO/PAHO region account for approximately 51.6% of the enrolments in the Spanish language course. The second most popular language choice for AMRO/PAHO users is English (17.2%), with a majority of English course users enrolling from the United States of America, totalling more than a quarter (28.8%) of the total English course users ( Figure 6).  The introductory course has become most popular in the AMRO/PAHO region, with 72% of total enrolments across all languages concentrated in this region. This trend might be explained in part by the language availability of the course for the first month, which was limited to English and Spanish along with the French and Chinese versions (Figure 1). Totalling 48.9%, of the total course enrolments, the Spanish version has 95.6% enrolments from the AMRO/PAHO region. With 38.8% of the total course enrolments, the English version has 46.4% enrolments from the AMRO/PAHO region. The latter statistic speaks to the more even distribution of the English language course across WHO regions (AFRO 8.4%, EMRO 12.1%, EURO 15.8%, SEARO 9.2%, WPRO 7.0% and territories 1%). Even taking into consideration the gradual release of the language versions' impact on these statistics, it is worth noting that no other language has had similar slopes as those seen for the Spanish and English versions ( Figure 2 61.5% of the total users who indicated their language of preference selected Spanish (Figure 7). This language preference helps the OpenWHO team further target the courses and prioritize languages.

Figure 7. Users as per their indicated language preference
The indicated language preference correlates with the language course use ( Figure 8), but as the option for the preferred language only includes the six UN official languages and Portuguese, the OpenWHO team is not able to capture if there are any national and local languages popular along with the official UN languages.

Figure 8. Proportion of enrolments by course language version
The same introductory course was published on the Pan-American Health Organization's (PAHO) Virtual Campus (VC) in Spanish on 11 February and in Portuguese on 28 February 2020. This article and data excludes the 92 000 users of these identical courses on the PAHO VC, which would bring the global merged user numbers even higher.

User background information
Before the COVID-19 pandemic, users were spread over the 11 distinct affiliation options provided by the platform, with small fractions of users identifying themselves as "Other". With the epidemic accelerating into a global pandemic, the largest number of the COVID-19 introductory course users selected "Other" (31.7%), suggesting a large non-health professional and academic user load. Students are the largest identifiable group (28.6%) among those who indicated their affiliation. Health ministries and health experts make up 14.2%, while UN country teams and WHO staff amount for 4% ( Figure 10). Figure 10. Users' professional affiliation n= 52214); 32.4% of the total users specified their professional affiliation When looking at the top countries and user affiliations in Figure 11 below, we see that, as previously depicted, students account for a large proportion of the different users coming from those countries. For Chile, Colombia and Mexico for instance, students represent almost 45% of the total enrolments. As mentioned earlier for the overall analysis, the Figure shows that many users are not affiliated with a specific health background. If we examine the Figure below, we find that the percentage of users who selected "Other" as their affiliation varies from roughly 20% (Mexico) to approximately 40% for the United States and the United Kingdom. Health institutes are also represented in the affiliation of the users of the top 10 countries. The share of health institute workers is the highest for the United Kingdom at more than 20%. Another interesting finding is the attraction of health ministry representatives, especially significant for Ecuador as they represent around 20% of total users for the country. Besides Ecuador, Peru is the only other country reporting enrolments from health ministry professionals reaching at least 10%. Other health experts also enrolled in the course: they represent for instance about 10% of the total enrolments reported from Spain. Volunteers are also represented in all of the top 10 countries, with India reporting the highest percentage of this specific user affiliation with approximately 10% of its enrolments. In addition, the course registrations suggest women are a larger user group (57.6%) than men (42.2%) (Figure 12).

Figure 12. Gender distribution of the COVID-19 course users
When comparing the English and Spanish course age groups (Figures 13, 14), the Spanish course in particular has a large cohort of participants in the 70+ age range, much higher than the platform average or the English course. As COVID-19 was stated to be most impacting older people, this use case comes with no surprise. On average the English course has a younger use case than the platform average, especially in the age groups 30-39 and 40-49.

Spanish and English courses and completion rates
36.4% of participants enrolled in the Spanish introductory course completed all six learning items within the course. In comparison to industry standards, which place the completion rate for MOOCs at roughly 7.5%, [9] this rate is high. Again, in relation to the Spanish introductory course, 44.6% of participants completed at least 80% of the course material and 49% visited at least 60% of the course material. When the completion rate was calculated across all language versions of the course, the trend continued, with 21.6% of users visiting at least 100% of the course items, 27.6% visiting at least 80% and 32% of users visiting at least 60% of the total learning resources in the course. Participants enrolled in the English version did not perform as well. Only 2.9% of participants enrolled in the English course completed all course material, with 6.6% completing at least 80% of the course material and 16.6% visiting at least half of all course items. Further investigation is required to determine the cause of the discrepancy between the completion rates for the English version of the course compared with the completion rate across all language versions combined. Unlike the subsequent language versions, the English introductory COVID-19 course was assembled over a period of several weeks, with new materials being made available as they were constructed and cleared by the technical experts responsible. In contrast, most of the subsequent language versions were launched as full packages, with all course items available at once. This difference could begin to explain the discrepancy in completion rates, as the first set of users who enrolled in the English course would have had to return to the course at later dates to view new material as it was added.

Platform user surge during the early weeks of the COVID-19
Since

Discussion
OpenWHO learning resources are usually published initially in English, as this language is shared by many native and non-native speakers across the globe, and is the most commonly used working language of the United Nations system. Recent trends seen on the platform during the pandemic, notably the popularity of the Spanish versions of the COVID-19 resources surpassing that of the original English courses, stress the importance of a multilingual learning platform which allows learners to access information in the language they are the most comfortable using. This has been a continuous effort since the platform's inception, with courses even produced in local languages for localized emergencies. During the Ebola outbreak in the Democratic Republic of the Congo, for example, the Ebola knowledge resources for responders course was published in Lingala and Swahili. This multilingual approach is even more important in the middle of a pandemic, where OpenWHO is experiencing record-breaking enrolments from learners across the globe who are seeking the latest WHO guidance to support the preparation and response to COVID-19. Indeed, in its first two months, the introductory course to COVID-19 comprised more than a quarter (241 749) of the accumulated total enrolments on a platform that has been running since June 2017. The geographic analysis reveals that the enrolment surge for this emergency-related course has mainly originated from Central and South America, with Ecuador, Mexico, Columbia, Chile and the USA rounding out the top five countries, reflecting a new demographic attracted to the platform. Interestingly, the age distribution analysis for the Spanish course, with enrolments mainly concentrated in the WHO AMRO/PAHO (95.5%) region reveals the popularity of the course within the 70+ age group compared to the other courses hosted on the OpenWHO platform. This is consistent with the at-risk group seeking reliable knowledge on the emerging respiratory disease. Prior to the pandemic, the top countries for the most popular emergency courses were on the African continent, along with recurrent appearances of the USA and India as these populous countries are big MOOC users. This geographic shift is consistent with the nature of a pandemic. Rather than affecting specific parts of the world, as was the case for the 2014 Ebola outbreak for example, the COVID-19 epidemic has accelerated into a pandemic reaching almost every country on the planet. In contrast with the top emergency courses prior to the COVID-19 outbreak, the analysis reveals that nearly 30% of users have indicated they are not affiliated with the student and professional health sectors. This shift reflects the impact a pandemic has on the audience profile, with the general public enrolling in the course to become informed on the novel coronavirus.
OpenWHO hit the 1 million enrolments milestone on 27 March 2020. About ¾ of the total enrolments were on the COVID-19 related courses. This reflects a massive increase in the popularity of OpenWHO and the critical role it is playing in supporting preparedness and response during this unprecedented pandemic.

Limitations and future research
This analysis is limited to the two-month period following the launch of the introductory COVID-19 course on OpenWHO. Data such as affiliation, gender and age were only available for approximately 30% of users who indicated their background. The geographical data is based on the 36% of users who identified their location, as well as users' IP addresses which would not account for potential manipulation by virtual private networks (VPNs) or other factors. In addition, completion rates were measured by user visits to each of the course's learning items, as quizzes have not yet been implemented due to the evolving nature of the emergency guidance. Future research should examine the use case of additional COVID-19 courses on the OpenWHO platform that are more targeted to specific audiences, such as courses designed for clinicians and public health professionals. Data should also be analysed against the global, regional and country level epidemic curves for COVID-19 to identify broader use trends.

Conclusions
During health emergencies, life-saving information must be packaged and delivered in the languages spoken by target audiences. Everyone has the right to access life-saving knowledge and OpenWHO is continuing to work with partners to make its resources available in as many languages as possible.
The OpenWHO team has never before experienced such high levels of crowd-power and volunteerism for platform language production, and has relied on crowdsourcing across the globe to publish the national language versions. This has promoted the localization of materials into a variety of languages, helping people better protect themselves and fight the pandemic. The OpenWHO platform offers courses on six distinct topics to support the COVID-19 response. These are products of transforming WHO guidance into learning packages that users can grasp and digest more easily. These courses have been translated and published fully or partially into 40 language versions during the first two months of the response, and OpenWHO has experienced an unprecedented increase in platform use. Amidst huge demand for reliable resources of knowledge to understand and decipher the evolving situation, OpenWHO has served as one source of digitized information.