In 2021, as part of its global response to the COVID-19 pandemic, Meta spearheaded a pilot course with UNICEF, Capulet, and other partners to support public health agencies and nongovernmental organizations in developing effective health communications for the digital space. In 2022, the DMHO course was proposed to build on this work, and the course syllabus was developed collaboratively during an all-partner meeting in August 2022.

Delivered over 5 modules, the course was designed to support public health agencies and organizations in creating effective digital health behavior change campaigns. From November to December 2022, we piloted the course with a private cohort of 100+ health and communication practitioners from partner organizations over a 6-week period to test, validate, and strengthen content and delivery. The course was launched for public consumption in May 2023. As a part of course maintenance and improvement, we performed regular course evaluations and updates (Figure 1).

The course consisted of the following 5 modules (Table 1). Each module consisted of instructional videos, in-video quizzes, readings, and expert case studies. Progress was assessed through a simple, relevant homework assignment graded through the Coursera peer review system, by which course learners were able to leave feedback on each other’s work. The course was hosted on Coursera, an online learning platform offering free and tiered payment access [15]. The course was available in all 9 United Nations and African Union languages, including Arabic, Bahasa Indonesia, Chinese, English, French, Portuguese, Russian, Spanish, and Swahili. Translations of all course content were done by WeTranslate, a professional translation agency. The course content was available for free; if learners wanted to receive a Yale certificate upon course completion, they could pay for a version of the course costing approximately US $50, payable either by individual learners, learner organization, or a Coursera Plus subscription. However, all those who enrolled could apply for Financial Aid and receive the certificate for free.

The course was first launched for a pilot cohort of public health practitioners from November 2022 to December 2022. Learners, most of whom were UNICEF, WHO, or Ministry of Health employees, were invited based on course partner recommendations to participate in a 6-week, guided version of the course, and asked to provide in-depth feedback on course elements and overall flow. The private cohort included 107 learners representing 42 countries and 55 unique organizations; 83% (88/107) completed the course and submitted postcourse evaluation surveys. Furthermore, 10 completers were randomly selected to participate in individual postcourse interviews, conducted online by the Yale Institute for Global Health and lasting approximately 30-45 minutes. The results of these evaluations were used to update the course for public launch. A public version of the course was officially launched in English, Arabic, French, and Spanish in May 2023. Versions in the additional 5 languages were launched in September 2023.

The course was evaluated using a pre-post survey design hosted on Qualtrics [16]. At the start of the course (module 1), learners were asked to complete a confidential survey (referred to as the “baseline survey”) to establish their baseline experience in using digital media to promote health outcomes. At the end of the course, learners were asked to complete a similar survey (referred to as the “endline survey”), which also asks them to reflect on elements of the course, what they liked, and what they did not. Survey questions were validated during the pilot version of the course and are available in Multimedia Appendix 1. Survey translations (from English to other languages) were done using Qualtrics’ built-in translation software and verified by knowledgeable team members using DeepL, an online translation program recognized for accuracy. All translations were carefully reviewed for consistency with the English version before survey launch. Within the survey, participants were able to toggle between languages as desired.

Both surveys asked learners a series of demographic questions, including what type of organization they worked for and where their work was based. In both surveys, key performance indicators (KPIs) included learners’ confidence and experience in applying six fundamental campaign skills including (1) understanding and identifying behavioral insights, (2) identifying a target audience online, (3) using best practices in designing creatives, (4) planning a digital communication strategy to drive health outcomes, (5) using practical tactics to implement the communication strategy, and (6) evaluating campaign outcomes. Learners were also asked to rank their confidence from low to high (0-10), with scores of 0-5 categorized as low confidence and scores of 6-10 categorized as high confidence. Similarly, learners were asked to report their perceived confidence in and importance of, ease of, likelihood to, and effectiveness of using digital media to drive health outcomes.

Data presented in this study were collected from May 2023 to April 2025. All data were self-reported. Proportions and 95% CIs were calculated for all confidence questions, and proportion change was compared between baseline and endline surveys. Incomplete surveys were removed before starting the analysis.

We also conducted a subanalysis of respondents for whom we could link baseline and endline survey results. Response datasets were merged by respondent age, gender, and current organization, provided in the baseline and endline surveys. Demographics and KPIs were assessed as in the main analysis. Deidentified data from both surveys were analyzed using STATA (StataCorp).

This analysis was classified as a program evaluation (a type of nonhuman participant research) by the Institutional Review Board at the University of Texas Southwestern Medical Center. As such, the study was approved as nonregulated research (STU-2024-0962). Elements of this report were included based on the Western Michigan University Checklist for Program Evaluation Reporting Content. Participants received no compensation for participation. The first page of both surveys advised participants that their participation was entirely voluntary and that their responses would be kept confidential and used to evaluate course performance over time. Respondents provided their consent to participate by proceeding to the next page of the survey.

As of April 2025, a total of 14,170 individuals were enrolled in the DMHO course, and 5971 learners had started the course. A participant flow diagram is presented in Figure 2.

Of these, 4329 (73%) completed the baseline survey (Table 2). The top 5 language versions among learners were English (3394/4329, 78%), Spanish (390/4329, 9%), French (211/4329, 5%), Arabic (116/4329, 3%), and Russian (89/4329, 2%). Most learners identified as women (3010/4329, 69%), were older than 25 years (1974/2464, 80%), and had received at least a bachelor’s degree (3519/4329, 81%). Many learners worked at academic institutions (505/4329, 12%), ministries of health and government agencies (447/4329, 10%), and in the private sector (600/4329, 14%). Several learners were university students (427/4329, 10%). Geographically, learners represented 162 countries and all UNICEF regions. The five countries with the greatest shares of learners were the United States (986/4329, 22%), Chile (208/4329, 5%), India (196/4329, 5%), Canada (153/4329, 4%), and Nigeria (140/4329, 3%; Figure 3). Regionally, learners were predominantly from North America (1121/4311, 26%), Latin America and the Caribbean (597/4311, 14%), and Western Europe (582/4311, 14%).

When asked to rank which skills they wanted to learn from the course, learners ranked “understanding and identifying behavioral insights” as the most important skill to learn (Figure 4). Learners ranked “identifying target audiences online” second, “using best practices in designing creative [campaign images and graphics]” and “planning a digital communication strategy to drive health outcomes” third and fourth, “learning practical tactics to implement the communication strategy” fifth, and “evaluating campaign outcomes related to SBCC and health communications” last. Overall, most learners at baseline were confident in using these skills and found using digital media to drive health outcomes to be important, easy, and effective, and reported they were likely to do so (Table 3).

When asked how often they used digital media to drive health outcomes, 39% (1668/4329) learners responded “never or rarely,” 34% (1467/4329) responded “sometimes,” and 28% (1194/4329) responded “often or always.” Stratifying by learners’ organization type, we saw that students were significantly less likely to use digital media to drive health outcomes. Similarly, when asked how often they evaluated health outcomes in response to digital campaigns, 61% (2551/4157) responded “never or rarely,” 28% (1156/4157) responded “sometimes,” and 11% (450/4157) responded “often or always” (Table 4). Stratification by the learner organization did not show any significant differences.

Of the 682 (93%) learners who started the endline survey, 634 learners completed it as of April 2025 (Table 5). The English (316/634, 50%) and Spanish (234/634, 37%) versions of the course had the most completers. Most completers identified as women (464/634, 73%), were between 18 and 34 years of age (379/548, 69%), and had earned at least their bachelor’s degrees (399/634, 63%). Many course completers were students (226/634, 36%) or worked for ministries of health and government agencies (30/634, 4.7%), academic organizations (59/634, 9%), or the private sector (56/634, 9%). The top three countries represented were Chile (240/634, 38%), the United States (69/634, 11%), and Madagascar (26/634, 4%), and the top 3 regions were Latin America and Caribbean (271/634, 43%), North America (79/634, 13%), and Eastern and Southern Africa (76/634, 12%; Figure 5).

There was a significant increase in skill confidence among course completers compared with baseline learners (Table 3). Respondents at endline were significantly more confident in all skills, especially planning a digital communication strategy, with 66% (2835/4329) reporting high confidence at baseline compared to 96% (611/634) at endline (+16%, 95% CI 14%-18%, P<.001), using practical tactics to implement that strategy (+30%, 95% CI 19%-33%, P<.001), and evaluating health outcomes after campaign implementation (+28%, 95% CI 26%-29%, P<.001). Other KPIs also showed significant improvement from baseline to endline. Learners were significantly more confident in their ability to use digital media (+18%, 95% CI 17%-20%, P<.001). Similarly, more endline respondents indicated using digital media was important (+5%, 95% CI 4%-6%, P<.001), easy (+9%, 95% CI 7%-11%, P<.001), effective (+8%, 95% CI 7%-9%, P<.001), and that they were likely to use it (+6%, 95% CI 5%-7%, P<.001).

When asked how often they used digital media to drive health outcomes, 33% (206/634) responded “never or rarely,” 34% (217/634) responded “sometimes,” and 33% (211/634) responded “often or always.” (Table 4). Stratifying by learners’ organization type showed that students were significantly less likely to use digital media as compared with other learners. When asked how often they evaluated health outcomes in response to digital campaigns, 50% (293/594) responded “never or rarely,” 34% (204/594) responded “sometimes,” and 16% (97/594) responded “often or always.” Stratification by organization type did not reveal any differences in the frequency of evaluation. From baseline to endline, there was a significant decrease in the number of respondents who never or rarely used digital media (–6%, P=.004), and a significant increase in the number of respondents who often or always used digital media (+6%, P=.003; Table 4). Similarly, there was a significant decrease in the number of respondents who never or rarely evaluated health outcomes from baseline to endline (–12%, P<.001), and significant increases among respondents who sometimes (+7%, P=.001) or often or always evaluated outcomes (+6%, P<.001).

We compared baseline and endline survey responses for 360 (of 634) completers for whom we could link unique survey submissions (Table 6). Of these, most completed the English (210/360, 58%) and Spanish (120/360, 33%) versions of the course, 79% (285/360) identified as women, 55% (149/271) were between 18 and 24 years old, and most had completed at least their bachelor’s degree (252/360, 70%). The top-represented country was Chile (124/360, 34%), followed by the United States (43/360, 12%).

As in the main analysis, course completers reported significantly higher confidence in key skills compared to baseline (Table 7). Notably, completers saw increases in confidence of 14% percentage points for planning a digital communications strategy (95% CI 0.09-0.19, P<.001) and 16% percentage points for outcome evaluation (95% CI 0.1-0.21, P<.001). Similarly, completers’ confidence in their ability to use digital media, the likelihood they would do so, and their perception of its effectiveness improved significantly after taking the course (Table 7). Between baseline and endline, no significant change was seen in how often course completers used digital media for health campaigns or evaluated health outcomes following those campaigns.

Course completers were asked to evaluate the overall course and specific modules. Generally, it took most learners less than 1 month to complete the course (348/634, 55%) and <4 hours to complete each module (475/634, 75%). Most respondents (570/634, 90%) reported using a laptop or computer to take the course, and 90% (570/634) of learners agreed that, from where they took the course, the content was easy to load and view.

Over 70% (444/634) of respondents reported that they had applied some or all skills from the course to their work. Commonly applied skills included “using behavioral insights to develop social and behavior change communication” (190/634, 30% and 133/634, 21% indicated often or always, respectively), “developing a strategic brief” (197/634, 31% and 139/634, 22%), and “using Canva or a similar platform to create campaign assets” (184/634, 29% and 228/634, 36%).

When asked if they had used Meta to design custom target audiences, used A/B testing to assess their campaign strategy, or evaluated health outcomes following their campaign, the most common response was “not yet, but I plan to” (197/634, 31%, 228/634, 36%, and 208/634, 33% respectively; Figure 6).

Overall, 92% (583/634) of course completers felt that the course had met most, all, or exceeded their expectations. When asked how satisfied they were with individual course elements, most learners agreed they were satisfied with the instructor videos (577/634, 91%), case studies (583/634, 92%), the Coursera interface (570/634, 90%), course pacing (577/634, 91%), homework assignments (558/634, 88%), quizzes (552/634, 87%), and translation (539/634, 85%). The peer-review process was the least satisfactory, with only 78% (494/634) satisfaction. Anonymous (and optional) comments at the end of the survey cited wishes that all assignments could be reviewed by course instructors or content experts and graded at more regular intervals rather than by the Coursera peer-review system. Individual modules were also well-received by respondents. For each module, the majority of course completers agreed that the material was new to them, easy to understand, and relevant to their work (Table S1 in Multimedia Appendix 2).

When asked how likely they were to recommend the course to a colleague or friend, 85% (539/634) indicated that they were likely to do so. When asked whether they would share their certificate of completion (if applicable) on LinkedIn, 64% (406/634) indicated that they would (159/634, 25% indicated “not applicable” either because they chose to complete the course without certification or because they do not have a LinkedIn page).

The DMHO course was designed to be accessible and useful to public health practitioners with limited experience in behavioral evidence-based health communication. Notably, the course has reached thousands of learners from countries in Asia, Africa, and Latin America. A course that is available online, free of charge, and in multiple languages has the potential to make useful training available to audiences worldwide.

The COVID-19 pandemic demonstrated the need for strategic campaigns on digital media, specifically social media, to reach diverse audiences with essential information, especially in the context of widespread online misinformation. Organizations must be equipped both to communicate quickly and effectively during a crisis and to maintain credibility as a trusted voice for public health information. The DMHO course provides an evidence-based framework for the development of campaigns that can be applied to systematically target health behaviors, offer compelling information, and promote demand for services. Encouragingly, the evaluation revealed that the course improved self-reported confidence among learners from baseline to endline, along with self-reported endorsement that using digital media to drive health outcomes was important, easy, and effective. Direct comparison of baseline versus endline survey results for learners who completed both revealed similarly promising results to overall analyses. Learners’ perceived ability to apply key skills improved from baseline to endline, as did their confidence in using digital media, how effective they believed it to be, and how likely they were to use it. Moreover, more respondents indicated they were likely to use digital media to achieve their organization’s goals at endline than at baseline. These results are especially promising in the context of our diverse learner pool, about half of whom are based in Africa, Latin America, and the Caribbean.

Many organizations, including our partners on the DMHO course, have used digital campaigns to share pertinent information about health topics and encourage recommended behaviors. While few academic studies have documented the direct impact of such campaigns on health outcomes, targeted social and behavior change communication has been used to improve health knowledge, attitudes, and health-seeking behaviors [9,18-20]. To develop the DMHO course, we combined evidence from multiple experts and experiences to produce a practice-based training that walks learners through the communication process, emphasizing behavioral theory and encouraging empirical evaluation of health outcomes. Designing and implementing campaigns without an evidence-based framework can limit their impact or have unintended consequences [21,22]. Skills-based massive open online courses (MOOCs), like DMHO, have the capacity to improve skills and shift attitudes toward important topics within and beyond public health. Similar evaluations have described successful launches of MOOCs to improve skill-based knowledge and reach a wide and diverse audience at a scale not possible with in-person training [23-25]. These evaluations noted that MOOC success is predicated upon engaging learners with creative and interactive teaching styles, and ensuring material is accessible in terms of format and language, all factors that we considered and tried to address when developing this course.

Our results highlight the need for further capacity strengthening, particularly in terms of applying best practices and evaluating impact. Most practitioners believe these skills are important, but many do not apply them in practice. While course completers reported they were more likely to use digital media often or always than baseline learners, most reported rarely doing so. Notably, many of the learners who completed the course were university students from Chile, and while they may not be using digital media to drive health outcomes now, the course may have exposed them to skills they could apply in the future. Similarly, many course completers indicated they were sometimes, often, or always evaluated health outcomes; however, nearly half reported never or rarely doing so. Evaluation of health outcomes is an important step in the development and implementation of effective digital health campaigns, but it can be challenging, particularly when the intended outcome is a behavior change. More information is needed to understand the say-do gap in the implementation of digital health communications among public health agencies, and how we can encourage public health practitioners to evaluate the impact of their campaigns.

This study has several limitations. All data are self-reported and subject to social desirability and bias, representing those who register and enroll for such a course. The diverse baseline sample mitigates this limitation, as do the results of the completer subanalysis. The subanalysis is limited by its small sample size, which is a function of the anonymity of the surveys. The global and likely heterogeneous nature of our results may obscure regional differences at baseline and endline assessments. While regional analysis was beyond the scope of this initial performance evaluation, we anticipate conducting future analyses at the regional level as course completion increases across regions.

Furthermore, our content is insufficient to keep pace with the dynamic and changing environment of social media platforms and the different ways that various platforms leverage their audience through advertising and other algorithms. Nonetheless, the course provides a foundation for how to approach any SBCC campaign through whatever digital means are prevalent for a target audience. Further investigation is also needed to understand why there is such a high level of enrollment yet a relatively small percentage of completers. Data provided by Coursera course management dashboards shows that most learners who do not complete the course drop off in the first module, either immediately after registering for the course (before the baseline survey) or before the first homework assignment. While this is consistent with other Coursera courses presented by Yale University, a deeper analysis is warranted to promote sustained enrollment in the course through completion.

Finally, this performance evaluation is not sufficient to assess the “practice gap.” Understanding the long-term outcomes of digital campaign development training is essential to improving communication capacity among public health organizations and should be the focus of future studies.

The DMHO course suggests the potential of a practical, theory-based massive online course to improve public health practitioners’ confidence and skills in using digital media to drive health outcomes. However, the need for further capacity strengthening is clear. Digital health communication is an increasingly useful tool for public health practitioners, one which—if wielded strategically—has the continued ability to reach different audiences with the critical information they need and improve health outcomes.