The effectiveness of using Facebook to deliver the CPE program was measured by the knowledge gain between the pre and postintervention knowledge tests, calculated as the difference in scores. The knowledge gain in the two groups was also compared.

The knowledge tests consisted of formative evaluation, 20 multiple choice questions, and the 25-item Dementia Knowledge Assessment Scale (DKAS). Content for the multiple choice questions was drawn directly from the learning materials throughout the 8 learning weeks. The DKAS has been validated as a reliable measure of dementia knowledge for a wide spectrum of populations, including health professionals. The DKAS elucidates respondents’ knowledge characteristics across four coherent domains: Causes and Characteristics, Communication and Behavior, Care Considerations, and Risks and Health Promotion [33]. A pilot assessment of the knowledge tests was performed to determine the coverage of questions, level of difficulty, and potential ceiling effects.

Participant compliance was quantified by the number of week(s) (out of 8) the material was accessed by the participants, recorded by the “Seen by” feature of Facebook in the intervention group and by the Bitly link in the control group.

In Facebook, the “Seen by” feature of a post automatically records the individual group members that had seen the post (Figure 1). In the control group, each participant had their own 8 unique Bitly links to access the corresponding 8 weeks of materials. A participant clicking on the Bitly link was counted as having accessed the materials for that week. Bitly links were also created to track participant compliance in accessing the recommended external learning resources such as videos and websites.

In studying the communication artifacts occurring in Facebook, engagement was defined as the metric of actions that participants took in the group during the intervention period. In addition to ordinary actions such as comment and reactions, a unique feature of the Facebook Social Learning Group is the “Done” button (Table 1). Participants were encouraged to click the “Done” button to share their learning progress. A set of descriptive data was also extracted from analyzing participant engagement in the intervention group, considering features that have been generally used in previous Facebook-based studies [30,31].

All participants were invited to complete a satisfaction evaluation form developed for the effective appraisal of training delivery for health professionals [34]. The form contained 10 5-point Likert-style positively phrased statements focusing on the achievement of program objectives, impact on learning, learning experiences, and application to clinical practice. There were 3 questions that evaluated the program length, information quantity, and difficulty level. A pilot assessment of the evaluation form was performed to determine the coverage of questions.

Participants’ attitudes on the professional use of Facebook were evaluated in a questionnaire modified from the Continuous Medical Education App Attitudes Survey Instrument, a validated measure of participant attitudes toward mobile app use in a continuous medical education program [35]. There were 10 5-point Likert-style positively phrased statements evaluating the educational value, acceptability, usability, and future feasibility of using Facebook to deliver the program. A pilot assessment of the survey was performed to determine the question coverage.

To investigate the pilot study’s feasibility and acceptability, all participants were invited to complete a feedback form on their learning experiences and to take part in short semistructured interviews. To encourage participation of interviews, reminders were sent twice by Facebook messages (intervention group) and emails (control group).

In the feedback form, qualitative data were obtained via two free-form questions that asked the participants about their perceptions of the most valuable sections as well as suggestions for improvement.

The semistructured interviews were conducted following a flexible interview guide to explore a wider range of respondents’ perceptions and comments. Participants were invited to share their views on (1) promotion, recruitment, and randomization arrangement; (2) delivery of the educational intervention; (3) usability of the learning materials; (4) feasibility and acceptability of the study; and (5) comments for further improvement. Respondents could refuse to answer any question or ask questions. Interviews were audio-recorded and transcribed manually. Data were organized using a Microsoft Excel 2016 spreadsheet.

Thematic analyses of the contents of feedback forms and semistructured interviews were conducted independently by the authors. Discrepancies on data analyses, coding, and themes generated were resolved by discussion until consensus was reached [36].

A literature search did not identify any RCTs conducted on the effects of an educational intervention utilizing SNSs that could be used as a basis for calculating sample size. This study conservatively assumed a small effect size of 0.2. For a full RCT designed with 90% power and a two-sided 5% significance threshold, the recommended sample size for a pilot trial is 30 per group [37]. Assuming a dropout rate of 10%, the total target number of participants was 66. Slightly more participants (N=80) were recruited for this study due to positive responses from professional bodies.

Participant demographics and SNS usage characteristics were treated as categorical variables. The Chi square test and Fisher exact test were used to analyze dichotomous and categorical variables. Participant performance on the knowledge test is presented descriptively. To compare the knowledge gains of the two groups (primary outcome), an independent sample t test was used. The effect size of the outcome measure between the two groups was estimated according to the formula proposed by Morris [38] for mean pre-post changes between the intervention and control groups. An intention-to-treat approach was adopted in this study. Scores in the evaluation and survey on the use of Facebook (secondary outcome) are presented as descriptive statistics and were analyzed by a two-sample t test or Fisher exact test. P≤.05 was considered significant. Statistical analyses were conducted using R v. 3.1.2 (R Foundation for Statistical Computing, Vienna, Austria) and IBM SPSS Statistics version 23 (IBM Corp, Armonk, NY, USA).

Figure 2 shows the flow of study subject recruitment and randomization according to the CONSORT guidelines. Eighty participants underwent randomization with 40 allocated to the intervention group and 40 allocated to the control group.

The overall retention rate was 75% (60/80). Significantly more intervention group participants (35/40, 88%) completed the study compared to those in the control group (25/40, 63%; P<.001). Twelve participants, six from each group, attended postintervention interviews.

At baseline, participant characteristics and habits when using SNSs were comparable in the two groups (Table 2 and Multimedia Appendix 1). The majority (64%, 51/80) of participants ranged in age from 20-39 years. The participants, who worked in various health disciplines, included pharmacists, nurses, optometrists, and physiotherapists.

Among all participants, 95% were daily SNS users. WhatsApp Messenger and Facebook were the most frequently used SNSs, followed by Instagram, WeChat, and Line. SNSs were mainly used for personal use and communication as well as for general knowledge. Less than half of the participants had used SNSs for professional communication. Only two participants had previously participated in a CPE program delivered via SNSs (Multimedia Appendix 1).

At baseline, the knowledge test overall mean scores were similar in the two groups (Multimedia Appendix 2). In both groups, the lowest mean correct responses in the DKAS were found in the subscale Communication and Behavior.

At the postintervention assessment, intervention group participants demonstrated significant knowledge gains, reflected by significant increases in the total DKAS scores, each of the DKAS subscale scores, and multiple choice questions (Table 3). In the control group, significant knowledge gains were not identified in the DKAS subscales Communication and Behavior and Care Considerations.

A significant difference in changes in the mean knowledge gain scores between the two groups was identified in the DKAS subscale Communication and Behavior, with a medium effect size (Table 3). The CPE program delivered by Facebook was more effective than the control (email provision) at increasing participants’ understanding and knowledge of the communication and behavior of people with dementia. There was no significant difference between the two groups in the total DKAS, multiple choice questions, and other DKAS subscale scores.

The effect size in the change in the overall DKAS score was small. For a full-scale two-armed RCT, assuming a standard error of 5% with 95% power, a total of 272 participants would be required (ie, 136 per group). With attrition rates of 10%, 20%, or 30%, the total number of participants required is 304, 340, and 390, respectively.

Participant compliance with the educational intervention was significantly higher in the intervention group (mean 6.30, SD 2.28 weeks) than in the control group (mean 3.53, SD 3.21 weeks) (P<.001). Overall, 75% of the intervention group participants completed at least 6 weeks or more of the intervention compared to only 35% of the control group participants (Table 4). The mean number of participants accessing the materials was 31.5 (SD 3.9) and 17.6 (SD 5.2) in the intervention and control group, respectively.

Participant compliance was significantly higher in the intervention group (mean 14.9, SD 14.1) than in the control group (mean 2.1, SD 1.8) (P<.001) with respect to accessing the 18 external learning resources (9 videos and 9 website links) (Multimedia Appendix 3). Participants who learned via Facebook were significantly more compliant in viewing multimedia content.

A total of 97 Facebook posts were created for the intervention group. All posts received a considerable number of participant engagement actions (mean 19.03, range 11-33). Polls, which were question-related posts designed to encourage member interaction, attracted the highest degree of participant engagement (mean 20.17), followed by videos (mean 19.56). Links (mean 17.33) generated the lowest number of engagement actions during the intervention period.

As illustrated by item 10 in the satisfaction evaluation (Table 5), participants were satisfied with the training overall. There was no significant difference between the two groups. Evaluating achievement in each of the four course objectives (items 1-4), the overall mean scores ranged from 3.8 to 3.9 (SD 0.6-0.7). More than 80% of participants regarded the program length, quality, and difficulty level as appropriate.

Overall, the respondents were positive about using Facebook for a CPE program. Intervention group participants rated the use of Facebook significantly higher (P=.03) (Table 6).

Significant differences between the two groups were identified in responses to the survey questions “I would recommend Facebook for other CPE courses” (P=.01), “Using Facebook improved my learning experience” (P=.02), and “I would be more likely to attend a CPE/CNE course if it used Facebook” (P=.02).

At completion of this pilot trial, participants who had learned via Facebook were more willing to use Facebook in future CPE programs.

Twenty-nine responses were collected from the feedback forms. Twelve participants, six from the intervention group and six from the control group, attended the semistructured interviews. A wide range of inputs were collected, including comments on current models of dementia care and CPE frameworks. We here focus on the findings that helped in evaluating the design of the study and the feasibility and acceptability of interventions. The qualitative data analyses led to the generation of the following themes.

This pilot RCT, as the first to evaluate the effects of using an SNS to deliver a CPE program, demonstrates how a novel educational intervention can result in significant knowledge gains in dementia care to health professionals. The high participant compliance and engagement, together with high satisfaction and acceptability, indicate that Facebook can be an effective and feasible learning tool that could easily be rolled out in practical settings.

The Facebook intervention succeeded in offering the participants significant knowledge gains in all aspects of the knowledge assessments.

Comparing the means of changes in scores between the two groups, the Facebook intervention excelled at producing significant knowledge gains according to the DKAS subscale Communication and Behavior with a medium effect size. The six items included in this subscale assess how respondents interpret the behavioral symptoms of people with dementia and the proper way to communicate efficiently. Communication is a crucial skill in dementia care, but has been difficult for health personnel to learn [39]. For effective learning, rather than using plain text, all participants were instructed to view a collection of videos, websites, and online resources illustrating actual patient behaviors and communication. Although participants in both groups could access the same resources, significantly more intervention group participants accessed the materials. In addition to attracting participants by showing previews of multimedia contents, Facebook’s mobile app enables users to browse external links within the interface. In contrast, the control group participants tended to skip the links, because opening a separate browser appeared to disrupt their reading. Likewise, respondents in previous studies appraised the key benefit of SNSs as being efficient channels for a complex stream of information and multimedia files with no restriction in terms of location or office hours [40,41].

Facebook includes important features affecting participant performance in an online course, improving instructor effectiveness in facilitating learning [42]. On Facebook, posts were formatted in a concise and appealing way to reduce readers’ cognitive loads. With the Facebook tracking function, participants could review the contents nonsequentially and refer back to materials based on their study preferences. By contrast, in the control group, participants usually perceived reading the learning materials as a heavy workload that hindered their learning. Therefore, Facebook is a more effective learning platform by facilitating the integration of learning into routine habits of daily life.

Learning through Facebook encouraged participant engagement and compliance. Indeed, participant retention in the intervention group (88%) was remarkably higher than that in the control group (63%) at study completion. Compared to other Facebook interventions for patients or the public, similarly high retention rates have been noted in studies promoting health-related behavioral changes such as running (78%) [43], physical activity (86%) [44], and weight loss (96%) [45]. The sense of presence and interactions with members in the Facebook group were appreciated by respondents in the evaluation and interviews. A higher degree of engagement could contribute to higher attendance [46], explaining the promise of learning through Facebook. However, the control group participants lacked a sense of belonging and involvement, which may explain the high dropout rate in this group.

Facebook is superior to email communication in providing a learner-centered, instructor-facilitated, more interactive platform for participants, which are demonstrated influencing factors of successful CPE programs [47]. Of note, in many jurisdictions, CPE schemes are voluntary, making it a challenge to motivate health professionals to complete learning programs [48,49]. The main deterrents to participation in a CPE include cost, resources, inability to get time off from work, household responsibilities, and lack of time [48,49]. Facebook features good usability to be an attractive CPE program by providing a learning platform unrestricted by time and geographical factors.

Analyses of intervention group activities were conducted to determine the post type generating the highest participant engagement. Polls attracted the most engagement, followed by videos. Links generated the lowest engagement rates, because they were perceived to lead to an external site, disrupting participants’ reading on Facebook. These findings are consistent with previous studies in the marketing field investigating brand engagement [30,31].

Participants in this study were found to have positive views toward utilizing Facebook for a CPE program.

Intervention group participants rated their learning experience as significantly improved when using Facebook. They were significantly more supportive of recommending the use of Facebook for other CPE courses. Participants appreciated the convenience, flexibility, and usability offered by Facebook that enables ready participation for health professionals living in different regions, negating geographical boundaries. Likewise, respondents in previous studies appraised the key benefit of SNSs as being efficient channels with no restrictions in terms of location or office hours [40,41].

With the extensive use of SNSs in our daily lives, these positive attitudes toward Facebook highlight the feasibility of conducting further research on SNS use by health professionals.

This pilot RCT has several strengths, including the randomized controlled design, learning curriculum developed from a multidisciplinary approach, and evaluation of a novel educational intervention utilizing Facebook with both quantitative and qualitative research approaches. The utilization of validated tools allowed for reliable quantitative measures. The qualitative data analyses enabled gaining deeper understanding of experiences on the educational interventions and identification of factors that were important to the participating health professionals.

There are limitations of this study that warrant mention. Health professionals who had been frequent SNS users were more willing to join this study, which introduced bias in evaluating users’ perceptions of Facebook. A majority of participants were aged between 20 and 39 years. These factors would affect the generalizability of the results to the overall workforce.

Many control group participants dropped out of the study, which made the overall dropout rate (25%) larger than the assumption in estimating the sample size (10%). This may have introduced bias and reduced power, affecting the generalizability, validity, and reliability of the results [50]. This issue is slightly mitigated by the increase in sample size from 60 to 80 and the intention-to-treat approach. However, it is understood that the estimate of effect becomes conservative due to dilution as a result of dropouts [51]. Very few participants took part in the semistructured interviews. Therefore, these responses might not adequately reflect the experiences of all those who completed the study, and response bias was likely.

For practicality, the knowledge tests were self-administered, which reduces the ability to standardize test conditions. Although participants were reminded not to check other resources, it was unfeasible to prevent participants from seeking help from others or checking references.

This pilot RCT brings forward several implications for further research and practice.

Facebook was found to offer substantial advantages over conventional online CPE programs, being beneficial to those who prefer distance learning, while maintaining interactions with classmates. Leadership by the administrator and member engagement were identified as crucial factors in SNSs [52]. A larger trial is required to thoroughly examine the potential effectiveness of an SNS-based educational intervention as compared to traditional learning approaches. Given the effect size of 0.44 of this study, the total number of participants required would be 272 for a full-scale two-armed RCT.

The benefits of utilizing multimedia components on Facebook highlight strategies for designing suitable educational content delivered by SNSs. Facebook can be an effective training tool for understanding patients or caregiver communication and behavior. Not restricted to dementia care, Facebook may also be used to deliver training on other mental health conditions such as depression and schizophrenia, or behavioral conditions, including speech and movement disorders.

In future research, in addition to measuring changes in knowledge, the impacts on practical implementation and even patient care would be important to investigate. However, the complexity of behavioral change increases as evaluation of an intervention ascends the hierarchy [53]. For dementia, there are some feasible, validated measurement tools available for this purpose, including attitudes toward dementia and self-confidence in caring for people with dementia [54].

Further research would consider including an economic analysis to investigate the cost of delivering CPE through Facebook compared to other channels such as face-to-face meetings and email communication. One of Facebook’s economic advantages is that the post content can easily be reused. Owing to Facebook’s user-friendly features such as post scheduling, the resources required for program administration are minimal.

Significantly more intervention group participants completed the study than control group participants (P<.001). The high levels of participant compliance and engagement make Facebook a promising method for delivering professional educational programs. Attending a CPE program through Facebook significantly improved participants’ knowledge of dementia (P<.001). Compared to the control group, although a significant difference was not observed in the mean of changes in total scores, the Facebook intervention was significantly more effective at improving participants’ knowledge of communication and behavior in dementia patients (P=.02). Participants demonstrated positive attitudes toward utilizing Facebook for professional learning. As the first pilot RCT examining the effectiveness of an SNS for professional education for health professionals, the findings of this study provide evidence for the feasibility of using Facebook for intervention delivery in a practical setting.