This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.
Communicating official public health information about infectious diseases is complicated by the fact that individuals receive much of their information from their social contacts, either via interpersonal interaction or social media, which can be prone to bias and misconception.
This study aims to evaluate the effect of public health campaigns and the effect of socially communicated health information on learning about diseases simultaneously. Although extant literature addresses the effect of one source of information (official or social) or the other, it has not addressed the simultaneous interaction of official information (OI) and social information (SI) in an experimental setting.
We used a series of experiments that exposed participants to both OI and structured SI about the symptoms and spread of hepatitis C over a series of 10 rounds of computer-based interactions. Participants were randomly assigned to receive a high, low, or control intensity of OI and to receive accurate or inaccurate SI about the disease.
A total of 195 participants consented to participate in the study. Of these respondents, 186 had complete responses across all ten experimental rounds, which corresponds to a 4.6% (9/195) nonresponse rate. The OI high intensity treatment increases learning over the control condition for all symptom and contagion questions when individuals have lower levels of baseline knowledge (all
This project demonstrates that exposure to official public health information increases individuals’ knowledge of the spread and symptoms of a disease. Socially shared information also facilitates the learning of accurate and inaccurate information, though to a lesser extent than exposure to OI. Although the effect of OI persists, preliminary results suggest that it can be degraded by persistent contradictory SI over time.
During a contagious disease outbreak, public health campaigns provide people with relevant information, including symptoms and methods of transmission. The public’s understanding is critical for people to know which behaviors they should avoid and whether they should seek medical attention. To this end, public health campaigns are led by federal, state, local, and other organizations, which we refer to as
Public health campaigns have been shown to be broadly effective. A recent examination of the COVID-19 pandemic in Italy found that most survey respondents knew of and believed in OI related to the disease [
People often rely on information from others to inform their own beliefs, attitudes, and behaviors related to health [
There is reason to believe that official, accurate health information is in competition with socially circulating inaccurate information. Rumors and misinformation are likely to spread across many domains [
This paper examines the interaction between official and social information (SI) about the disease. We use the term
Individuals’ psychological dispositions interact with information from the external environment when making decisions [
Public health information from official sources, where official sources include public health agencies such as the CDC and the World Health Organization (WHO), is likely to be accepted as factual by the public [
Although research has consistently found that individuals accept and view information from official sources as credible, the reality that individuals receive much of their information from social sources makes the overall information environment more challenging to navigate. Information from the social environment can contain correct, factual information about a disease, such as repetition of information shared by the CDC or WHO, or it can be misleading or even completely false. During the 2018 Ebola outbreak, for example, inaccurate socially supplied information about the disease was widespread, and individuals who believed in the misinformation were less likely to adopt preventative behaviors such as agreeing to be vaccinated and seeking formal care [
Several other features of the information environments that people encounter may affect how they process the messages they encounter. First, communication environments, particularly social media, are often dynamic in that there may be repeated interactions between users over time, and the use and effects of social media can be reciprocal [
In addition to being dynamic in time, communication on social media is often multidimensional even when it is limited to one overarching topic. For example, in the context of an infectious disease, two highly important dimensions of understanding the disease are the symptoms of the disease, so that a person may observe if they or others around them are likely to have contracted the disease, and how it spreads, so they know which actions are relatively risky or relatively safe in the context of the disease. Although it is important to know how people understand particular messages associated with either dimension in isolation, in many instances people process messages with multiple dimensions simultaneously, which may impact the effectiveness of a given message.
One of the primary features of social media is its ability to control self-presentation. Using the affordances of social media sites, users may selectively disclose information they wish their social contacts to know regarding their attitudes or beliefs [
Taken together, we expect that OI about a disease, that is, information received directly from official sources, will lead individuals to hold more factual beliefs about the disease. At the same time, socially shared information should lead to learning about disease, though to a lesser extent than OI. Specifically, we expect OI to be more effective in increasing learning about a disease when individuals are also exposed to accurate SI. On the other hand, we expect the effect of OI about a disease to persist but degrade as individuals are exposed to inaccurate SI over time.
To examine how official and SI interact in a dynamic information environment, we designed an experiment in which participants were exposed to both OI and structured SI over a series of ten rounds of computer-based interactions. The social connections between people are simulated with
The research design for this study was reviewed and approved by the institutional review board of The Ohio State University (protocol #2014B0543).
Participants fluent in German were recruited from a participant research pool of a European university in 2016 and were compensated with a modest monetary incentive. Students signed up to participate in short sessions scheduled during the course of a single week. All students participating in the same session were randomly assigned to the same treatment combination, as described below.
The experiment was conducted using the oTree [
Participants were then asked a battery of 14 true or false knowledge questions about hepatitis C, including six questions about the modes of transmission and eight about the symptoms. The topic of each question is listed in
Our novel 14 question instrument gauges participants’ professed knowledge about hepatitis C. We reduce the impact of guessing by providing an uncertain answer, as well as true and false. The correct answers to all of the questions are presented in
Summary of knowledge questions included in the experimental instrument, including the raw percent of participants who answered each question correctly at baseline, whether the question was subject to different social information, and the experimental round (if any) during which relevant official information was presented to participants in each intensity condition (high, low, and control). Note that infographics viewed by the low official information group were also viewed in the same round by participants in the high official information group.
Question topic | Correct answer | Correct at baseline (n=186), n (%) | Social manipulation | Round presented | ||
|
|
|
|
High | Low | Control |
Kissing | False | 96 (51.6) | Different across conditions | 1a | —b | — |
Loss of appetite | True | 81 (43.5) | Different across conditions | 3a | 3c | — |
Headache | False | 44 (23.7) | Different across conditions | — | — | — |
Vomiting | True | 78 (41.9) | Identical across conditions | 3a | 3c | — |
Unprotected sex | True | 154 (82.8) | Identical across conditions | 5a | 5c | — |
Fever | True | 127 (68.3) | Identical across conditions | 7a | — | — |
Fatigue | True | 140 (75.3) | Identical across conditions | 7a | — | — |
Needle sharing | True | 179 (96.2) | Identical across conditions | 9a | 9c | — |
Breastfeeding | False | 32 (17.2) | Identical across conditions | — | — | — |
Diarrhea | False | 35 (18.8) | Identical across conditions | — | — | — |
Skin rash | False | 43 (23.1) | Identical across conditions | — | — | — |
Hair loss | False | 83 (44.6) | Identical across conditions | — | — | — |
Gym equipment | False | 156 (83.9) | Identical across conditions | — | — | — |
Tattoo equipment | True | 165 (88.7) | Identical across conditions | — | — | — |
aOfficial information treatment group.
bNot available.
cSocial information treatment group.
We chose to measure knowledge about hepatitis C because it is a common disease with low public salience. Hepatitis C affects more than 3 million people in the United States (Department of Health and Human Services [
Ten experimental rounds were conducted at this point. We outline the overall structure of the rounds before describing how the treatments determine the nature of various components. Each round consisted of 3 parts, as shown in
We took care to create an environment in which participants believed that the observed SI was actually a response from other participants. First, participants completed the experiment in the same room as many other participants (sometimes participating in a different experiment). Second, our preprogrammed patterns of socially shared information were inspired by a pilot test of the experiment, such that relatively easy questions had more frequent accurate preprogrammed responses. Finally, we included time delays intended to mimic other participants that progressed more slowly. Although this SI does not mirror information shared from
Each round of the experiment consists of three parts. First, participants are exposed to official information, then they are exposed to ostensibly social information, and finally they are asked to respond to 14 knowledge questions.
Participants were assigned by session via cluster randomization to one of three degrees (
Depending on their assigned OI condition, each participant could view 5, 3, or 0 OI infographics, each of which contained information directly relevant to 1 or 2 of the knowledge questions during the course of the experiment.
The SI treatment conditions were intended to provide some participants with systematically accurate social influence and others with systematically inaccurate social influence, while maintaining the plausibility of real response sharing among participants. Thus, all participants viewed identical sequences of preprogrammed SI responses for 11 of the 14 knowledge questions. These preprogrammed responses were designed to mimic real participant responses to the same battery of questions deployed in a similar experiment. For the three remaining questions related to kissing, loss of appetite, and headache, participants assigned to the
Proportion of correct responses for a representative subset of knowledge questions, averaged across sessions within the same cross-classified treatment assignments, as indicated by line color (official information [OI]) and type (social information [SI]). Direct interplay with treatment assignments differs across questions, as indicated by the columns (OI) and rows (SI) of panels. For OI, relevant infographics are viewed by the groups of participants indicated in the column heading in the round indicated by the vertical gray bar. For SI, the proportion of correct answers provided by bots are displayed as o and the proportion of incorrect answers provided by bots are displayed as x symbols. These symbols are always overlaid in the first row.
For exploratory data analyses, we present question-specific longitudinal trends in the average proportion of correct responses across sessions stratified by treatment assignment. We colloquially refer to increases in the percentage of responses that are correct as learning, but recognize that these changes could also reflect changes in expressed attitudes rather than sincerely held beliefs. For formal statistical tests, we capture learning across rounds separately for each knowledge question via differences in average log odds ratios across sessions stratified by treatment assignment. This approach is similar to a random effects logistic regression where round is treated as a categorical factor but allows us to implement adjustments (
Differences in learning over rounds and across treatment groups were summarized by subtracting the appropriate average log odds or log odds ratios. Most within-group summary statistics compared the log odds of a correct answer in a particular round to those at baseline, and some compared responses at round ten with those in the round where correct OI was viewed.
Finally, we compare learning across groups by taking the differences in within-group average log odds ratios across relevant treatment conditions. Large differences across treatment conditions imply that treatment causes differences in learning. We used permutation to approximate the exact reference distributions. To test the equality of learning across groups, we estimated 2-sided
A total of 195 participants across 23 sessions consented to participate; only data for the 186 who completed all ten rounds were included in the primary analyses and are described in
We tested four expectations linked to the relationship between accurate official public health information and accurate SI. These expectations imply that increases in correct responses across rounds (which we term
Baseline demographic and other characteristics for study completers (n=186).
Characteristic | Values | |
Age (years), mean (SD; range) | 22.85 (2.44; 19 to 31) | |
Sex (female), n (%) | 121 (65.1) | |
|
||
|
Limited | 35 (19.4) |
|
Middle | 95 (52.8) |
|
Upper | 50 (27.8) |
|
||
|
City | 47 (25.4) |
|
Small town | 61 (32.9) |
|
Suburbs | 11 (5.9) |
|
Country | 66 (35.7) |
Freshmen or sophomore class rank, n (%) | 105 (56.5) | |
Political science major, n (%) | 34 (18.3) | |
Political orientation scorea (n=159), mean (SD; range) | −0.38 (0.75; −2 to 2) |
aPolitical orientation was self-assessed on a 5-point scale from
W
Visualization of data summaries highlights trends that may support our expectations. The lines in
For the question related to fatigue, we expected to see a similarly large learning in round 7 among the high intensity groups (and not among the low and control intensity groups who did not view a relevant infographic). However, differences in participant answers did not clearly follow this pattern, perhaps because of high baseline knowledge about this symptom or consistent longitudinal trends in bot responses. Further experiments are required to confirm this hypothesis. Finally, as expected, we see no substantial sustained learning about the potential contagion of breastfeeding in any group, as no group was shown a relevant infographic. Among the questions in the second row where the SI treatments differed, we also see substantial learning in rounds where relevant infographics are viewed, although these trends are complicated by differential socially shared information, as explained below.
Formal tests that compare groups assigned to view relevant infographics with those that did not are summarized for germane questions in the first row of
Observed (points) and permutation reference distributions (violins) for observed contrasts in average log odds of correct responses across treatment assignment groups (first two rows) and in difference-in-difference of average log odds (third row). Stars indicate permutation
We expect (in) accurate socially shared information to affect learning of (in) correct information. For three of the questions (related to kissing, appetite, and headache), the (in) accurate bot responses were always (in) accurate; for the remaining 11 questions, all responses were identical across both types. Thus, for the 3 questions, we expect those assigned to the accurate bot treatment groups to learn more than those assigned to the inaccurate bot groups. The second row of
significant differences in learning from baseline to round 10 across the assigned bot types for all three questions, where the information shared by bots differed (all
To the extent that official and SI interact, we expect OI to overwhelm SI. That is, we expect the effect of socially shared information to be smaller within groups who viewed germane infographics as compared with those who did not view germane infographics. Infographics relevant to appetite and kissing were shown to some groups, and the accurate and inaccurate bots differed in their responses to these questions. Thus, we focus on these two questions to explore information-mode interactions.
First, we consider the question related to appetite (lower left panel of
Treatment group-specific change in average log odds from baseline to round ten. Dotted lines connect pairs of groups with the same official information treatment group; parallel lines suggest no interaction between information modes.
Although we saw no global interactions among the effects of mode on learning from baseline to round 10, we explored a fourth, more targeted hypothesis that the effect of receiving official public health information persists but degrades in the face of contradictory SI. For the question related to appetite, we see that high- and low-information intensity groups achieved nearly unanimously correct responses after viewing the infographics in round 3 (all 4 yellow and blue lines in
It may also appear that there is more degradation in learning because of inaccurate SI for participants in the low OI group versus the high OI group (
This work demonstrates that both official and SI influence people’s reported understanding of infectious diseases. As expected from the previous literature, exposure to official public health information about hepatitis C increased the learning related to its spread and symptoms. Learning followed socially supplied information, whether accurate or inaccurate, though to a lesser extent. Trends in our data suggest that learning from OI is remarkably resilient even in the presence of persistent contradictory SI, although some modest degradation may be hidden by low statistical power. In an era in which official public health campaigns are frequently in competition with information shared on social media, this study provides some reason to be optimistic that public health campaigns may be able to overcome socially shared misperceptions.
Our work is consistent with previous work on misperceptions in the context of health, which finds that corrective information may inhibit false beliefs [
This study addresses the longitudinal effect of SI about diseases in tandem with official public health information in an experimental setting. Many studies of this type would include a one-shot intervention or follow up with participants after a short period. Here, we investigated how expressed beliefs update over multiple rounds and through multiple interactions with members of their experimentally constructed social networks. Because of this, we are able to provide a more nuanced understanding of competing messages in dynamic information environments.
It is important to note that our study used expressed beliefs about hepatitis C as the primary dependent variable. Of course, nearly any study of beliefs about disease relies on expressed beliefs in one way or another. However, because our study is premised on participants’ understanding that they are viewing the responses of other participants who are completing the study contemporaneously, these expressed beliefs may be impacted by not only their own beliefs and any changes in them because of the experimental stimuli but also by social desirability bias that may impact their expressed beliefs [
Although our work focuses on the context of a contagious disease, the results may contribute to our understanding of how social media may affect other aspects of health or may relate to other domains. Of course, OI may be in competition with (or be reinforced by) socially shared information across a number of domains, including other diseases, such as Zika [
Our ability to control available information lends strong credibility to the causal interpretation of our results. Experiments on networks help bridge the gap between observational studies of people in their natural social environments and lab experiments that abstract away social influences.
Reliable extension of our conclusions to real-world situations relies on participants’ interpretation of the information presented as legitimate. This hurdle may be easier to clear for OI than for SI. One major strength of our controlled experiment is that participants were embedded in an environment that promoted realism in a fictional social network.
This experiment has several important limitations, which we elaborate on here. First, the social networks we examined appeared to be among participants who were anonymous to one another within the setting of the experiment. Although this situation may be encountered on an anonymous message board or comment thread, it is possible that in real social networks, in which participants have strong social bonds and reputations, the effects of SI may be different. Future work may wish to investigate whether and how SI is transmitted in social networks among participants who are already connected to one another.
Second, our experiment limited SI responses to true, false, or unsure. In reality, people make intentional and nuanced attempts to convince others in the social sphere of their beliefs. It is difficult to envision a controlled experiment in which participants are allowed to communicate so freely and also maintains a degree of control that enables clear causal interpretation, but this should be addressed in future studies. We view this as a trade-off between tight control that enables us to make clear causal claims and less tight control that not only enables a greater variety of communication but also reduces the ability to interpret the social component in a clear causal fashion.
Third, the participants in our study were all similar in age. Previous work in a different context (politics) found that an official message with a reinforcing social component was more effective at changing behavior among older participants than an official message without the social component [
Fourth, the study did not test for potential mediating and moderating variables that may explain how or to what degree official and SI affect beliefs. This limitation is related to, but distinct from, concerns about how anonymous networks may impact network effects. Mediating and moderating relationships may be a result of aspects of existing relationships, in which case anonymous studies would help to control for these effects but would also limit researchers’ ability to examine them. However, it is also possible that mediating or moderating relationships are a result of aspects of relationships between people that develop rapidly, perhaps during the course of a few interactions. In this case, even in a short network experiment among anonymous individuals, mediating or moderating relationships may be examined. For example, the degree of trust in information sources may affect the degree to which participants respond to the beliefs shared by either official or SI sources. In this study, participants were informed that they would be connected to other anonymous participants; so, any interpersonal trust would have to be established during the course of the study. We believe it is unlikely that, in the context of this study, varying levels of trust would be created during the course of the study. However, in real-world situations in which people have ongoing relationships with those from whom they receive SI, trust and reputation are likely to impact how people process and respond to the messages shared by their social contacts. Future studies may wish to investigate trust, perhaps through experiments that enable researcher control over not only message content but also cues related to trust. We note that trustworthiness is one of multiple possible mediating or moderating variables that may enable a more nuanced understanding of the competing or reinforcing effects of social and OI.
Finally, the small number of experimental sessions limit the power of permutation-based statistical inference. The permutation-based approach used here makes very few assumptions and, therefore, enables us to make clear claims based on the statistical evidence that does not depend on common assumptions used in model-based analyses that may not hold for the data we collected. Future reanalyses based on longitudinal models may provide more nuanced, though more model-dependent results. In addition, reliance on college student volunteers and participant buy in to treatment legitimacy raises external validity concerns.
Nonetheless, our results provide some good news for sponsors of public health campaigns even in a time of overwhelming prevalence of SI with a wide range of accuracy. Consistent fact-based official messages can break through the SI noise.
This experiment is one of multiple related projects examining social processes of information spread in networks and includes 2 additional experiments that address some of these limitations. First, human participants are randomly assigned to prespecified networks, which provides insight into how aspects of the network structure affect the diffusion of information about the disease. Second, participants share disease information with individuals in a real-world social network, providing a test of the external validity of our findings. Although this analysis focuses solely on the
In the real world, our results present a puzzle for web-based health communities, particularly those that are moderated by professionals. In such communities, moderators may play a role similar to that of the official sources of information in our experiment [
Our results underscore the importance of public health practitioners taking into account the effects of both OI sources, which may have a degree of control over, and SI sources, over which they may have limited control. We also show that OI is substantially more effective at promoting learning than socially supplied information, underscoring the importance of public health campaigns for inducing correct beliefs. Although we do not find evidence that official and SI interact, it is possible that the effects of such an interaction are small and nuanced, and we would nonetheless encourage practitioners to consider how OI may be transmitted through social ties.
Supplementary information concerning the design, analysis, results, examples of the treatments used, and primary dependent variable measurement.
Centers for Disease Control and Prevention
official information
social information
World Health Organization
This work was generously supported by the Alexander von Humboldt Fellowship for Experienced Researchers, the National Science Foundation (SES1357622, SES1461493, and SES1514750), and the National Institute of Drug Abuse at the National Institutes of Health (R34, DA043079). We thank the three anonymous reviewers for their insightful comments.
None declared.