This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.
The public typically believes psychotherapy to be more effective than pharmacotherapy for depression treatments. This is not consistent with current scientific evidence, which shows that both types of treatment are about equally effective.
The study investigates whether this bias towards psychotherapy guides online information search and whether the bias can be reduced by explicitly providing expert information (in a blog entry) and by providing tag clouds that implicitly reveal experts’ evaluations.
A total of 174 participants completed a fully automated Web-based study after we invited them via mailing lists. First, participants read two blog posts by experts that either challenged or supported the bias towards psychotherapy. Subsequently, participants searched for information about depression treatment in an online environment that provided more experts’ blog posts about the effectiveness of treatments based on alleged research findings. These blogs were organized in a tag cloud; both psychotherapy tags and pharmacotherapy tags were popular. We measured tag and blog post selection, efficacy ratings of the presented treatments, and participants’ treatment recommendation after information search.
Participants demonstrated a clear bias towards psychotherapy (mean 4.53, SD 1.99) compared to pharmacotherapy (mean 2.73, SD 2.41;
We conclude that the psychotherapy bias is most effectively attenuated—and even eliminated—when popular tags implicitly point to blog posts that challenge the widespread view. Explicit expert information (in a blog entry) was less successful in reducing biased information search and evaluation. Since tag clouds have the potential to counter biased information processing, we recommend their insertion.
In the last decade, patients’ preferences have increasingly been taken into account when choosing a treatment for depression [
This paper investigates how biases like this one can be reduced. For our study, we chose the domain of depression treatment and made use of the psychotherapy bias. Specifically, we expected that laypeople’s bias towards psychotherapy leads to a confirmation bias in information search and evaluation. The confirmation bias refers to the robust findings that individuals tend to process information in a manner that confirms their pre-existing beliefs. Therefore, a confirmation bias in searching for information is not only of interest for depression treatment or the comparison of psychotherapy and pharmacotherapy, but for health-related information search in general. Individual convictions lead to one-sided information processing. When these convictions are not justified by scientific evidence, people run the risk of being misinformed.
Therefore, we investigated two factors that might reduce one-sided information processing. One of the most reliable and objective information sources on the Web is expert information. We tested whether facing explicit expert information would reduce the bias. Moreover, we were interested if aggregated expert information presented in tag clouds would reduce the bias as well.
In the last decade, the Internet has become one of the most important sources for health-related information [
In order to provide an overview of the relevant content of a blog and to organize related blog posts, popular blogging sites such as Technorati, WordPress, or Counselling Resource [
Tags have two important functions. First, tags organize content. When people provide the same tag for different blog posts, blog posts with a common topic are quickly found via a common tag (eg, the topic with the tag “health” on WordPress [
Previous research on the perception of tag clouds has demonstrated that the popularity of tags (presented as tag size) influences information search and information evaluation [
Tag cloud versions used in the study.
In order to investigate the confirmation bias in health-related information search, we chose the topic of depression treatment with pharmacotherapy or psychotherapy because previous research has demonstrated a discrepancy between laypeople’s beliefs and scientific evidence. As mentioned, psychotherapy is viewed to be more effective [
The confirmation bias describes people’s need to confirm their beliefs and attitudes when engaged in search for online information [
Accordingly, our first hypothesis is that the psychotherapy bias—the conviction that psychotherapy is more effective than pharmacotherapy—leads to a confirmation bias in online information search where people prefer to select psychotherapy-related tags and content (H1).
If this confirmation bias determines information search, the question arises as to how the bias can be reduced. Research has shown that people perceive expert information as credible [
Participants were recruited via two mailing lists, to which mostly university students from a broad range of disciplines had voluntarily enrolled. They were provided with a link that led them to a fully automated online survey. We reminded all participants twice via email to take part in the study. We did not use cookies or an IP (Internet protocol) check to detect or prevent multiple participation. However, all the provided email addresses were unique. There were no specific eligibility criteria with the exception of computer literacy as an implicit criterion. In order to have an 80% chance to detect a moderate effect (
We outlined in the invitation mail that we were conducting a study on the treatment of depression, with the main task of rating short blog posts about different treatment options. We emphasized that participation would be voluntary, could be withdrawn at any point, and that the study would not cause harm of any kind. We also assured anonymity and the option to withdraw the data at the end of the study without providing reasons. Participants were informed about the duration of the study and the possibility to win €25 or €50 Amazon gift certificates. They were informed that by clicking the next button, they would provide informed consent. Moreover, they were asked to contact the experimenter (email was provided) in case of questions or considerations of any sort. There was no institutional affiliation presented in the invitation mail, but during the online study (see upper left part of
Screenshot of information search environment.
The study comprised a 2 (prior expert information: supporting, challenging) x 3 (tag popularity: psychotherapy, balanced, pharmacotherapy) between-subjects design. Participants were randomly assigned the following simple randomization procedures (computerized random numbers) to the different treatment groups, with the only restriction that a maximum of 35 individuals (who completed the study) were allowed per condition. We manipulated
As a second factor, we manipulated
After the first two pages where participants were informed about the study and provided informed consent, the algorithm randomly assigned participants to one of the six conditions and a series of online forms followed. Participants filled out demographic data, followed by questionnaires (eg, prior beliefs about treatment efficacy, cf. measures section).
In the first phase of the experiment, participants read two blog entries. Participants were randomly assigned to read either two blog posts emphasizing the efficacy of psychotherapy (supporting the bias, n=93) or to read two blog posts emphasizing the efficacy of pharmacotherapy (challenging the bias, n=81) in the treatment of depressive disorders. The first blog entry reported that a large global network of “neurologists and psychologists” (expert information) agree on the efficacy of either pharmacotherapy or psychotherapy in the treatment of depression. The second blog entry presented the positive results of a neuroimaging evaluation study, arguing for the respective interpretation. Prior information was held constant, so the reasoning in both conditions was exactly the same; we interchanged only the terms antidepressants and psychotherapy. Note that no comparison to other types of treatment was provided in the blog posts. After each blog post, participants rated its persuasiveness.
After the first phase, participants were informed about the nature of tags and tag clouds. It was stated that tags describe and categorize online content, and an example of a tag cloud was shown. Participants were told that experts provided the tags in the following task. The more often a certain tag had been provided by these experts, the larger the tag in the cloud appeared. Therefore, participants were aware that large tags described popular topics among experts.
In the second phase of the experiment, participants searched for treatment-related information. The task for participants was to find useful information to provide information to a hypothetical friend who suffered from major depressive disorder. After the instructions, the information search environment appeared. Participants were randomly assigned to one of the three versions of a tag cloud (
At the end of the study, all participants were thoroughly debriefed and informed about the fact that the presented materials were not genuine materials and that tag clouds thus did not reflect actual scientific knowledge but had been experimentally designed.
The two blog posts in the two different conditions of expert information contained matched main arguments for the efficacy of psychotherapy versus pharmacotherapy. Therefore, all blog posts in this study were fictitious. The first blog post in both conditions described the establishment of a database with scientific studies by an extensive and worldwide network of researchers. The second blog post in both conditions described the successful remediation of neuronal brain activity and brain structures, after treatment with either psychotherapy (supporting prior expert information) or pharmacotherapy (challenging prior expert information). Text length ranged from 98 to 118 words.
The tagging environment for information search consisted of two main sections (
At the left side of the screen in the tagging environment, for each tag, related blog posts were presented (
The tagging environment displayed in the Web browser (programmed with Adobe Flash Builder) was developed by software developers at the Knowledge Media Research Center. The tagging environment was used for the first time; there were no changes of functionality during the period of data collection. Personal information (email address, demographic data) was stored separate from the survey data on a local server.
Items of all the questionnaires were in fixed order; up to 7 items were displayed per screen. We implemented a completeness check so no items could be skipped by participants. Participants could not use a back button of the browser or within the survey. The measures are described in the order they appear in the experiment.
Prior knowledge about depressive disorders was examined by 24 items regarding general knowledge (eg, false: “Women suffer from depressive disorders as often as men do”; true: “People suffering from diabetes are more likely to suffer also from depressive disorders compared to the general population”) and symptoms of depressive disorders according to the Diagnostic and Statistical Manual of Mental Disorders, 4th edition (DSM IV) and the International Classification of Diseases (ICD) 10 (eg, true: “Depressive disorders are often characterized by heightened or lowered appetite”; false: “People with a depressive disorder show an obsessive need for cleanliness and order”). The answer format had the three categories: true/false/I don’t know (Cronbach alpha=.72).
Efficacy ratings were inquired for all the treatments that were presented prior to and after the experimental manipulations (see pre- and posttest,
Study procedure.
After reading each of the two prior blog posts, participants rated the degree to which each blog post stated the efficacy of the presented treatment (either psychotherapy or pharmacotherapy) on a 7-point Likert scale (1=I agree not at all, 7=I completely agree). This rating served to ensure that the texts in both prior expert information conditions were equally convincing.
In order to analyze the psychotherapy bias in information search, the number of selected pharmacotherapy tags was subtracted from the psychotherapy tags. Thus a positive value represented a searching bias towards psychotherapy. The same procedure was applied to the number of blog posts that participants read.
After the experimental manipulations, participants were asked to provide a treatment recommendation for a hypothetical friend. They were instructed to give reasons for the recommendation in about five sentences. Recommendations were coded from 1-5 (5: recommendation for psychotherapy only, 4: psychotherapy preferred, 3: combination therapy, 2: pharmacotherapy preferred, 1: pharmacotherapy only).
At the end of the study, participants had the opportunity to provide qualitative feedback through a feedback form.
In order to test our main hypotheses, we conducted a 2 (prior expert information: supporting, challenging) x 3 (tag popularity: psychotherapy, balanced, pharmacotherapy) ANOVA with planned contrasts for the factor tag popularity. With additional
Initially, 440 individuals followed our invitation and started the online experiment. As can be seen in
Sample characteristics (N=174).
Characteristics | n | % | |
|
|||
|
Not yet graduated | 130 | 74.7 |
|
Graduated | 43 | 24.7 |
No higher education | 1 | 0.6 | |
|
|||
|
Health care related subject | 37 | 21.3 |
|
Non–health care related subject | 126 | 72.4 |
Not specified | 11 | 6.3 | |
|
|||
|
15-19 | 26 | 14.9 |
|
20-24 | 97 | 55.7 |
|
25-29 | 36 | 20.7 |
|
30-39 | 10 | 5.7 |
|
40-49 | 4 | 2.3 |
62 | 1 | 0.6 | |
Total | 174 | 100 |
Participant flow diagram.
First, we checked the equivalence of groups regarding participants’
In order to assure equivalent
In the following analyses, we investigated whether participants showed a psychotherapy bias regarding pre-existent beliefs. To this end, we analyzed efficacy ratings of psychotherapy and pharmacotherapy that had been assessed prior to the information search. Efficacy ratings on a scale ranging from 1-7 showed that participants expressed strong superiority of psychotherapy (mean 4.53, SD 1.99) over pharmacotherapy (mean 2.73, SD 2.41;
We first tested whether the psychotherapy bias emerges in information search (H1). This hypothesis was confirmed, since participants generally selected more psychotherapy tags (mean 4.66, SD 2.28) compared to pharmacotherapy tags (mean 3.87, SD 3.35;
Beyond demonstrating the biased information search behavior, we hypothesized that the psychotherapy bias is reduced by providing prior expert information (H2) and popular tags (H3) that challenge the psychotherapy bias. We will report two separate 2 (prior expert information: supporting, challenging) x 3 (tag popularity: psychotherapy, balanced, pharmacotherapy) ANOVAs for tag selection on one hand, and blog post selection on the other. With regard to
With regard to the second dependent measure of information search,
In an additional analysis, we exploratively examined whether participants in the challenging
Taken together, we found evidence for a confirmation bias with participants selecting significantly more resources that were consistent with their previously held beliefs that psychotherapy is more effective. Our results also demonstrate, however, that this biased information selection can be significantly reduced. Whereas prior expert information reduced the biased selection of blog posts (but not of tags), tag popularity affected both measures of information search. Being exposed to a tag cloud that contained pharmacotherapy tags as the most popular ones did not only significantly decrease the biased search, but eventually eliminated the confirmation bias in that participants selected as many tags and resources of both treatment types. Hence, challenging tag clouds led to a balanced (ie, unbiased) information search.
Information search bias (pharmacotherapy scores subtracted from psychotherapy scores; positive scores indicate a preference for psychotherapy over pharmacotherapy; negative scores indicate a preference of pharmacotherapy over psychotherapy).
With regard to information evaluation, we hypothesized that prior expert information (H4) that challenges the psychotherapy bias decreases biased evaluation of information, compared to prior expert information, which confirms psychotherapy bias. We also expected popular tags (H5) that challenge psychotherapy bias to reduce biased evaluation of information, compared to balanced tag popularity and even more compared to popular tags that support the bias. In order to analyze both hypotheses, we conducted a 2 (prior expert information: supporting, challenging) x 3 (tag popularity: psychotherapy, balanced, pharmacotherapy) ANOVA, with efficacy ratings as the dependent measure. The main effect of
Popularity of tags challenging psychotherapy bias, in contrast, decreased biased information evaluation as indicated by a significant main effect of
Further explorative analyses supported what can be derived from
In sum, our interventions were differentially successful in reducing the confirmation bias with regard to the evaluation of information. Whereas prior expert information failed to exert a significant influence, tag clouds with tags that challenged the psychotherapy bias not only reduced biased information evaluation, but eventually eliminated any bias. Efficacy ratings in this condition were thus eventually in line with scientific evidence.
Efficacy ratings of blog posts (pharmacotherapy scores subtracted from psychotherapy scores; positive scores indicate a preference for psychotherapy over pharmacotherapy; negative scores indicate a preference of pharmacotherapy over psychotherapy).
Beyond information selection and evaluation, we expected that
We conducted an exploratory qualitative analysis of the reasons for the treatment recommendation. Most of the participants did not provide any reasons but among those who did, the most frequently mentioned aspects regarded etiology or negative consequences of antidepressants. Specifically, 16.7% (29/174) participants argued for psychotherapy because they were convinced that biographical and social causes are crucial for the causation and treatment of depression. Another 10.3% (18/174) participants mentioned side effects, and 6.3% (11/174) reasoned that antidepressants are addictive. Finally, 4.6% (8/174) revealed that they believed that overcoming depression is an act of will or a personal responsibility.
Treatment recommendation after experiment.
This study investigated potential measures to decrease biased beliefs and their influence on information selection and information evaluation. To this end, we made use of laypeople’s (erroneous) convictions that psychotherapy is more effective in treating depression and examined whether this conviction guides online information search. In line with prior findings, participants did believe in the superiority of psychotherapeutic treatment and thus exhibited a psychotherapy bias. When searching for information online about the treatment of depressive disorders, participants showed a general bias towards selecting psychotherapy treatments compared to pharmacotherapy treatments.
We took two measures to reduce biased information processing. First, we exposed participants to expert information explicitly challenging the superiority of psychotherapy, by demonstrating the effectiveness of pharmacotherapy. This manipulation led participants to select fewer blog posts that were related to psychotherapy compared to the presentation of expert information supporting the effectiveness of psychotherapy. It did not affect, however, tag selection, and there was only a trend for it to exert an influence upon subsequent efficacy ratings. Hence, explicit expert information was only partially successful in reducing biased information processing.
Second, we attempted to decrease biased information processing by presenting participants with tag clouds in which the most popular tags referred to pharmacotherapy (vs psychotherapy). Consistent with our hypotheses, participants in the pharmacotherapy condition selected these popular pharmacotherapy tags more frequently and read more of the underlying blog posts. Moreover, treatment efficacy ratings were affected. In contrast to our expectations, however, we did not find any effects on treatment recommendations.
Although both manipulations had an impact upon search behavior and efficacy evaluation, the manipulations did not exert an impact on providing recommendations to other people. The gap between the efficacy ratings and treatment recommendations might be due to other beliefs people have with regard to both therapies, such as side effects [
Previous research on confirmation bias has shown that people’s prior beliefs influence their information search in a way that they seek to confirm their beliefs [
In order to make people more aware of expert information and to overcome their individual biases, it seems to be useful to provide them with tag clouds. If these tag clouds challenge their subjective beliefs, users are motivated to select more popular tags (that are inconsistent with their own beliefs) and to read more information challenging their own views. This leads to a reduced confirmation bias, not just with regard to information search, but also with regard to evaluation.
A “correction” of subjective biases can only be achieved, however, if the information provided is not also biased. Thus, whether the effect that tag clouds have is really positive depends on the quality of tags and resources: does tag popularity really represent the scientific knowledge about a topic? In order to ensure that, it is important that people with high expertise provide the resources and tags. The provision of such expert information could be fostered if experts were encouraged to publish scientific studies in a style suitable for a broad audience, as this is already sufficient to reduce biased attitudes.
In the current study, we carefully balanced the quality of arguments for both types of treatment. We therefore provided information only about the efficacy of treatments, not about other aspects such as side effects, which would be specific for each treatment. For future studies, it may be desirable to test this in more depth by including diagnostic information with respect to relative efficacy of both treatment types (eg, information on treatments that are less effective compared to others or placebo), as well as providing information on side effects or other treatment-specific information.
Second, it must be pointed out that the present sample consisted mainly of university students or persons with a degree in higher education. Some of our participants had a health care related background. Our analyses showed, however, that the pattern of results was identical when these more knowledgeable participants were excluded. Hence, our findings should be valid with regard to laypeople. Nevertheless, future studies should also include participants without a higher education, as well as older persons.
Our major aim in this study was to investigate whether people exhibit a biased online information search behavior that is guided by biased beliefs. We examined the biased perception of laypersons that psychotherapy is more effective than pharmacotherapy, when it comes to the treatment of depression [
CONSORT-EHEALTH checklist V1.6.2 [
analysis of variance
Diagnostic and Statistical Manual of Mental Disorders, 4th edition
International Classification of Diseases
This study was funded by the Knowledge Media Research Center.
None declared.