This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.
Internet sites typically contain visual design elements that are unrelated to the quality of the health information presented but that could influence credibility judgments and responses to health advice. To assess the effects of such design elements, or credibility cues, experimentally, we exposed women with different levels of weekly alcohol consumption to a website containing high quality but unpalatable information about a related health risk (breast cancer). The information was presented alongside either positive or negative credibility cues unrelated to information content.
We explored four research questions: (1) Did the cues influence how the women engaged with the site? (2) Did they influence how the women responded cognitively and emotionally? (3) Did they influence whether the women subsequently acted on the advice? (4) Did the impact of the cues vary with how much alcohol the women reported drinking?
A total of 85 women were randomly assigned to view one of two versions of a website containing the same high-quality content but different cues. One version had positive credibility cues (trustmarks), the other had negative ones (adverts, pharmaceutical sponsorship, and a donation button). Objective measures included visual attention (using eye-tracking equipment), time studying the material, and recall. Subjective measures included cognitive and affective responses and intention to change. Measures of subsequent behavior were taken 1 week later.
First, the cues did not affect how long the women spent on the site or how long they spent reading the text. However, women in the negative cues condition spent more time looking at a donation button than those in the positive cues condition spent looking at a TRUSTe seal (β = −.43,
Content-irrelevant images and logos can influence the behavioral response to quality health-risk information. These effects may be subtle, changing with time.
The Internet has become an important source of health information and advice, with between 40% and 80% of those with Internet access in the United States and Europe using it for health care purposes [
Briggs and colleagues addressed this issue in a series of studies of trust in eHealth that led to the development of a staged model of trust and a set of guidelines [
Sillence et al [
Health consumers, then, do not always choose the best-quality health sites or follow the best advice. Indeed, they can show a marked reluctance to trust advice they perceive to be inconsistent with their important prior beliefs, even displaying a “defensive” response to such information [
This is particularly problematic in the eHealth context, where consumers can often find conflicting advice and are able to navigate to websites that tell them what they would prefer to hear. In one study, for example, a population struggling to modify their drinking behavior sought pro-drinking rather than anti-drinking material when left free to choose via the Internet [
We present an experimental study in which women with different levels of weekly alcohol consumption were exposed to high-quality, uncongenial, Internet-based information concerning a related health risk (breast cancer). This unpalatable information was presented alongside positive or negative trust cues unrelated to the information content. We explored four research questions: First, do the cues influence direct engagement with the material on the site? We examined how long the women spent on the site, their pattern of eye movements when examining the site’s pages, and their ability to recall site content. Second, do the cues influence how they respond cognitively and affectively to the site? We examined their subsequent message acceptance, emotional and risk perceptions, and their intentions to adopt the recommended behavior. Third, do the cues influence the extent to which they subsequently act on the advice given? We examined reports of behavior at follow-up 1 week after exposure to the site. Finally, does the impact of the cues vary with how much alcohol the women typically drink? We tested whether baseline alcohol consumption moderated the effects of condition on any of the above outcome measures. That is, we tested whether any cue effects were most pronounced among those who drank most (i.e., those who might display the biggest defense motivation) [
To test these research questions we developed a two-page website containing information about the link between alcohol consumption and the risk of breast cancer. All material was based on a recent definitive study showing such a link [
The sample consisted of 85 women, all regular Internet users and mainly students of psychology (mean age = 22.9 years, SD = 6.5 years). Of these, over a quarter (n = 22) reported drinking more than the UK Government’s recommended limit of 14 units of alcohol per week. (In the UK, a unit of alcohol is 8 g, which is approximately half a pint of beer, a standard measure of spirits, or a glass of wine.) All participants were paid £5.
The study had a between-participants experimental design, with one independent variable: participants were randomly assigned to the positive (n = 42) or negative (n = 43) cues conditions. The experiment had a prospective component, with a follow-up after 1 week.
Along with baseline alcohol consumption, we also measured age, sex, Internet use, attitudes toward alcohol [
The health message was on two Web pages and contained information about the link between alcohol consumption and the risk of breast cancer. The text of the Web pages was closely based on that used by Harris and Napper [
Two different designs of the Web pages were created (see
The SensoMotoric Instruments iViewX Eye Tracking System (SensoMotoric Instruments GmbH (SMI), Teltow/Berlin, Germany) was used to record the eye movements of participants as they were viewing the Web pages on a desktop PC (1.2 GHz Pentium 4 with 256 Mb memory and a 17 inch LCD monitor). In accordance with the eye-tracking analysis software, the different design features of interest on each Web page were defined as object areas; 15 such objects were identified (
After viewing the message, participants completed the post-manipulation questionnaire. This opened with the manipulation check items, comprising a measure of mood [
information access: eight items (e.g., “The site told me most of what I needed to know”)
personalization: eight items (e.g., “It felt like the advice was tailored to me personally”)
credibility through impartiality: four items (e.g., “The advice appeared to be impartial and independent”)
credibility through design: four items (e.g., “The site had a professional design”)
Responses were given on a 5-point scale from “strongly disagree” to “strongly agree.”
The subjective measures comprised the following:
cognitive responses indicating acceptance of the message: three items (e.g., “I believe that drinking alcohol increases the chances of women developing breast cancer) rated on a 7-point scale from “strongly disagree” to “strongly agree”
negative affective responses to the message: six items (e.g., “The material on the website made me feel…”) rated on a 7-point scale from “not at all anxious” to “extremely anxious”
perceived vulnerability: three items (e.g., “How likely do you think you will be to get breast cancer as a result of your current level of alcohol consumption) rated on a 10-point scale from “impossible” to “extremely likely”
intentions to cut down on alcohol: two items (e.g., “I intend to reduce my alcohol consumption in the next 7 days by at least 2 units”) rated on a 7-point scale from “definitely do not intend to” to “definitely intend to”
Items were taken from previous studies (e.g., [
The questionnaire closed with a test of recall of message content (three items; e.g., “According to the report how many deaths would be avoided annually in Britain if women stopped drinking?”). Each correct answer scored 1.
After 1 week, participants received a brief follow-up questionnaire by email, measuring reported alcohol consumption over the previous 7 days and containing the measures of intentions, vulnerability, cognitive response (one item), and affective response (three items) from the post-manipulation questionnaire (Cronbach alpha from .70 to .97). The questionnaire was designed to be brief in order to maximize response rate.
Participants were tested individually. Upon arriving at the laboratory, they were told that the study involved an evaluation of health information on the Internet. After completing the pre-measures, they were asked to sit comfortably in front of the eye-tracking monitor while the researcher calibrated the system. Each participant was told that she was about to see a website consisting of two pages. They were instructed to imagine that they had found the website by following a link from a search engine and to look at and visually examine the website as they would any site they had found in this way. They were told that they could spend as little or as much time as they wanted on the website. This phase of the study ended when the participant clicked on a link at the bottom of the second Web page. They then completed the post-manipulation measures. Participants were sent the follow-up measures by email approximately 7 days later.
One-way analysis of variance (ANOVA) with randomized condition as the between-participants independent variable or two-step hierarchical regression analyses were used to test for differences in pre-manipulation measures or other variables between the groups. An alpha level of
One-way analysis of variance (ANOVA) with randomized condition as the between-participants independent variable revealed no significant differences between the groups (
Consistent with the intended manipulation, participants in the positive cues condition trusted the site more on the two credibility trust factors, seeing the site as being higher in credibility through impartiality (F1,82 = 4.74,
Mean pre-manipulation measures by condition groupa
Negative Cues (n= 43) |
Positive Cues (n= 42) |
|
Baseline alcohol (units)b | 10.03 (9.10) | 10.95 (9.80) |
Attitudes toward alcoholc | 4.23 (0.95) | 4.25 (0.92) |
Self-esteem | 19.33 (3.73) | 20.05 (3.87) |
Dispositional optimism | 14.21 (4.60) | 14.56 (4.74) |
Breast cancer risk (Harvard risk calculator) | 2.56 (0.85) | 2.63 (0.80) |
Breast cancer risk (Halls risk calculator) | 16.72 (3.36) | 18.02 (4.72) |
aHigher scores indicate more alcohol consumption, favorable attitudes, and greater self-esteem, optimism, and risk.
bAlcohol measured in UK units (one unit is 8 g).
cAttitudes were measured on a 7-point bipolar scale.
Unless stated otherwise, the remaining data were analyzed using two-step hierarchical regression analyses. This allowed us to analyze the effects of baseline alcohol consumption as a continuous variable and to assess the effects of the predictors in combination as well as individually. Where an individual predictor was significant, we report it (β) below; otherwise, in the interests of space, we report only the statistics for the predictors in combination, both in terms of significance and effect size (
There were no significant overall effects of the predictors on the amount of time participants spent on the site (F3,84 < 1,
Percentage of time participants spent visually examining objects on the website, by conditiona
Objectb | Negative Cues (n= 43) |
Positive Cues (n= 42) |
1. Donation/TRUSTe seal | 5.03 (4.33) | 1.80 (2.29) |
2. Logo | 5.92 (4.07) | 12.82 (9.39) |
3. Menu 1 (“You are here”) | 5.00 (4.13) | 9.80 (7.14) |
4. Menu 2 (“Health issues”) | 2.75 (2.86) | 3.42 (3.31) |
5. Advert/HON code | 1.44 (1.80) | 0.95 (1.68) |
6. Page last updated | 0.00 (0.00) | 0.18 (0.50) |
7. Sponsor – Pharmaceutical/NHS | 0.25 (0.91) | 0.34 (1.34) |
8. Photo | 1.50 (1.75) | 1.39 (1.62) |
9. Alcohol and breast cancer subtitle | 1.50 (1.82) | 1.45 (1.55) |
10. Next page link | 0.08 (0.28) | 0.39 (1.67) |
11. Text paragraph 1 | 6.31 (4.44) | 6.32 (4.80) |
12. Text paragraph 2 | 10.47 (7.13) | 7.47 (7.05) |
13. Text paragraph 3 | 3.40 (3.83) | 2.82 (4.14) |
14. Application windowc | 57.25 (28.27) | 62.03 (26.00) |
15. Entire screend | 60.58 (28.15) | 65.16 (25.80) |
aStandard deviations are given in parentheses.
b Objects with different cues (i.e., Advert/HON code and Pharmaceutical/NHS sponsor) are negative cue/positive cue.
c Application window refers to Web browser.
d Entire screen refers to everything visible on the monitor (i.e., application, borders, and task bar).
Condition did not affect any of the measures. No interaction was significant. Level of baseline alcohol consumption did not affect cognitive responses to the material, but higher levels of alcohol were associated with more negative affect (β = .48,
Other dependent variables, by conditiona
Negative Cues (n= 43) |
Positive Cues (n= 42) |
Total (n= 85) | |
|
|||
Trustfactor 1 (access) | 30.05 (4.60) | 31.00 (3.56) | 30.51 (4.13) |
Trustfactor 2 (personalization) | 16.63 (3.70) | 16.88 (2.90) | 16.75 (3.31) |
Trustfactor 3 (impartiality) | 12.33 (2.10) | 13.51 (2.86) | 12.90 (2.55) |
Trustfactor 4 (design) | 11.56 (2.36) | 12.93 (3.24) | 12.23 (2.89) |
Positive mood | 2.90 (1.03) | 2.47 (1.05) | 2.71 (1.05) |
Negative mood | 1.00 (1.41) | 1.09 (1.53) | 1.04 (1.46) |
|
|||
Recallb | 1.07 (0.93) | 0.66 (0.85) | 0.87 (0.91) |
Intentions to cut down alcohol | 2.60 (1.64) | 2.88 (1.81) | 2.79 (1.73) |
Cognitive responses | 4.29 (1.12) | 3.90 (1.12) | 4.09 (1.14) |
Negative affective response | 3.54 (1.34) | 3.58 (1.66) | 3.56 (1.50) |
Perceived vulnerability | 2.31 (1.09) | 2.44 (1.16) | 2.38 (1.12) |
Baseline alcohol consumptionc | 10.03 (9.10) | 10.95 (9.80) | 10.48 (9.41) |
|
(n= 36) | (n = 39) | (n = 75) |
Alcohol consumptionc | 10.63 (9.48) | 9.68 (9.36) | 10.13 (9.37) |
Belief in link | 4.53 (1.60) | 4.64 (1.31) | 4.59 (1.44) |
Intentions to cut down alcohol | 2.60 (1.64) | 2.98 (1.81) | 2.79 (1.73) |
Negative affective responses | 3.39 (1.90) | 3.72 (1.96) | 3.56 (1.93) |
Perceived vulnerability | 2.18 (0.87) | 2.34 (1.15) | 2.26 (1.02) |
aHigher scores indicate more trust, more positive/negative mood, better recall, stronger intentions, more acceptance of the message, more negative affect, higher perceived vulnerability, and more alcohol consumption.
bMaximum possible recall is 3.
cAlcohol measured in UK units (one unit is 8 g).
A total of 75/85 (88%) participants responded to the follow-up questionnaire. These women did not differ significantly from nonresponders on any of the randomization or manipulation check measures taken at post-manipulation (maximum F1,82 = 1.1 [trust factor 3],
As for the post-manipulation, higher levels of baseline alcohol in the follow-up were associated with more negative affect (β = .52,
Simple slopes for the interaction between condition and baseline alcohol consumption on alcohol consumption at 1 week follow-up; simple slopes have been calculated at mean (moderate consumption), +1 standard deviation (higher consumption), and −1 standard deviation (lower consumption) levels of baseline consumption
Did the cues influence engagement with the material on the site? We examined how long the women spent on the site, their pattern of eye movements when examining it, and their recall of site content and found that, whereas the cues did not affect the overall time the women spent on the site or how long they spent reading the text, they did influence where the women directed their gaze and how much they subsequently recalled. Participants paid more attention to certain negative cues, such as the donation button, than to positive cues in the equivalent location, and those in the negative cues condition were subsequently more accurate in their recall of information, suggesting that they paid closer attention to the site content.
Did the cues influence how the participants responded cognitively and affectively? No. We found no effects of the cues on any of the explicit ratings of cognition, affect, vulnerability, or intentions—at least not initially (see below). However, those who drank more reported more negative affect and vulnerability and expressed stronger intentions to cut down on their drinking, effects that persisted at follow-up. This suggests that the material had been persuasive, even among the heavier drinkers. Interestingly, these effects occurred even though drinking level had not influenced how the women examined the site visually or their recall of its contents.
Did the cues influence the extent to which the women subsequently acted on the advice given? Yes, the important influence of credibility cues was seen most clearly at follow-up, where the positive cues promoted greater alcohol reduction than did the negative cues among those who had previously reported heavier drinking. Thus, the answer to our final question— Does the impact of the cues vary with how much alcohol the women typically drank?—is also yes. Indeed, the cues now interacted with baseline drinking level to change the women’s belief in the evidence: specifically, women who had reported drinking the most expressed less belief in the evidence at follow-up, especially the women in the positive cues condition.
Consequently, the results of this study indicate that seemingly superficial design elements of a website can influence responses to health-risk information. As predicted, cues known to be positively or negatively associated with credibility affected engagement with the site and influenced subsequent health behavior and cognition.
We interpret these data as being consistent with findings in the broader literature on defensive responding to threatening information [
The findings thus present an intriguing picture of a group of participants developing their response to a website containing credible, but unwelcome, health-related information. The eye-tracking data also contribute to our understanding of how people allocate their attention to features of websites. For instance, those in the positive cues condition spent relatively little time looking at the HON code or the TRUSTe seal relative to other elements of the site, such as the logo (see
We have shown here (to our knowledge for the first time) that seemingly superficial credibility cues embedded in health information and advice can continue to affect responding over time. However, we acknowledge that our study has several limitations. We recruited a mainly student sample to participate in a somewhat artificial environment, under experimental conditions. We cannot rule out the possibility that some participants may have felt themselves “too young” to be at risk of breast cancer or felt obliged to accept the health message contained in the website. We tried to minimize this by explicitly instructing participants to take their time and to explore the site visually as if they had discovered it themselves through a search engine. We also ensured that the health message was aimed at young women in particular. We recognize, however, that the experimental conditions also meant that participants were forced to look at the website and therefore could not choose to “select out” sites as perhaps they would in more natural settings.
Despite these limitations, these findings are potentially very significant since at-risk groups are typically the hardest to persuade [
More studies of this kind are needed to explore both the immediate and longer term effects of design cues on health cognition and behavior. The message in this study was strong and persuasive and not easily rejected. It may be that the effects of negative design cues such as advertising are more immediate in evaluations of health risk messages that are weaker. Similarly, future research needs to explore what happens when sites containing credible design cues present incorrect information but noncredible sites present correct information. Longer-term follow-ups are also required, along with studies focusing on other health risk messages and populations.
Written information, often in the form of patient leaflets, has usually been seen as an adjunct to verbal messages provided by the medical profession [
The research reported here was supported by ESRC E-Society grant number RES-335-25-0019. We wish to thank Nick Heather for helpful comments on a previous draft of this paper.
None declared.
Screenshot of the positive cues website
Screenshot of the negative cues website