This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.
The Web has become an important information source for appraising symptoms. We need to understand the role it currently plays in help seeking and symptom evaluation to leverage its potential to support health care delivery.
The aim was to systematically review the literature currently available on Web use for symptom appraisal.
We searched PubMed, EMBASE, PsycINFO, ACM Digital Library, SCOPUS, and Web of Science for any empirical studies that addressed the use of the Web by lay people to evaluate symptoms for physical conditions. Articles were excluded if they did not meet minimum quality criteria. Study findings were synthesized using a thematic approach.
A total of 32 studies were included. Study designs included cross-sectional surveys, qualitative studies, experimental studies, and studies involving website/search engine usage data. Approximately 35% of adults engage in Web use for symptom appraisal, but this proportion varies between 23% and 75% depending on sociodemographic and disease-related factors. Most searches were symptom-based rather than condition-based. Users viewed only the top search results and interacted more with results that mentioned serious conditions. Web use for symptom appraisal appears to impact on the decision to present to health services, communication with health professionals, and anxiety.
Web use for symptom appraisal has the potential to influence the timing of help seeking for symptoms and the communication between patients and health care professionals during consultations. However, studies lack suitable comparison groups as well as follow-up of participants over time to determine whether Web use results in health care utilization and diagnosis. Future research should involve longitudinal follow-up so that we can weigh the benefits of Web use for symptom appraisal (eg, reductions in delays to diagnosis) against the disadvantages (eg, unnecessary anxiety and health care use) and relate these to health care costs.
The Web has become an important resource for lay information about health, with almost three-quarters of the population in developed countries accessing the Web to research health topics [
The way the Web is used by patients who have obtained a specific diagnosis from a health care professional is likely to differ from the way it is used in the absence of a professional diagnosis when appraising symptoms. Postdiagnosis, individuals have specific medical terms they can use as search terms. Most focus their Web search on treatment options, illness management, and prognosis [
Web use for symptom appraisal may have important implications, although it is unclear whether it plays a beneficial or detrimental role in health care delivery. For example, some evidence suggests it could lead to unnecessary anxiety about health and increase use of health service resources [
To leverage the potential for reducing strain on health care resources and promoting earlier diagnosis, we need to understand the current role of the Web in help seeking and symptom evaluation, and the strategies people use to access information, taking differing contexts into account. Because these questions cannot be addressed in a single study, a systematic review is required, involving a thorough and comprehensive search of the literature, critical appraisal of individual studies, and extraction and synthesis of relevant findings.
This systematic review addresses the following five review questions:
What proportion of different populations (eg, general, specific disease, or demographic groups) use the Web to appraise symptoms?
Which symptoms are likely to be researched online?
How is Web use for symptom appraisal conducted (search strategies)?
What are the behavioral consequences of Web use for symptom appraisal?
What are the emotional consequences of Web use for symptom appraisal?
A protocol was developed by the research team based on the review questions and an initial broad search of the available literature, using the Centre for Reviews and Dissemination’s guidance for undertaking reviews in health care [
We included studies that addressed use of the Web to appraise symptoms (ie, to research symptoms and their potential causes). This could include both actual symptoms and symptoms in fictional scenarios. This did not have to be the primary focus of the study; some reference to Web use for symptom appraisal was sufficient. If studies examined health-related Web use in general, they were screened during full-text review and excluded if no specific reference to symptom appraisal was made. Studies that analyzed anonymous logs were included if they examined symptom-related searches. We included only studies that focused on human behavior; studies that evaluated the performance of Web-based tools were excluded.
Studies on Web use for symptom appraisal of any physical health conditions were included. Studies examining mental health/psychiatric conditions were excluded to focus the scope of the review. Studies on Web use by health professionals were excluded. Studies from any country were included, as long as the publication was written in English.
Our initial scoping suggested a scarcity of research in this area, thus we did not limit included studies to any particular design. Nonempirical studies (eg, theoretical papers and literature reviews) were excluded.
Full paper, English-language publications were included, regardless of the original language of the research.
We searched PubMed, EMBASE, PsycINFO, ACM Digital Library, SCOPUS, and Web of Science for relevant publications up to September 30, 2016. To minimize publication bias, grey literature was explored by searching OpenGrey, an open-access database containing more than 700,000 bibliographical references of grey literature. We also searched the British Library Integrated catalog, which contains reports, conference abstracts, and theses. Finally, authors in the field were contacted to inquire about any unpublished material, if two or more of their papers were among the included studies, or if their paper was judged as particularly relevant to the review (eg, if examining Web use for symptom appraisal was the primary focus of the study).
The terms Internet, Web, online, search engine, Google, help seeking, health information seeking, symptom, and diagnosis were entered into the databases (note Google was used as a search term because this is by far the most widely used search engine worldwide [
The study selection process followed the guidelines provided in the PRISMA statement [
From each study, any information regarding use of the Web for symptom appraisal was extracted, as well as details on study design, procedure, population, sampling method, entry and inclusion criteria for study participants, sample size, measures, and details of analysis methods (data extraction sheet in
A quality appraisal of selected articles was conducted based on five criteria designed for reviews incorporating mixed study designs [
The extracted data were synthesized using Thematic Analysis, which has been identified as one of the main approaches used to review and synthesize qualitative and quantitative evidence [
Data familiarization: familiarization with the data was achieved by reading all included studies several times and extracting the relevant information into the data extraction sheets.
A priori grouping: data from the data extraction sheets were grouped according to the review question they pertained to and summarized in a matrix. Studies were entered into the rows of the matrix, whereas study characteristics, limitations, and review questions were entered into the columns. This matrix enabled us to compare the findings of different studies pertaining to the same review question, taking methodological aspects into account (example matrix shown in
Generation of initial codes: the data were initially coded using semantic codes within the NVivo10 environment, using the matrix to compare results across studies.
Searching for themes: once all data extracts were coded, codes were sorted into broader, more conceptual categories to create themes.
Reviewing themes: finally, we reviewed the data extracts the themes related to, to determine whether the created themes satisfactorily captured the raw data.
For quantitative studies reporting proportions without confidence intervals, 95% confidence intervals were computed using the asymptotic (Wald) method based on a normal approximation [
Thirty-two studies were identified as eligible for inclusion in the review. The search process is illustrated in
PRISMA diagram for the study identification process.
Study design and aim of the studies included in the review (N=32).
Author, date | Study design | Aim |
Attfield et al, 2006 [ |
Qualitative interview study; cross-sectional | Explore information seeking of patients before and after consultations, its situational influences, and its impact on patient-provider relationships |
Briet et al, 2014 [ |
Quantitative; cross-sectional analysis of website queries | Explore the nature and content of questions and answers on a health website, and to examine the situations of patients asking questions |
Cartrightet et al, 2011 [ |
Longitudinal log-based study | Analyze the search activity of users researching health information online and identify goals and patterns of search behavior |
Chin, 2009 [ |
Experimental between subjects design: 2×2 (ill–well-defined tasks, younger-older users) | Compare older and younger adults in their performance and search behavior in ill and well-defined tasks |
Chin & Fu, 2010 [ |
Experimental between subjects design: 2×2×2 (older-younger adults, parts-systems interface, parts-system task) | Examine differences between older and younger adults in interacting with different online search tasks and interfaces |
Cooper et al, 2013 [ |
Qualitative study (focus groups) | Explore how women would evaluate symptoms associated with gynecologic cancers |
Cumming et al, 2010 [ |
Cross-sectional Web-based survey study | Evaluate digital storytelling videos (videos of people talking about their own experiences) about help seeking for menopausal symptoms |
De Choudhury et al, 2014 [ |
Cross-sectional survey study (quantitative + qualitative data) + longitudinal log-based study | Research the prevalence of health activities on social media and search engines; characterize health activities on the different platforms and describe how people evaluate information obtained from these |
Fiksdal et al, 2014 [ |
Qualitative focus group study | To gain a deeper understanding of online health-searching behavior to inform future developments of personalizing information searching and content delivery. |
Fox & Duggan, 2013 [ |
Nationwide cross-sectional survey | The Pew Internet & American Life Project is an initiative of the Pew Research Center, a nonprofit “fact tank” that provides information on the issues, attitudes and trends shaping America and the world |
Hay et al, 2008 [ |
Mixed-methods survey and interview study | Understand the extent and reasons for online research prior to first appointments for patients in a rheumatology clinic |
Keselman et al, 2008 [ |
Cross-sectional qualitative interview and Think Aloud study. | Explore users’ information-seeking difficulties by conceptualizing information seeking as a form of hypothesis testing, and to examine the role of users’ competencies in online information seeking |
Lauckner & Hsieh, 2013 [ |
Experimental 2×2 design (position: top-bottom; frequency: high-low) | Does the position and frequency of serious conditions in search results affect perceived severity and susceptibility, and are they related to negative emotional outcomes? Do health literacy and experience with online health seeking moderate these relationships? |
Luger, 2014 [ |
Experimental 2×2 design: two different symptom vignettes (mononucleosis or scarlet fever), either Google or WebMD | Explore older adults’ online health seeking to determine the cognitive and diagnostic processes involved |
Medlock et al, 2015 [ |
Cross-sectional online survey | To determine which information resources seniors who use the Internet use and trust for health information, which sources are preferred, and which sources are used by seniors for different information needs |
Morgan et al, 2014 [ |
Analysis of inquiries posted to a health website | Describe what information people seek from a US website about genetic and rare diseases, and why |
Mueller et al, 2016 [ |
Experimental (randomized trial) | To assess the feasibility of testing a symptom appraisal tool for lung cancer symptoms in an online randomized trial |
Norr et al, 2014 [ |
Experimental within-subjects design | Investigate whether viewing medical websites adversely affects anxiety sensitivity |
North et al, 2011 [ |
Cross-sectional analysis of clicks on a health website and calls to a telephone triage system | Establish what symptoms Internet users tend to look up online and whether telephone triage algorithms could be applied to these |
Perez et al, 2015 [ |
Experimental study with Think Aloud | Describe Internet search processes and identify demographic and personal characteristics associated with use of system 1 (does not include hypothesis testing and evidence gathering) and system 2 (includes hypothesis testing and evidence gathering) processing |
Powell et al, 2011 [ |
Cross-sectional survey with embedded qualitative semistructured interviews | Identify the characteristics and motivations of online health information seekers accessing the NHS Direct website |
Powley et al, 2016 [ |
Cross-sectional survey and observational study | Evaluate whether patients with inflammatory arthritis and inflammatory arthralgia use the Internet for symptom appraisal and to assess the advice given and diagnoses suggested by the NHS and WebMD symptom checkers |
Rice, 2006 [ |
Cross-sectional survey study; secondary analysis of existing dataset | Understand what influences online health seeking, what the reported benefits of online health seeking are, and to identify similarities among online activities |
Teriaky et al, 2015 [ |
Cross-sectional survey | Understand how outpatients awaiting initial gastroenterology consultation seek medical information on the Internet and how wait times affect Internet usage |
Thomson et al, 2012 [ |
Cross-sectional survey study | Explore characteristics of colorectal cancer patients who used the Web to appraise symptoms prior to diagnosis |
White & Horvitz, 2009 [ |
Longitudinal log-based study and cross-sectional survey | (1) Describe escalations that occur when users search for common symptoms and how this escalates to queries about serious conditions, and (2) examine how this persists over several sessions |
White & Horvitz, 2009 [ |
Cross-sectional survey study | Explore how lay individuals use the Web to find explanations for symptoms, what activities they pursue, and what their experiences are |
White & Horvitz, 2010 [ |
Longitudinal log study using logs from Windows Live toolbar | Predict escalations in searches based on characteristics of websites visited |
White & Horvitz, 2010 [ |
Longitudinal log-based study | Establish predictors of when searches for common symptoms lead to health care utilization |
White & Horvitz, 2012 [ |
Longitudinal log-based study | Explore how users search for medical concerns and particularly how these concerns impact on future behavior (eg how this influences focus and attention of future searches) |
White & Horvitz, 2013 [ |
Longitudinal log-based study | (1) Whether snippets in search results are biased toward serious conditions when symptoms are entered into search engines and 2) how these snippets influence user behavior |
Ybarra & Suman, 2006 [ |
National, longitudinal telephone survey | Examine which factors predict whether a Web user is likely to contact a health professional |
Characteristics of the study populations of studies included in the review (N=32).
Author | Study population | Setting | Sample size |
Attfield et al [ |
2 groups of 8 NHS patients: 1 group from a Patient Advice and Liaison Service (PALS) patient panel (43-81 years, mean 64) and one group of MSc students for HCI (25-42 years, mean 31) | UK | 16 |
Briet et al [ |
Users asking hand surgery-related questions from a free online health consultation website | USA (American website; no restriction regarding location of website users) | 131 questions |
Cartright et al [ |
A set of filtered logs from a toolbar deployed by the Windows Live search engine, containing at least 1 symptom | USA (English-language logs, but no restriction regarding location of users) | 2,329,231 actions (=queries issued to a search engine) |
Chin [ |
Younger and older adults from a university community | USA | 69; 41 younger adults (18-35), 28 older adults (60-83) |
Chin & Fu [ |
Younger and older adults from community of a medium-sized city | USA | 46, 23 younger (18-28) and 23 older (60-77) adults |
Cooper et al [ |
Women aged 40-60 years | USA | 132 |
Cumming et al [ |
Visitors of a UK-based menopause website | UK (UK website; no restriction regarding location of website users) | 539 |
De Choudhury et al [ |
Survey: US adults 18-70 years (census representative sampling); Twitter: 15-month sample of Twitter’s Firehose stream, English-language Tweets relating to health; log: data from a major Web search engine | USA (survey with US residents, only English-language log data but not restricted to a certain country) | 210 survey respondents; 125,166,549 tweets; 174,605,024 searches |
Fiksdal et al [ |
Adult, English-speaking members of the Olmsted County, MN community (where Mayo Clinic is located) and Mayo Clinic patients, employees, and family visitors | USA | 19 |
Fox & Duggan [ |
Adults living in the United States | USA | 3014 |
Hay et al [ |
English-speaking US adults (≥17 years) | USA | 120 |
Keselman et al [ |
Lay individuals (convenience sample) | USA | 20 |
Lauckner & Hsieh [ |
Students from an undergraduate communication course at a large Midwestern university | USA | 274 |
Luger [ |
Older US adults, ≥50 years, community resident, without cognitive impairment, who owned a computer | USA | 79 |
Medlock et al [ |
Members of a local senior (Christian) organization | Netherlands | 118 |
Morgan et al [ |
Random sample of English-language inquiries posted by lay people to the question and answer section of the GARD website and inquiries sent via email | USA (American website but no restrictions on locale of users) | 278 inquiries, 68 from 2006 and 210 from 2011 |
Mueller et al [ |
Adults living in UK with undiagnosed symptoms potentially related to lung cancer | UK | 97 |
Norr et al [ |
Undergraduate students from a large university in the Southern United States. | USA | 56 |
North et al [ |
All symptom assessment callers to Ask Mayo Clinic (telephone triage) and all clicks to specific symptoms on the symptom-checker page of MayoClinic.com | USA | 70,370 calls; 2,059,299 clicks |
Perez et al [ |
Young adults aged 21-35 with experience of online health information and reported barriers to accessing health services | USA | 78 |
Powell et al [ |
Users of the NHS Direct website | UK | 792 for survey, 26 for interviews |
Powley et al [ |
Newly presenting patients with either clinically apparent synovitis or a new onset of symptoms consistent with inflammatory arthritis but without clinically apparent synovial swelling attending a secondary care based rheumatology clinic | UK | 34 |
Rice [ |
US adults: respondents from studies conducted within the Pew Internet and American Life project | USA | 13,978 respondents in 2000 who reported health seeking online, 500 of these were telephone interviewed in 2001 |
Teriaky et al [ |
Patients awaiting appointments at a general gastroenterology clinic in London, ON, Canada | Canada | 87 |
Thomson et al [ |
Newly diagnosed colorectal cancer patients (<6 months) | USA | 242 |
White & Horvitz [ |
Log data related to symptom queries (no mention of restriction by locale) from all major Web search engines (eg, Google, Yahoo!, or Live Survey): randomly selected employees of the Microsoft Corporation who had performed at least 1 health-related online search; survey: Microsoft employees | USA (survey with US residents, no restriction mentioned regarding locale for logs) | Logs: 8732 users with symptom-related queries; survey: 515 participants |
White & Horvitz [ |
5000 Microsoft employees were invited via email, from these volunteers were chosen who indicated in a prescreening that they searched the Web for medical information | USA | 515 survey respondents |
White & Horvitz [ |
Logs from windows live browser toolbar, English-speaking USA relating to 6 basic symptoms | USA (log data issued from US locale) | “Many thousands of logs were mined” |
White & Horvitz [ |
Logs from consenting Windows live toolbar users over a 6-month period relating to 3 symptoms: chest pain, muscle twitches, abdominal pain | USA (log data issued from US locale) | 700 queries with symptom to HUI transition; 700 queries with symptoms to no HUI transition |
White & Horvitz [ |
Logs from consenting Windows live toolbar users over a 3-month period | USA (log data issued from US locale) | 169,513 queries |
White & Horvitz [ |
Log data related to symptoms queries generated in English-speaking US locale | USA (log data issued from US locale) | 2070 symptom queries from 714 users |
Ybarra & Suman [ |
Americans living throughout the 50 states and the District of Columbia | USA | Year 1=2104; year 4: 2010, 570 of these were year 1 participants |
Nature of measures and procedures of studies included in the review (N=32).
Author | Nature of measures and procedure |
Attfield et al [ |
Semistructured interviews, eliciting accounts of health information-seeking episodes and how they relate to ongoing health care |
Briet et al [ |
Questions and answers to a health website were categorized and analyzed descriptively |
Cartright et al [ |
Logs were mined and categorized as either evidence-directed, hypothesis-directed with diagnostic intent, or hypothesis-directed with informational intent, according to defined algorithms |
Chin [ |
Participants were randomized to complete either an ill-defined task (find possible causes for a list of symptoms) or well-defined task (find a specific medical term), using a health website; cognitive measures (working memory capacity, processing speed), health literacy measures, medical knowledge measure, search performance for both tasks were measured |
Chin & Fu [ |
Participants were given a symptom vignette and asked to find possible causes. Participants were randomized to complete either a parts task (described symptoms based on body parts) or a systems task (described symptoms by functional systems). Tasks were completed either in the parts interface (categorized symptoms based on body parts) or systems interface (categorized symptoms based on functional body systems). Measures included Patients’ Medical Background Knowledge, Mental Interface Match Index, Broadness (no. of links), link decision time: time spent reading. |
Cooper et al [ |
Discussion in focus groups: which symptoms from a list would be of most concern, why, and what could cause them, what would be their hypothetical response to them, what were actual responses in the past? |
Cumming et al [ |
Participants viewed a storytelling video online and then completed a questionnaire evaluating the effect of the video on feeling informed, planned future help seeking, etc |
De Choudhury et al [ |
In the survey, participants were asked questions about their experiences using Twitter and search engines to share and seek health information; on the log analysis, tweets and logs were categorized as relating to 4 categories: (1) symptoms of major diseases, (2) benign explanations (nonlife-threatening illnesses), (3) serious illnesses, and (4) disabilities; logs were then analyzed descriptively |
Fiksdal et al [ |
Moderators used a semistructured moderator guide to facilitate discussion in focus groups about: (1) participants’ perception and understanding of health care information, (2) the process of information collection on the Internet, (3) understanding and usage of information, and (4) implications of health care information for health and well-being |
Fox & Duggan [ |
People were contacted via telephone for telephone interviews about online health information seeking |
Hay et al [ |
Before their appointment, patients were interviewed about online health information (OHI) seeking, and completed the Wong-Baker-Faces Pain Scale; the consultation was audio-recorded to determine whether OHI was mentioned and then patients completed a satisfaction scale regarding the consultation |
Keselman et al [ |
Participants read a hypothetical scenario describing a relative who experienced symptoms typical of stable angina and then discussed possible causes of symptoms from the symptom vignettes in semistructured interviews; then Think Aloud while they researched symptoms on MedlinePlus |
Lauckner & Hsieh [ |
The study took place online; participants were presented with a symptom vignette and then with a search engine result page manipulated to show serious conditions either at the top or bottom, and low or high frequency of serious conditions; participants then completed several scales: perceptions of severity and susceptibility using the Risk Behavior Diagnosis scale, history of viewing online health information, their health status, how often they experienced each of the 4 symptoms, and their demographic information, health literacy using the Newest Vital Sign (NVS) |
Luger [ |
Participants were presented with 1 of 2 symptom vignettes and asked to diagnose them using Think Aloud, either on Google or WebMD. Measures taken included Think Aloud, self-reported age, gender, ethnicity, education, and income, recent health history, number of hours per week that they used a home computer as well as the number of years that they had owned a home computer, whether or not they had previous experience with the Internet tool to which they were assigned (Google or WebMD’s Symptom Checker). |
Medlock et al [ |
Participants completed an online questionnaire, which included questions about health information resources used; the Autonomy Preference Index was used to assess information needs and preferences for involvement in health decisions |
Morgan et al [ |
A random sample of questions posted to the GARD website were analyzed thematically; collected data included inquiry origin (domestic), type of contact (email and Web-based form), gender, date received at the information center, the specific condition for which they were inquiring, primary language (English), and their reason for inquiry |
Mueller et al [ |
Participants first completed a survey about their symptoms and risk factors. They were then randomized to receive the intervention (personalized, theory-based health webpages), or control conditions. Subsequently, participants completed a questionnaire which assessed demographic details, participants’ self-reported intention to seek help (scale 1-7), behavioral attitudes and beliefs about help seeking. |
Norr et al [ |
Participants first completed the Anxiety Sensitivity Index (ASI), Intolerance of Uncertainty Scale (IUS), and a health anxiety scale (SHAI). Participants were randomized to view either symptom-related websites or general health and wellness control websites. Afterwards, they completed the ASI and SHAI. |
North et al [ |
For the MayoClinic website, click data was collected using Google Analytics; for the telephone triage, all completed calls were counted and put into symptom categories based on the algorithm/guideline used during the call. |
Perez et al [ |
Participants were randomized to one of two symptom scenarios and instructed to search the Internet while using Think Aloud; participants’ Internet searches and think-out-loud vocalizations were digitally recorded using screen capture video-recording software |
Powell et al [ |
Users of the NHS Direct website completed an online questionnaire survey. A subsample of survey respondents participated in in-depth, semistructured, qualitative interviews by telephone or instant messaging/email. |
Powley et al [ |
Patients completed a brief survey on Internet use for symptom appraisal prior to attending clinic; patients were then asked to complete the NHS and WebMD symptom checkers based on their symptoms and their answers and the outcomes were recorded; demographic and disease-related data were obtained from clinic records. |
Rice [ |
Respondents were contacted via telephone for telephone interviews asking about online health seeking. |
Teriaky et al [ |
Patients awaiting gastroenterology consultation were asked to complete a questionnaire consisting of 16 multiple-choice questions to understand patient use of Web resources for medical information. Abstracted information included patient demographics, level of education, reason for referral, preceding investigations, patient resources utilized, websites browsed, information obtained, reasons for seeking information on the Internet, patient self-diagnosis, and lifestyle changes instituted. |
Thomson et al [ |
Semistructured interviews focused on patient sociodemographic and psychological factors, symptom recognition and appraisal, and communication with HCPs, friends, and family. |
White & Horvitz [ |
Analysis of logs: Formulated a list of symptoms and associated benign and serious conditions. Recorded all queries to search engines and clicks on result pages, and identified those that included symptoms as search terms. Escalations: Observed increases in medical severity of search terms within a search session. Nonescalations: Search progresses to benign explanation of the symptom; survey: Microsoft employees were sent a survey with open and closed-ended questions regarding participants’ medical history and online search behavior |
White & Horvitz [ |
Microsoft employees were sent a survey to elicit perceptions of online medical information, experiences in searching for this information, and the influence of the Web on health care concerns and interests. The survey contained “around 70” open and closed questions |
White & Horvitz [ |
Cases were identified where queries for symptoms were followed by a query about a related serious condition. Cases where it led to a benign query or no change were termed nonescalations. Using logistic regression, a model was developed to predict escalation using website features of the previously visited page; website features: structural features, title and URL features, firs-person testimonials, page reliability/credibility, commercial intent |
White & Horvitz [ |
Log analysis: logs containing symptoms as search terms were filtered, and it was determined whether subsequent searches showed health care utilization intent (HUI). Logistic regression was used to predict HUI based on search characteristics; log entries include a user identifier, a timestamp for each page view, and the URL of the page visited; HUI: queries that indicate searching for contact information for medical facilities |
White & Horvitz [ |
Queries were labeled to identify medical and symptoms related queries, and escalations. Subsequently occurring searches were examined. Log entries included a unique user identifier, a timestamp for each page view. Search sessions on Google, Yahoo!, and Bing. Escalation queries were categorized as within-session and between session |
White & Horvitz [ |
Log data relating to symptom queries were filtered. Subsequent behavior on the search engine result page was examined, including hovering, cursor movements, clicks, scrolling, as well as bounding boxes of |
Ybarra & Suman [ |
Respondents were contacted via telephone and completed a telephone survey about online health information seeking and help-seeking behavior (seeking help from a health professional or others) |
As
Symptoms and diagnoses examined in included studies.
Author, date | Were participants symptomatic, asymptomatic, or previously symptomatica? | Type of symptoms examined | Did the study follow up whether Web use was followed by a diagnosis? |
Attfield et al [ |
Previously symptomatic | General (any symptoms) | Not assessed |
Briet et al [ |
Unclear, participants were users asking questions about symptomsb | Hand illness-related symptoms | Not assessed |
Cartright et al [ |
Unclear, participants were users issuing symptom-related queries to a search engineb | Generalc | Not assessed |
Chin [ |
Asymptomatic, participants were presented with a symptom vignette | Symptom vignettes included: pain and stiffness in the body; burning, itching, and sometimes tingling sensation on their body; feeling feverish and chilly after an overseas trip; fatigue, sudden weight gain and difficulty dealing with cold; however, results were not analyzed separately for different symptoms | Not applicabled |
Chin & Fu [ |
Asymptomatic; participants were presented with a symptom vignette | General (participants received 6 different vignettes with different symptoms, not assessed separately) | Not applicabled |
Cooper et al [ |
Asymptomatic; participants were presented with a list of symptoms | Symptoms related to gynecologic cancers | Not applicabled |
Cumming et al [ |
Most symptomatic (448/492), but some asymptomatic (44/492) | Menopausal symptoms | Not assessed |
De Choudhury et al [ |
Unclear, participants were users issuing symptom-related Tweets and queries to a search engineb | General, logs were filtered for references to symptoms using a comprehensive list of symptoms from the Merck medical dictionary | Not assessed |
Fiksdal et al [ |
Previously symptomatic | General (any symptoms) | Not assessed |
Fox & Duggan [ |
Previously symptomatic | General (any symptoms) | Participants were asked whether their diagnosis was confirmed by a health professional; 45% said it was confirmed, 35% did not present, 19% said it was not confirmed/inconclusive |
Hay et al [ |
Symptomatic; participants were newly diagnosed rheumatology patient | Rheumatoid symptoms | Yes, patients’ diagnoses were gathered after the appointment or at follow-up appointment |
Keselman et al [ |
Asymptomatic; participants received a symptom vignette | Symptoms typical of stable angina | Not applicabled |
Lauckner & Hsieh [ |
Asymptomatic; participants received a symptom vignette | Symptom vignettes involved one of four symptoms: headaches, chest pain, muscle twitches, or abdominal pain, but the different symptoms were not analyzed separately | Not applicabled |
Luger [ |
Asymptomatic; participants received a symptom vignette | Symptom vignettes involved either mononucleosis or scarlet fever | Not applicabled |
Medlock et al [ |
Previously symptomatic | General (any symptoms) | Not assessed |
Morgan et al [ |
Unclear, participants were users issuing symptom-related Tweets and queries to a search engineb | Symptoms related to any type of genetic or rare disease | Not assessed |
Mueller et al [ |
87 participants were symptomatic, 10 were asymptomatic but searching on behalf of someone else | Symptoms related to lung cancer | Not assessed |
Norr et al [ |
Asymptomatic; participants viewed a list of symptoms | General (“websites focused on symptoms of medical conditions”) | Not applicabled |
North et al [ |
Unclear, participants were users searching the MayoClinic website or using a telephone triageb | General (any symptoms) | Not assessed |
Perez et al [ |
Asymptomatic; participants received a symptom vignette | One of two clinical symptom scenarios: (1) fever, mild headache, dry cough, and myalgia, suggestive of influenza, and (2) fever, severe headache, and stiff neck, suggestive of meningitis | Not applicabled |
Powell et al [ |
Unclear, participants were users of the NHS websiteb | General (any symptoms) | Not assessed |
Powley et al [ |
Symptomatic; participants were patients attending a secondary care based rheumatology clinic | Either clinically apparent synovitis or a new onset of symptoms consistent with inflammatory arthritis but without clinically apparent synovial swelling | Yes, rheumatological diagnosis was recorded after consultation |
Rice [ |
Previously symptomatic | General (any symptoms) | Not assessed |
Teriaky et al [ |
Symptomatic; participants were patients awaiting gastroenterology appointments | Symptoms related to gastroenterology | Not assessed |
Thomson et al [ |
Symptomatic; participants were colorectal cancer patients | Symptoms related to colorectal cancer | Yes; all participants were diagnosed with colorectal cancer |
White & Horvitz [ |
Logs: Unclear, participants were users issuing symptom-related queries to a search engineb; survey: previously symptomatic | Logs related to 3 common symptoms (headache, muscle twitches, and chest pain) | Not assessed |
White & Horvitz [ |
Previously symptomatic | General (any symptoms) | Not assessed |
White & Horvitz [ |
Unclear, participants were users issuing symptom-related queries to a search engineb | Queries related to any of 6 common symptoms: headache, chest pain, muscle twitches, abdominal pain, nausea, and dizziness | Not assessed |
White & Horvitz [ |
Unclear, participants were users issuing symptom-related queries to a search engineb | Queries related to one of 3 symptoms: chest pain, muscle twitches, and abdominal pain | Not assessed |
White & Horvitz [ |
Unclear, participants were users issuing symptom-related queries to a search engine | Generalc | Not assessed |
White & Horvitz [ |
Unclear, participants were users issuing symptom-related queries to a search engineb | Generalc | Not assessed |
Ybarra & Suman [ |
Previously symptomatic | General (any symptoms) | Not assessed |
a Symptomatic: participants experienced the symptoms at the time of the study; asymptomatic: participants did not have symptoms and were surveyed regarding fictional symptoms; previously symptomatic: participants were surveyed about symptoms they experienced previously.
b Participants were users asking questions about symptoms (could be own symptoms or asking on behalf of someone else).
c Any queries related to a comprehensive list of symptoms from the Merck medical dictionary.
d Patients were not symptomatic.
Quality assessment of the studies is shown in
Four studies, all surveys, reported the proportion of the study sample that engaged in Web use for symptom appraisal (
Percentage of people engaging in Web use for symptom appraisal reported by included studies (n=4).
Reference | Study population | Sample size | Reported Web use for symptom appraisal, % (95% CI) |
Fox & Duggan [ |
Adults living in the US | 3014 | 35% (33%-37%) |
White & Horvitz [ |
US Microsoft employees | 515 | 75% (71%-79%) |
Medlock et al [ |
Members of a senior church organization, Netherlands | 118 | 23% (15%-31%) |
Thomson et al [ |
Colorectal cancer patients, US | 242 | 25% (20%-31%) |
In Fox and Duggan’s [
White and Horvitz’s [
Medlock et al [
While the previous surveys focused on diagnostic searches for any conditions/symptoms, Thomson et al [
To conclude, Fox and Duggan’s [
Six studies examined characteristics of symptoms that were searched for online [
North et al [
In their study on colorectal cancer patients, Thomson et al [
Finally, in Fiksdal et al’s [
In conclusion, it appears Web use for symptom appraisal occurs when symptoms are persistent, have a history of being undiagnosed by health professionals, are potentially embarrassing or stigmatized, and/or when they are perceived as superficial/nonserious.
Three distinct approaches to searching were identified: (1) symptom-based searches, which used symptoms as search terms; (2) condition-based searches, which involved searches for particular conditions, and (3) treatment-based searches, which involved researching treatments for symptoms without prior research on possible causes.
Log data from search engines suggest the majority (65%) of exploratory health-related searches (ie, those aimed at diagnosing a condition) are symptom-based rather than condition-based [
An experimental study that observed people (N=79) as they used Google or a symptom-checker tool to diagnose symptom vignettes reported that most users conduct symptom-based searches because most people began their search by entering symptoms and only 24% began by specifying a condition [
In an experimental study conducted by Perez et al [
Keselman et al [
Overall, it seems most Web use for symptom appraisal searches are symptom-based and both log-based studies, which have high external validity, and experimental studies, which have high internal validity, confirm this finding. No validation was reported for the algorithms used for the log-based studies, however, and experimental and qualitative studies used to observe search behavior have limited generalizability to real-world contexts.
Keselman et al [
Three studies reported on age differences in search behavior [
In another study (N=46), Chin and Fu [
Luger at al [
Thus, there are some indications that older adults perform differently in Web searches for symptom appraisal than younger adults, possibly due to medical knowledge. However, the available studies used small sample sizes, thus inferences to the wider population may not be appropriate.
Several studies examined how users select information from their search results. We identified four subthemes relating to selection of information.
Lauckner and Hsieh [
Thus, the top results returned by search engines will have maximum impact on symptom appraisal, whereas those located below the fold may have little to no effect. Because these findings all relate to laboratory-based studies, however, further investigation in naturalistic settings would be beneficial.
In their study using Think Aloud with 79 adults aged 50 years and older, Luger et al [
In Luger et al’s [
White and Horvitz [
Although we do not know searchers’ intentions or how they used the information found, these findings suggest those researching symptoms online are more likely to engage with websites relating to serious causes.
To summarize, Web use for symptom appraisal typically involves inputting information into a search tool and subsequently narrowing down results returned by the search tool. When inputting information, most users appear to choose search terms based on symptoms rather than hypothesized conditions, but users do not appear to utilize all information available (eg, some symptoms may be omitted, as well as the frequency/duration of symptoms). Furthermore, there is some limited evidence that older adults perform differently in Web searches for symptom appraisal than younger adults, and that this may be due to medical knowledge. Once a selection of results is provided by the search tool, users tend to narrow results down by taking into account the results’ position on the results page, the degree of seriousness of the condition, the credibility of the source, and the extent of overlap between the listed and the experienced symptoms.
In Fox and Duggan’s [
Using logistic regression with a survey sample of more than 2000 Americans aged 12 years and older, Ybarra et al [
Some studies suggest that the mode of presenting information on a website may affect users’ decisions to seek medical advice: in a UK-based qualitative study [
Using log-based search engine data, White and Horvitz [
By observing how patients attending a rheumatology clinic completed the NHS and WebMD symptom-checker tools, Powley et al [
In Powell et al’s [
Fox and Duggan [
Two studies found indications that Web use for symptom appraisal is related to reduced communication with a health professional [
From the preceding findings, we can conclude that Web use for symptom appraisal is used to inform the decision of whether to present to health services and that online self-diagnosers are more likely than other health information seekers to contact a health professional. This can potentially be increased, where appropriate, with novel methods such as “digital storytelling” or theory-based components. Some evidence also suggests that online health information can potentially reduce help seeking by calming users’ fears. It is unclear, however, what proportion of users feel encouraged or discouraged to seek help appropriately (ie, what proportion of users who feel encouraged to seek help actually have a condition warranting medical attention, and what proportion of users who feel discouraged to seek help actually do not need medical attention). Furthermore, it is unclear whether those engaging in Web use for symptom appraisal are more or less likely to seek medical advice than those experiencing the same symptoms without researching online because this comparison was not made in any of the included studies. Web use for symptom appraisal can also play a role in communication with health professionals by influencing how individuals prepare for consultations and prompting discussion of online health information.
In White and Horvitz’s [
Powell et al [
Teriaky et al [
Lauckner et al [
Another experimental study conducted by Norr et al [
Therefore, some evidence suggests there is a relationship between Web use for symptom appraisal and health anxiety. Findings from experimental studies were mixed regarding causal relationships.Surveys and interviews indicate there is a potential for calming effects and decreases in anxiety, and that the proportion who report feeling calmed by Web use for symptom appraisal is higher than those reporting anxiety. It is also possible that those who engage in Web use for symptom appraisal are more anxious about their health generally. It is unclear when anxiety is warranted because participants’ actual diagnoses were not followed up, and comparisons to those who did not research symptoms was lacking.
This is the first systematic review and synthesis of the literature available on Web use for symptom appraisal. Our main findings were:
Approximately 35% of the general population engage in Web use for symptom appraisal, but the proportion can vary considerably (25%-75%) depending on the population under study.
Symptoms tend to be researched online when they are long term, potentially embarrassing/stigmatized, have been presented to health services previously with inconclusive outcomes, and/or when they are perceived as trivial.
Searches tend to be based on symptoms rather than hypothesized conditions; users seem to focus on particular symptoms while disregarding other symptoms and aspects such as frequency and duration.
Once a selection of results is returned by the search tool, people use specific techniques to narrow results down (eg, taking into account the position on the results page or the credibility of the source).
Evidence indicates that online information is used to inform the decision of whether to contact health services and is related to (increased and decreased) anxiety, but the precise impact cannot be discerned due to lack of follow-up and appropriate comparison groups.
Subsequently, we discuss whether Web use for symptom appraisal should be viewed as an asset or a liability in health care delivery based on currently available evidence, and make recommendations for the improvement of online health information.
Criticisms of online self-diagnoses include concern over unnecessary anxiety and health care utilization [
First, it is important to note limitations of approaches used to examine relationships between Web use for symptom appraisal and health anxiety or help-seeking behavior. Cross-sectional surveys cannot show direction of causality. It is possible that using the Web to appraise symptoms causes anxiety, or that anxiety triggers Web use for symptom appraisal, or that a third factor influences both. Furthermore, the surveys that reported on anxiety among online self-diagnosers were biased toward certain demographic [
Log-based studies, which evaluate behavior based on search engine log data, do not allow firm conclusions regarding users’ actual behaviors and motivations. For example, White and Horvitz [
Experimental research shows that users asked to research certain symptoms may report feeling anxious following Web searches [
Using the Web to appraise symptoms may also decrease anxiety in some cases [
There are also indications that Web-based information can help individuals recognize their symptoms as signs of serious conditions [
Finally, it should be noted that worry can also have positive effects on health behaviors [
A limitation we discovered across different methodologies was the lack of follow-up on participants’ help-seeking behavior and diagnoses. Without this information, we cannot discern whether individuals’ self-diagnoses and decisions regarding help-seeking behavior are appropriate or not. We also cannot determine long-term impacts on health care utilization. Furthermore, essential comparison groups are generally lacking. For example, it would be necessary to compare those who research symptoms online with those who do not (rather than surveying only online self-diagnosers), and to compare those who present to health services with those who do not (rather than surveying only patients presenting in clinic) to determine impacts of Web use.
Based on the findings of this review, we suggest changes to health websites, Web apps, and search engines such that they can provide useful information to those researching symptoms.
Our analyses reveal that users tend to search inductively based on symptoms. Search engines and symptom-checker tools need to ensure users are directed to useful information when symptoms are entered. The review also shows that searchers tend to omit dimensions such as duration and frequency of symptoms in their search terms [
Our review also reveals that online health information can impact on the decision to seek help and on communication with health professionals. Health websites and apps need to ensure they provide useful information to support searchers in their decisions and health care interactions. Health websites providing symptom information should, for example, provide clear guidelines on when medical advice should be sought (eg, if a symptom has a certain quality or duration) and how help should be sought (eg, immediately via emergency services or within the next week via primary care).
As the review includes a diversity of study types and methods, a quantitative synthesis or meta-analysis was not possible. However, traditional forms of systematic review that do not make use of all forms of evidence often do not take differing contexts into account, limiting their use to policy makers and practitioners [
In this review, we considered a diversity of symptoms and conditions; when more research in this area becomes available, it would be useful to carry out more focused reviews because the nature of the symptom is likely to influence Web use online [
Finally, it should also be noted that this review did not examine Web use for mental health symptoms. Web use for symptoms related to mental health and its impact on help seeking represent an important field of study and should be assessed in a separate review of the literature.
This systematic review indicates that the Web can disseminate information to those worried about symptoms and can affect their decisions to present to health services. It also suggests Web use for symptom appraisal can impact on how patients prepare for consultations with health care professionals. Thus, we can conclude that Web use for symptom appraisal has the potential to influence the timing of help seeking and the communication between patients and health care professionals during consultations.
At present, limitations of the reviewed studies mean it is not clear when the Web plays a beneficial role in health care delivery and when it is detrimental. Web use for symptom appraisal has been linked to increased as well as decreased anxiety and health care contact. However, the evidence does not show when this is warranted because most studies did not follow up whether participants ultimately sought help following their Web searches and whether they received a diagnosis. Furthermore, comparison groups are lacking to determine the effects of Web use for symptom appraisal.
We need longitudinal research that follows up whether participants seek help and are ultimately diagnosed following Web searches, and compare Web searchers to non-Web searchers. These data can then be used to weigh the benefits of Web use for symptom appraisal (eg, reductions in delays to diagnosis and avoidance of unnecessary health care use) against the disadvantages (eg, unnecessary anxiety and health care use) and relate these to health care costs. Research should focus on real-world samples of people experiencing symptoms and could involve novel methods of tracking behavior, such as analysis of search engine log data and mobile geotracking as used in some of the included studies to follow people over time. These studies have the advantage of high external validity and large sample sizes. However, the algorithms used to analyze these data should first be tested extensively for reliability and validity before further work to evaluate cost effectiveness can meaningfully be conducted. Moreover, further experimental studies would allow a detailed analysis of search behavior. Future research could examine how the different search strategies identified here—symptom-based, condition-based, and treatment-based—relate to cognitive biases and link this to theory.
Example search strategy.
Data extraction sheet.
Quality appraisal of included studies.
NVivo Matrix Screenshot.
Preferred Reporting Items for Systematic Reviews and Meta-Analyses
This project was funded by a Medical Research Council (MRC) Doctoral Training Partnership studentship (Ref: 1354978) and by a President’s Doctoral Scholar Award from the University of Manchester.
None declared.
JM drafted the review protocol, and all coauthors critically revised the protocol. JM, AD, and JV independently completed the study selection procedure and critically discussed all studies for inclusion in the review. JM extracted and analyzed the data and wrote the first draft of the manuscript. CT, SH, and CJ contributed to discussion of the content and analysis method. All coauthors reviewed and edited the manuscript before submission.