Maintenance Notice

Due to necessary scheduled maintenance, the JMIR Publications website will be unavailable from Wednesday, July 01, 2020 at 8:00 PM to 10:00 PM EST. We apologize in advance for any inconvenience this may cause you.

Who will be affected?


Citing this Article

Right click to copy or hit: ctrl+c (cmd+c on mac)

Published on 03.05.19 in Vol 21, No 5 (2019): May

This paper is in the following e-collection/theme issue:

    Original Paper

    Internet Searches for Medical Symptoms Before Seeking Information on 12-Step Addiction Treatment Programs: A Web-Search Log Analysis

    1Teachers College, Columbia University, New York, NY, United States

    2Social Computing Department, Qatar Computing Research Institute, Hamad Bin Khalifa University, Doha, Qatar

    3Microsoft Research, Redmond, WA, United States

    4Microsoft Research, Herzeliya, Israel

    5Faculty of Industrial Engineering and Management, Technion - Israel Institute of Technology, Haifa, Israel

    *all authors contributed equally

    Corresponding Author:

    George Nitzburg, PhD

    Teachers College

    Columbia University

    525 West 120th Street

    New York, NY, 10027

    United States

    Phone: 1 212 678 3267



    Background: Brief intervention is a critical method for identifying patients with problematic substance use in primary care settings and for motivating them to consider treatment options. However, despite considerable evidence of delay discounting in patients with substance use disorders, most brief advice by physicians focuses on the long-term negative medical consequences, which may not be the best way to motivate patients to seek treatment information.

    Objective: Identification of the specific symptoms that most motivate individuals to seek treatment information may offer insights for further improving brief interventions. To this end, we used anonymized internet search engine data to investigate which medical conditions and symptoms preceded searches for 12-step meeting locators and general 12-step information.

    Methods: We extracted all queries made by people in the United States on the Bing search engine from November 2016 to July 2017. These queries were filtered for those who mentioned seeking Alcoholics Anonymous (AA) or Narcotics Anonymous (NA); in addition, queries that contained a medical symptom or condition or a synonym thereof were analyzed. We identified medical symptoms and conditions that predicted searches for seeking treatment at different time lags. Specifically, symptom queries were first determined to be significantly predictive of subsequent 12-step queries if the probability of querying a medical symptom by those who later sought information about the 12-step program exceeded the probability of that same query being made by a comparison group of all other Bing users in the United States. Second, we examined symptom queries preceding queries on the 12-step program at time lags of 0-7 days, 7-14 days, and 14-30 days, where the probability of asking about a medical symptom was greater in the 30-day time window preceding 12-step program information-seeking as compared to all previous times that the symptom was queried.

    Results: In our sample of 11,784 persons, we found 10 medical symptoms that predicted AA information seeking and 9 symptoms that predicted NA information seeking. Of these symptoms, a substantial number could be categorized as nonsevere in nature. Moreover, when medical symptom persistence was examined across a 1-month time period, a substantial number of nonsevere, yet persistent, symptoms were identified.

    Conclusions: Our results suggest that many common or nonsevere medical symptoms and conditions motivate subsequent interest in AA and NA programs. In addition to highlighting severe long-term consequences, brief interventions could be restructured to highlight how increasing substance misuse can worsen discomfort from common medical symptoms in the short term, as well as how these worsening symptoms could exacerbate social embarrassment or decrease physical attractiveness.

    J Med Internet Res 2019;21(5):e10946





    Alcohol use disorder (AUD) exerts a heavy burden on psychiatric and medical facilities, which has led to early identification and intervention efforts with problem drinkers who are engaging in regular risky drinking patterns but do not yet have physical dependence on alcohol or an immediate medical need for abstinence. For many problem drinkers, the first point of contact often occurs at routine office visits to primary care physicians, where roughly 10%-20% of primary care patients are found to be drinking alcohol at hazardous levels [1-3]. Risky drinking patterns (such as binge or regular heavy drinking) have been linked with numerous adverse medical consequences [4-9] including numerous cancers [5], heart disease and stroke [10-13], pneumonia [14], tuberculosis [15], epilepsy [16], diabetes [17], pancreatitis [18-21], and liver disease [22]. Although the abovementioned conditions are among the more severe long-term medical consequences of alcohol, heavy alcohol consumption has been implicated in the onset or exacerbation of over 100 diseases and conditions [23]. Since primary care office visits can often be prompted by the need to address medical symptoms partially or can fully arise from heavy drinking patterns, such visits offer an opportunity to motivate problem drinkers toward alcohol-reduction treatment options.

    To help reduce drinking among their patients, from hazardous levels down to low-risk levels, physicians have made efforts to integrate minimal treatment protocols into their routine care (via face-to-face or electronic delivery), known as brief intervention (BI) [24,25]. The largest venue for BI, which follows after The Substance Abuse and Mental Health Services Administration’s largest-of-its-kind federally funded Screening, Brief Intervention, and Referral to Treatment (SBIRT) initiative, is primary care clinics and emergency rooms. SBIRT has been a vital source of early identification of alcohol and drug use disorders, which has so far screened and offered BI or treatment referral in 29 US states [26]. SBIRT programs have demonstrated effectiveness with drug and alcohol reductions at 6 months postintervention [27], along with clinically and statistically significant improvements in physical and mental health as well as social, legal, housing, and employment outcomes [28]. In addition to expanding the scale of the existing SBIRT program, identifying ways to further maximize BI’s effectiveness is also of critical importance to SBIRT.

    One largely unexplored avenue for enhancing BI’s personalized feedback for more severe cases is to evaluate which negative medical consequences most contribute to information seeking for abstinence-based treatment. Twelve-step programs remain the most utilized form of substance use disorder (SUD) treatment [29,30]. Specifically, BI’s health risk information could be improved by understanding which of the >100 alcohol-implicated diseases and conditions are most likely to motivate severe cases to consider 12-step treatment options. Many potential improvements may be possible regarding this health risk information.

    In particular, many of the medical consequences emphasized in BI occur over the long term (eg, cancer). However, meta-analytic evidence has shown that individuals who engage in addictive behaviors make more impulsive decisions, have greater difficulty delaying gratification for greater rewards, and tend to discount delayed rewards or other reinforcers more quickly than healthy controls [31,32]. Thus, the actual and perceived speed of reward or reinforcer delivery is particularly critical to motivation in patients with AUD, and as a result, they may be more motivated by less chronic but more immediately discomforting and socially embarrassing symptoms such as gastrointestinal distress. Thus, BI’s health risk information could potentially be improved by focusing on an increased likelihood of experiencing specific sets of immediately discomforting and socially embarrassing symptoms rather than a risk of diagnosis with various medical diseases, some of which occur in the long term (eg, cancer). Reorganizing BI’s health-risk information around the medical symptoms that can be empirically shown to motivate treatment seeking holds the potential to increase BI’s efficacy for the more severe cases often seen in primary care settings.

    Prior Work

    Analysis of large-scale anonymized Web-search log data offers a promising new method for identifying medical symptoms that are most predictive of subsequent seeking of 12-step treatments. Recent analyses of subtle and temporal changes in medical symptom Web-search queries have been able to predict later medical events, including influenza [33], pancreatic and breast cancers [33,34], and adverse side effects of medications [35]. Such analyses can be readily adapted from the prediction of dichotomous health events (eg, search log evidence of having cancer versus not having cancer) to the prediction of dichotomous behavioral health care choices (eg, search log evidence of interest in 12-step treatment versus no such treatment). Although many researchers investigating online patterns in psychiatric disorders and SUD have opted to examine social media activity rather than search engine queries [36-48], Web-search log data analysis is particularly well suited to the study of alcohol-related medical symptoms. Past work has shown that medical health information seeking most often takes place using search engines, especially when medical symptoms may be perceived as potentially connected to stigmatized conditions (eg, symptoms perceived to be the possible negative medical consequences of addiction) [49]. It is most likely due to the associated stigma that regional variation in the interest in the topics of alcohol and Alcoholics Anonymous (AA) on Facebook show inconsistent patterns with respect to regional variation in the rates of alcohol abuse [50]. Due to its anonymous nature, aggregate and anonymous Web-search volume from Google Trends has also been used to study the link between macroeconomic conditions and problem drinking and has shown that a 5% rise in unemployment is followed by an approximate 15% increase in alcoholism-related searches in the next 12 months [51]. In addition, using data from Google Trends, other researchers have assessed the relationship between different legal statuses of marijuana and the level of search interests for “dabbing,” the vaping of high-potency marijuana concentrates [52].

    At the individual level, anonymized Web-search log data offer an ecologically valid and representative sampling of individuals who are likely to show up at primary care clinics and disclose their alcohol-related medical symptoms to a physician when prompted. A recent systematic review found that individuals experiencing acute or conspicuous medical symptoms searched for online health information prior to seeking real-world medical care. These individuals were also subsequently more persuaded to see a physician than those who did not do a Web search, endorsing less embarrassment and concern about bothering their physician with a trivial complaint [53]. Search log data analysis is also well suited to reorganizing BI’s brief advice by a physician, regarding the medical consequences of alcohol misuse, as the same type of analysis has already helped inform how online cancer information may be reorganized to best address the distinct needs of patients and their caregivers when patients are at different levels of disease progression [54]. Taken together, past work suggests that Web-search log data analysis is the best-suited technique to provide unique insights into which negative medical consequences of alcohol misuse are most persuasive for motivating interest in the 12-step treatment.

    Goal of the Study

    The present study used anonymous internet search log data to better understand how seeking medical symptom information contributed to seeking AA and narcotics anonymous (NA) treatment information, including searches for treatment locators. We examined anonymous Microsoft Bing search data to identify individuals that graduated from using Bing search to investigate their medical complaints (which may be potentially alcohol related) to using Bing search to seeking AA or NA information or finding AA or NA meetings. We then examined which of these medical symptom searches most increased the likelihood of conducting either of these two types of Bing searches. Such data can help inform BI-personalized feedback to enhance acceptance of SBIRT’s treatment referrals for patients spanning the broader range of alcohol and drug use severity levels often seen in primary care physicians’ offices.

    Of note, we chose to include NA-related queries in our primarily AUD-focused study since alcohol and drug use disorders are highly comorbid, AA meetings are far more ubiquitous than NA meetings, people with drug use disorders are not typically turned away from AA meetings after revealing their drug of choice is not alcohol, these two types of 12-step programs share many characteristics, and both meeting types can often have mixed attendance [55]. Given such comorbidity and cross-attendance, our online search data analysis was unable to reasonably estimate users’ diagnostic groups, and we instead opted to separately analyze all the various combinations and permutations of AA- and NA-related queries to gauge the convergence/divergence of symptom profiles that preceded them.


    We proposed the following hypotheses:

    1. The medical symptom queries motivating subsequent seeking of information on 12-step programs would be nonsevere but immediately discomforting and socially embarrassing.
    2. Symptoms or conditions considered severe or medically dangerous would not require persistence over time to induce queries for the 12-step programs, with individuals’ taking near-immediate action to seek out information on the 12-step programs soon after experiencing these symptoms.
    3. Symptoms or conditions considered common or nonsevere would require persistence over time to induce queries about the 12-step programs, with individuals only taking action to seek out information on the 12-step programs after a period of enduring discomfort from these symptoms.



    We extracted all queries made by people in the United Stated on the Bing search engine from November 2016 to July 2017 (inclusive). Each query contained an anonymous user-identifier tracked via a browser cookie, the time and date of the query, the text of the query, and the Web address of pages clicked by users in response to the answers shown to them. Although Bing’s US market share is currently estimated at ~24% [56], past work has found it to be representative of the US population [57,58] and has shown that Bing is a viable data source even in the absence of other search engines. This study was approved by the Institutional Review Board of the Technion, Israel Institute of Technology.


    Identifying Queries of People Seeking Information on Alcoholics Anonymous and Narcotics Anonymous

    We then found all queries made by 50 or more people, which resulted in a click to one or more of the 193 pages. There were a total of 100 such queries (eg, “alcoholic anonymous meeting directory”).

    Identifying the Target Population

    Based on the queries identified, we defined two target groups (ie, two 12-step information-seeking samples). The first larger overall sample comprised all 11,784 people who made an AA- or NA-seeking query during July 2017 and who did not make such queries in any of the preceding months (November 2016 to June 2017). The second sample, a subset of the first, included only those people who specifically queried for AA or NA meeting locators by seeking queries that also contained the term “meeting.” A total of 3820 people were included in this subsample.

    Our overall sample consisted of a total of 10,522 people who sought AA information, 1022 people who sought NA information, and 240 people who sought both AA and NA information. Our subsample (whose first 12-step–related search was for AA or NA meeting information) consisted of 2324 people who sought information specifically on AA meetings, 501 people who sought information specifically on NA meetings, and 115 people who sought information on both types of meetings.

    Of note, we did not require a medical symptom query to have been conducted in order for the query to be included in our comparison group. In other words, our overall sample also included people who searched for information on the alcohol or drug 12-step program without ever having queried a medical complaint.

    Identifying Symptoms Correlated With Seeking Treatment Information

    Queries were deemed to contain medical symptoms if their text included a symptom or a synonym thereof, as provided in the list of 195 symptoms and their synonyms in the study by Yom-Tov and Gabrilovich [35]. Colloquial symptoms were mapped to their medical symptom wording, which is why medical terminology descriptors appear in our data (eg, a search query for “sweating” would be mapped onto the more medical descriptor “diaphoresis”). This list included medical symptoms that were indicative of both medical and psychological distress (eg, fever and cough as well as anxiety and depression). The list also included both symptoms that would be immediately discomforting as well as distant or long-term diagnoses (eg, gastrointestinal distress versus cancer).

    We compared the symptoms prior to seeking information on the 12-step programs in two ways: First, we compared the entire population of Bing users in the United States. Second, we performed a within-group comparison, taking into account the temporal sequence of searches. When examining these temporal sequences, we categorized “symptoms with persistence in time” as symptoms that were repeatedly queried across any two of our three measurement points (measurement points at 0-7, 7-14, and 14-30 days prior to a 12-step–related search); symptoms were defined as persistent when they were queried during both the 0-7 day and 14-30 day time lags, during both the 7-14 day and 14-30 day time lags, or during both the 0-7 day and 14-30 day time lags. A full schematic of procedures for data collection and filtering as well as the between-group comparison and the time-lagged within-group analysis of medical symptom queries for identified users are provided in Figure 1.

    For the first analysis, we calculated the probability that users in the treatment population will ask about each symptom in their queries, divided by the same probability for all Bing users in the United States. For the second analysis, we defined a window of time prior to the first query seeking information on the 12-step programs for each person. We then constructed a 2 × 2 table for each symptom (Table 1) and calculated the Chi-square score of this table. We only retained symptoms where the probability of asking about a symptom was greater in the time window than all previous times and where the Chi-square score was statistically significant at P<.05 with Bonferroni correction (P=.05/[152*3]=.00011). This approach identified cases where there was a statistically significant increase in the queries for a symptom in the days prior to the first query seeking information about treatment.

    Figure 1. Procedure for data collection, filtering, and analysis with a between-group comparison and time-lagged within-group comparison of medical symptom queries. NA: narcotics anonymous; AA: alcoholics anonymous.
    View this figure
    Table 1. A 2 × 2 table for each symptom to evaluate the change in queries for the symptom in each population.
    View this table


    When comparing the data of all the people who queried a medical symptom in the United States, our analysis of 195 medical symptoms revealed that 10 medical symptoms predicted AA information seeking and 9 symptoms predicted NA information seeking. Results also identified 10 medical symptoms that predicted subsequent querying for an AA meeting and one symptom (bloating) that predicted subsequent querying for an NA meeting. Of these symptoms, a substantial number could be categorized as nonsevere in nature (Textbox 1). Moreover, when medical symptom persistence was examined across a 1-month time period, a substantial number of nonsevere, yet persistent, symptoms were identified (Table 2).

    We first hypothesized that given the evidence of heightened levels of delayed discounting in SUD populations compared to controls [31,32], more immediately discomforting and socially embarrassing medical symptoms and conditions would be most prevalent for those who subsequently search for information on the 12-step treatment program. This first hypothesis was only partially confirmed. As hypothesized, compared to all other Bing search users in the United States, the probability of a user querying about AA or NA, in general, or the meeting information was significantly increased by earlier queries for nonsevere symptoms and conditions such as bloating, sweating, hives, heartburn, dizziness, bulging eyes, impotence, and back ache. However, several severe symptoms significantly increased the likelihood of subsequent AA or NA queries, although it was unclear if they were primarily medical, psychiatric, or related to substance withdrawal.

    We also hypothesized that the symptom queries that were significantly linked to immediate/near-immediate seeking of 12-step information would be severe or medically dangerous; severe symptoms would not require persistence over time to produce treatment motivation. Surprisingly, this second hypothesis was not confirmed. No medical symptom queries were significantly linked to immediate seeking of NA information or concurrent NA and AA information (eg, co-occurring drug and alcohol addiction treatment). In addition, the symptom queries that were significantly linked to the immediate seeking of AA information primarily consisted of common and nonsevere medical conditions such as cramp, rash, toothache, and dry mouth (the exceptions were hallucination and phobia, which could also be psychiatric or related to withdrawal).

    Last, we hypothesized that the persistent symptom queries that were significantly linked to subsequent seeking of information on the 12-step programs would be common or nonsevere in nature; a common or nonsevere symptom would need to be persistent to sufficiently compel a search for addiction treatment information. This third hypothesis was only partially confirmed, as several common and nonsevere symptoms prompted near-immediate search for AA (but not for NA or for NA and AA) information. However, as hypothesized, most persistent and near-persistent symptom queries were about common or nonsevere symptoms (ie, blister, itch, swelling, tinnitus, dizziness, heartburn, and diarrhea for AA and cough, cramp, constipation, fever, and back pain for NA).

    Notably, some symptoms were vague regarding severity. Specifically, regarding blurred vision for AA, pain and weight loss for NA, and pain for NA and AA, the extent of pain and weight loss were not specified and blurred vision could potentially range from mild (yet persistent) to severely impairing vision loss. Queries about the vague symptom of “deformity” was also nearly persistent for AA information seekers, which may reflect concerns about appearance with increasing alcohol misuse but may also reflect concerns about pregnancy and alcohol-related birth defects.

    In contrast, symptoms such as hallucination, akathisia, amnesia, blindness, and cataplexy (ie, the brief loss of voluntary muscle movement) can more clearly be categorized as severe, although in the absence of comprehensive psychiatric interviews, it was not possible to discern whether these symptoms reflected a primary medical condition/psychiatric disorder or were secondary to substance intoxication, substance withdrawal, or the adverse effects of psychiatric medications (see Limitations).

    Textbox 1. Symptom queries preceding the first search for information on the 12-step programs. Medical symptoms and conditions ranked from the highest to lowest according to the ratio of the probability that the symptom will be queried by those who later search for information on the 12-step programs, compared to the rest of all United States Bing users. A Chi-square test found no significant differences between medical symptom queries that preceded searches for information on the 12-step programs versus meeting-specific searches for the 12-step programs. For symptoms appearing in both general and meeting-only categories, the differences in the number of searches is statistically significant.
    View this box
    Table 2. Medical symptom search increasing in a statistically significant manner over the 30 days preceding seeking of information on the 12-step programs.
    View this table


    Principal Results

    The present study used anonymous Web-search log data analysis to examine which medical symptom queries best motivated subsequent searches for the general and meeting information on the 12-step programs. Our results suggest that queries about more severe or dangerous medical consequences as well as more nonsevere or common symptoms were important in motivating individuals toward subsequently seeking information on the 12-step programs.

    Our findings support past work demonstrating the high prevalence of co-occurring psychiatric disorders in addiction [60,61] and the current knowledge base supporting the efficacy of physician advice about severe long-term medical consequences in motivating individuals towards addiction treatment options [62].

    These results underscore an important point: Our results do not support the notion that more severe, long-term medical consequences of drugs and alcohol should, in any way, be ignored during brief advice by a physician, but rather, that adding on to current BI practices with additional emphasis on common nonsevere symptoms may be additionally beneficial.

    Potential Underlying Mechanisms

    Our findings raise questions about the possible underlying mechanisms that might make common or nonsevere medical symptoms persuasive for people to seek information on 12-step programs. One possible underlying mechanism is that more common symptoms may better allow individuals to envision the negative medical consequences of AUDs and SUDs compared to more chronic or dangerous symptoms. In other words, since a symptom like heartburn is common, it may be easy to identify with the experience of the symptom, notice it worsening with increasing substance use, and envision it becoming even more persistent or chronic. In contrast, it may be difficult for an individual to envision obtaining a cancer diagnosis or its likelihood may be discounted as too distant for concern; moreover, for those who can envision a cancer diagnosis, such envisioning may be so frightening that it fosters denial rather than motivation. Emphasizing the potential for worsening frequency and discomfort of otherwise common symptoms (eg, heartburn, dizziness, hives, and back ache) in addition to severe medical consequences could help further motivate individuals to accept treatment after being newly identified in primary care settings as having an AUD or SUD.

    A second possible mechanism underlying our findings may be related to the social embarrassment associated with certain medical symptoms. For example, symptoms such as bloating, bulging eyes, and impotence may impact physical attractiveness. Social conformity has been shown to be an important motive for alcohol and drug use [63], with alcohol, in particular, being considered a social lubricant [64]. Adding to BI’s current practices, highlighting how the medical consequences of substance use can be socially embarrassing may also help motivate patients newly diagnosed with AUD or SUD to consider treatment options including 12-step programs. Although discussion of the social repercussions of drinking alcohol is already a component of the brief motivational interviewing that is sometimes incorporated into BI, our results suggest it may provide additional motivation to discuss how medical and nonmedical consequences can interact to intensify social embarrassment in patients with newly identified AUD or SUD.

    In sum, our results suggest that many common or nonsevere medical symptoms and conditions motivate subsequent engagement with 12-step programs. As a result, BI could be restructured in multiple ways to maximize patients’ motivation to engage in treatment, including accompanying current BI practices and highlighting the common medical symptoms that could immediately worsen with increasing substance misuse and how some of those worsening symptoms could exacerbate social embarrassment. Such findings may help inform SBIRT efforts to screen individuals who present with medical complaints for AUD, provide brief interventions that motivate individuals toward considering treatment options, and provide referrals to AUD treatments. Specifically, our findings suggest that it may be beneficial to modify SBIRT’s BIs to add focus on the contribution of alcohol to immediately discomforting and socially embarrassing medical symptoms.


    A primary limitation of the present study was the anonymous nature of Web-search log data, which did not allow us to collect information on multiple demographic factors including age, sex, ethnicity, and socioeconomic status. Another primary limitation was our inability to follow-up on querying behavior to gauge whether search queries translated into real-world action. For example, although someone in the Prochaska and DiClemente [65] precontemplation stage of change would be very unlikely to run a search query for “AA information” (and even more unlikely to query for “AA meeting near me”), we cannot be certain whether search queries denote a contemplation, preparation, or action stage of change. In other words, we were unable to determine if searching for “AA meeting near me” was a query soon followed by AA meeting attendance or whether it was only gathering information for those contemplating or preparing to make a change but not yet ready to take action.

    We also noted that in a minority of cases, people may be searching for others and multiple people may be using the same browser, thus conflating multiple people to the same user identifier. However, although this unlikely possibility cannot be ruled out entirely, it is estimated based on past work [66] that these searches are limited to only a small fraction of the data. For instance, although the keyword “alanon” was included, which typically reflects family members searching for loved ones, we found that only 1.4% of queries we studied contained this keyword.

    Our study also lacked comprehensive psychiatric interviews, and thus, it was not possible to discern if certain symptoms were suggestive of co-occurring psychiatric conditions, the adverse side effects of psychiatric medications, substance-withdrawal symptoms, severe medical consequences of substance misuse, the use of multiple substances, physical comorbidities related to the searched-for symptoms, or the social consequences of excessive drinking. For instance, symptoms like hallucination and akathisia could reflect substance withdrawal (ie, withdrawal-related hallucinations and severe agitation) or a co-occurring psychotic disorder alongside the adverse effects of the antipsychotic medication used to treat it. Similarly, phobia queries may signify a co-occurring anxiety disorder or withdrawal-related anxiety increases. Further, queries about amnesia may reflect concerns about blackout drinking, withdrawal-related states of delirium, or the severe cognitive repercussions of long-term substance misuse (ie, memory loss).

    Our anonymous online search data also grouped all people with NA queries together, and thus, our data did not allow us to differentiate between subgroups of narcotics being misused. For example, stimulants and sedatives have different and often opposing medical consequences. Thus, we did not have access to data that could have improved our precision in delineating the medical symptoms that preceded NA information queries.

    Our data were exclusively derived from Bing searches and did not include data from other search engines such as Google or Yahoo. Future studies are needed to replicate our findings in other search engines as well as at a smaller scale where demographic characteristics can be more comprehensively collected (and statistically compared and contrasted) rather than estimated based on profoundly large usage data.

    Our study scope was limited to an investigation of how delay discounting may impact online treatment information seeking, and indeed, there are many barriers to addiction treatment beyond delay discounting that our study did not investigate. This limited scope reflected our need to maintain study focus. Future studies are needed to expand on other potential barriers. Further, the conceptual scope of our study was also limited by the technical limitations of online search data analysis, which relies on an adequately large number of single keywords or short phrases that are well defined and unambiguous in their meaning and ability to be linked together. First, our treatments were limited to twelve step programs without other forms of evidence-based treatment such as counseling or medication. A prerequisite of studying online search data is that any chosen query sets must be reasonably likely to have sufficient public visibility to gather a large enough number of responses for analysis. As a result, we narrowed our scope to 12-step programs, the most highly utilized form of addiction treatment [67] and therefore the most likely to have the greatest number of search queries. Second, we limited our predictors to adverse medical symptoms without other kinds of addiction-related problems such as social or legal consequences. Online search data analysis is not yet able to adequately capture more nuanced metrics such as social and legal problems without a significant number of false positives. To address this limitation, we opted to solely focus on medical symptoms because of pre-existing mapping of Web-search queries to medical symptoms that in the past work, has successfully limited false positives and avoided off-base random exploration [68]. Although accommodating these technical limitations narrowed our study focus, elucidating how medical symptoms impact seeking information on the 12-step program can offer unique insights critical to SBIRT and other early intervention efforts in primary care settings. Nevertheless, future studies are needed to expand upon the scope of our line of research inquiry to include other forms of treatment and consequences of addiction.


    In conclusion, our anonymous search log data analysis indicated that querying about the 12-step program was preceded by the occurrence of a number of nonsevere medical consequences of alcohol and drug misuse. The persistence of these nonsevere symptoms also appeared to play an important role in motivating individuals to query about the 12-step program. These findings can help inform modifications to current SBIRT protocols by concurrently emphasizing the long-term medical consequences of alcohol and drug misuse alongside shorter-term sets of symptoms that are more immediately discomforting and socially embarrassing. Such modified SBIRT protocols may apply BI to improve the motivation of the ~17% and 22.7% of individuals who screen positively during primary care encounters (using brief screening instrument scores) for risky/problematic alcohol and drug use, respectively [69,27] in order to accept referrals to 12-step programs or other outside addiction-treatment resources.

    Conflicts of Interest

    EYT is an employee of Microsoft Corp, the owner of Bing. No other conflicts of interest are declared.

    Multimedia Appendix 1

    Alcoholics Anonymous and Narcotics Anonymous webpages queried using the Bing search engine.

    PDF File (Adobe PDF File), 91KB


    1. Aalto M, Seppä K, Mattila P, Mustonen H, Ruuth K, Hyvärinen H, et al. Brief intervention for male heavy drinkers in routine general practice: a three-year randomized controlled study. Alcohol Alcohol 2001;36(3):224-230. [Medline]
    2. Nilsen P, Aalto M, Bendtsen P, Seppä K. Effectiveness of strategies to implement brief alcohol intervention in primary healthcare. A systematic review. Scand J Prim Health Care 2006 Mar;24(1):5-15. [CrossRef] [Medline]
    3. Reid MC, Fiellin DA, O'Connor PG. Hazardous and harmful alcohol consumption in primary care. Arch Intern Med 1999;159(15):1681-1689. [Medline]
    4. Chou SP, Grant BF, Dawson DA. Medical consequences of alcohol consumption--United States, 1992. Alcohol Clin Exp Res 1996 Nov;20(8):1423-1429. [Medline]
    5. Corrao G, Bagnardi V, Zambon A, La Vecchia C. A meta-analysis of alcohol consumption and the risk of 15 diseases. Prev Med 2004 May;38(5):613-619. [CrossRef] [Medline]
    6. Danaei G, Ding EL, Mozaffarian D, Taylor B, Rehm J, Murray CJ, et al. The preventable causes of death in the United States: comparative risk assessment of dietary, lifestyle, and metabolic risk factors. PLoS Med 2009 Apr 28;6(4):e1000058 [FREE Full text] [CrossRef] [Medline]
    7. Ezzati M, Lopez AD, Rodgers A, Vander Hoorn S, Murray CJ, Comparative Risk Assessment Collaborating Group. Selected major risk factors and global and regional burden of disease. Lancet 2002 Nov 02;360(9343):1347-1360. [CrossRef] [Medline]
    8. Rehm J, Baliunas D, Borges GLG, Graham K, Irving H, Kehoe T, et al. The relation between different dimensions of alcohol consumption and burden of disease: an overview. Addiction 2010 May;105(5):817-843 [FREE Full text] [CrossRef] [Medline]
    9. Shield KD, Parry C, Rehm J. Chronic diseases and conditions related to alcohol use. Alcohol Res 2013;35(2):155-173 [FREE Full text] [Medline]
    10. Patra J, Taylor B, Irving H, Roerecke M, Baliunas D, Mohapatra S, et al. Alcohol consumption and the risk of morbidity and mortality for different stroke types--a systematic review and meta-analysis. BMC Public Health 2010 May 18;10:258 [FREE Full text] [CrossRef] [Medline]
    11. Puddey IB, Croft KD. Alcohol, stroke and coronary heart disease. Are there anti-oxidants and pro-oxidants in alcoholic beverages that might influence the development of atherosclerotic cardiovascular disease? Neuroepidemiology 1999;18(6):292-302. [CrossRef] [Medline]
    12. Samokhvalov AV, Irving HM, Rehm J. Alcohol consumption as a risk factor for atrial fibrillation: a systematic review and meta-analysis. Eur J Cardiovasc Prev Rehabil 2010 Dec;17(6):706-712 [FREE Full text] [CrossRef] [Medline]
    13. Taylor B, Irving HM, Baliunas D, Roerecke M, Patra J, Mohapatra S, et al. Alcohol and hypertension: gender differences in dose-response relationships determined through systematic review and meta-analysis. Addiction 2009 Dec;104(12):1981-1990. [CrossRef] [Medline]
    14. Samokhvalov AV, Irving HM, Rehm J. Alcohol consumption as a risk factor for pneumonia: a systematic review and meta-analysis. Epidemiol Infect 2010 Dec;138(12):1789-1795. [CrossRef] [Medline]
    15. Rehm J, Samokhvalov AV, Neuman MG, Room R, Parry C, Lönnroth K, et al. The association between alcohol use, alcohol use disorders and tuberculosis (TB). A systematic review. BMC Public Health 2009 Dec 05;9:450 [FREE Full text] [CrossRef] [Medline]
    16. Samokhvalov AV, Irving H, Mohapatra S, Rehm J. Alcohol consumption, unprovoked seizures, and epilepsy: a systematic review and meta-analysis. Epilepsia 2010 Jul;51(7):1177-1184 [FREE Full text] [CrossRef] [Medline]
    17. Baliunas DO, Taylor BJ, Irving H, Roerecke M, Patra J, Mohapatra S, et al. Alcohol as a risk factor for type 2 diabetes: A systematic review and meta-analysis. Diabetes Care 2009 Nov;32(11):2123-2132 [FREE Full text] [CrossRef] [Medline]
    18. Irving HM, Samokhvalov AV, Rehm J. Alcohol as a risk factor for pancreatitis. A systematic review and meta-analysis. JOP 2009 Jul 06;10(4):387-392 [FREE Full text] [Medline]
    19. Kristiansen L, Grønbaek M, Becker U, Tolstrup JS. Risk of pancreatitis according to alcohol drinking habits: a population-based cohort study. Am J Epidemiol 2008 Oct 15;168(8):932-937. [CrossRef] [Medline]
    20. Samokhvalov AV, Rehm J, Roerecke M. Alcohol Consumption as a Risk Factor for Acute and Chronic Pancreatitis: A Systematic Review and a Series of Meta-analyses. EBioMedicine 2015 Dec;2(12):1996-2002 [FREE Full text] [CrossRef] [Medline]
    21. Yadav D, Hawes RH, Brand RE, Anderson MA, Money ME, Banks PA, North American Pancreatic Study Group. Alcohol consumption, cigarette smoking, and the risk of recurrent acute and chronic pancreatitis. Arch Intern Med 2009 Jun 08;169(11):1035-1045. [CrossRef] [Medline]
    22. Rehm J, Taylor B, Mohapatra S, Irving H, Baliunas D, Patra J, et al. Alcohol as a risk factor for liver cirrhosis: a systematic review and meta-analysis. Drug Alcohol Rev 2010 Jul;29(4):437-445. [CrossRef] [Medline]
    23. Lopez AD, Mathers CD, Ezzati M, Jamison DT, Murray CJ. Global and regional burden of disease and risk factors, 2001: systematic analysis of population health data. Lancet 2006 May 27;367(9524):1747-1757. [CrossRef] [Medline]
    24. Donoghue K, Patton R, Phillips T, Deluca P, Drummond C. The effectiveness of electronic screening and brief intervention for reducing levels of alcohol consumption: a systematic review and meta-analysis. J Med Internet Res 2014;16(6):e142 [FREE Full text] [CrossRef] [Medline]
    25. Bertholet N, Daeppen J, Wietlisbach V, Fleming M, Burnand B. Reduction of alcohol consumption by brief alcohol intervention in primary care: systematic review and meta-analysis. Arch Intern Med 2005 May 9;165(9):986-995. [CrossRef] [Medline]
    26. Bray JW, Del Boca FK, McRee BG, Hayashi SW, Babor TF. Screening, Brief Intervention and Referral to Treatment (SBIRT): rationale, program overview and cross-site evaluation. Addiction 2017 Dec;112 Suppl 2:3-11. [CrossRef] [Medline]
    27. Madras BK, Compton WM, Avula D, Stegbauer T, Stein JB, Clark HW. Screening, brief interventions, referral to treatment (SBIRT) for illicit drug and alcohol use at multiple healthcare sites: comparison at intake and 6 months later. Drug Alcohol Depend 2009 Jan 01;99(1-3):280-295 [FREE Full text] [CrossRef] [Medline]
    28. Vendetti J, Gmyrek A, Damon D, Singh M, McRee B, Del Boca F. Screening, brief intervention and referral to treatment (SBIRT): implementation barriers, facilitators and model migration. Addiction 2017 Dec;112 Suppl 2:23-33. [CrossRef] [Medline]
    29. McCabe SE, Cranford JA, West BT. Trends in prescription drug abuse and dependence, co-occurrence with other substance use disorders, and treatment utilization: results from two national surveys. Addict Behav 2008 Oct;33(10):1297-1305 [FREE Full text] [CrossRef] [Medline]
    30. MacKillop J, Amlung MT, Few LR, Ray LA, Sweet LH, Munafò MR. Delayed reward discounting and addictive behavior: a meta-analysis. Psychopharmacology (Berl) 2011 Aug;216(3):305-321 [FREE Full text] [CrossRef] [Medline]
    31. Bickel WK, Jarmolowicz DP, Mueller ET, Koffarnus MN, Gatchalian KM. Excessive discounting of delayed reinforcers as a trans-disease process contributing to addiction and other disease-related vulnerabilities: emerging evidence. Pharmacol Ther 2012 Jun;134(3):287-297 [FREE Full text] [CrossRef] [Medline]
    32. Ginsberg J, Mohebbi MH, Patel RS, Brammer L, Smolinski MS, Brilliant L. Detecting influenza epidemics using search engine query data. Nature 2009 Feb 19;457(7232):1012-1014. [CrossRef] [Medline]
    33. Paparrizos J, White RW, Horvitz E. Screening for Pancreatic Adenocarcinoma Using Signals From Web Search Logs: Feasibility Study and Results. J Oncol Pract 2016 Jun 7:737-744. [CrossRef] [Medline]
    34. Paul MJ, White RW, Horvitz E. Search and Breast Cancer. ACM Trans Web 2016 May 25;10(2):1-27. [CrossRef]
    35. Yom-Tov E, Gabrilovich E. Postmarket drug surveillance without trial costs: discovery of adverse drug reactions through large-scale analysis of web search queries. J Med Internet Res 2013;15(6):e124 [FREE Full text] [CrossRef] [Medline]
    36. Andalibi N, Ozturk P, Forte A. Depression-related imagery on Instagram. 2015 Feb 28 Presented at: The 18th ACM Conference Companion on Computer Supported Cooperative Work & Social Computing (pp. ). ACM; Feb 28, 2015; Vancouver, BC, Canada p. 231-234. [CrossRef]
    37. Birnbaum ML, Ernala SK, Rizvi AF, De Choudhury M, Kane JM. A Collaborative Approach to Identifying Social Media Markers of Schizophrenia by Employing Machine Learning and Clinical Appraisals. J Med Internet Res 2017 Dec 14;19(8):e289 [FREE Full text] [CrossRef] [Medline]
    38. Chancellor S, Lin Z, Goodman EL, Zerwas S, De Choudhury M. Quantifying and predicting mental illness severity in online pro-eating disorder communities. 2016 Presented at: Proceedings of the 19th ACM Conference on computer-supported Cooperative Work & Social Computing; Feb 27-Mar 02, 2016; San Francisco, CA p. 1171-1184   URL: https:/​/www.​​publication/​311493030_Quantifying_and_Predicting_Mental_Illness_Severity_in_Online_Pro-Eating_Disorder_Communities [CrossRef]
    39. De Choudhury M, Gamon M, Counts S, Horvitz E. Predicting depression via social media. 2013 Presented at: Proceedings of the Seventh International AAAI Conference on Weblogs and Social Media; July 8-11, 2013; Cambridge, MA   URL:
    40. De Choudhury M, Kiciman E, Dredze M, Coppersmith G, Kumar M. Discovering Shifts to Suicidal Ideation from Mental Health Content in Social Media. Proc SIGCHI Conf Hum Factor Comput Syst 2016 May;2016:2098-2110 [FREE Full text] [CrossRef] [Medline]
    41. Harman G, Dredze M. Measuring post traumatic stress disorder in Twitter. 2014 Presented at: Proceedings of the Eighth International AAAI Conference on Weblogs and Social Media; June 1-4, 2014; Ann Arbor, MI p. 2098-2110   URL:
    42. Kumar M, Dredze M, Coppersmith G, De Choudhury M. Detecting Changes in Suicide Content Manifested in Social Media Following Celebrity Suicides. HT ACM Conf Hypertext Soc Media 2015 Sep;2015:85-94 [FREE Full text] [CrossRef] [Medline]
    43. Mitchell M, Hollingshead K, Coppersmith G. Quantifying the language of schizophrenia in social media. 2018 Jun 05 Presented at: The 2nd workshop on Computational linguistics and clinical psychology: From linguistic signal to clinical reality (pp. ); 2015; Denver, CO p. 11-20.
    44. Murnane E, Counts S. Unraveling abstinencerelapse: smoking cessation reflected in social media. 2014 Apr 26 Presented at: The SIGCHI Conference on Human Factors in Computing Systems; Apr 26, 2014; Toronto, ON, Canada p. 1345-1354. [CrossRef]
    45. Saha K, Weber I, Birnbaum ML, De Choudhury M. Characterizing Awareness of Schizophrenia Among Facebook Users by Leveraging Facebook Advertisement Estimates. J Med Internet Res 2017 Dec 08;19(5):e156 [FREE Full text] [CrossRef] [Medline]
    46. Tamersoy A, De Choudhury M, Chau DH. Characterizing Smoking and Drinking Abstinence from Social Media. HT ACM Conf Hypertext Soc Media 2015 Sep;2015:139-148 [FREE Full text] [CrossRef] [Medline]
    47. Yom-Tov E, Fernandez-Luque L, Weber I, Crain SP. Pro-anorexia and pro-recovery photo sharing: a tale of two warring tribes. J Med Internet Res 2012 Nov 07;14(6):e151 [FREE Full text] [CrossRef] [Medline]
    48. Kim SJ, Marsch LA, Hancock JT, Das AK. Scaling Up Research on Drug Abuse and Addiction Through Social Media Big Data. J Med Internet Res 2017 Oct 31;19(10):e353 [FREE Full text] [CrossRef] [Medline]
    49. Yom-Tov E. Crowdsourced Health: How What You Do on the Internet Will Improve Medicine. Cambridge, MA: MIT Press; 2016.
    50. Mejova Y, Weber I, Fernandez-Luque L. Online Health Monitoring using Facebook Advertisement Audience Estimates in the United States: Evaluation Study. JMIR Public Health Surveill 2018 Mar 28;4(1):e30 [FREE Full text] [CrossRef] [Medline]
    51. Frijters P, Johnston DW, Lordan G, Shields MA. Exploring the relationship between macroeconomic conditions and problem drinking as captured by Google searches in the U.S. Soc Sci Med 2013 May;84:61-68. [CrossRef] [Medline]
    52. Zhang Z, Zheng X, Zeng DD, Leischow SJ. Tracking Dabbing Using Search Query Surveillance: A Case Study in the United States. J Med Internet Res 2016 Sep 16;18(9):e252 [FREE Full text] [CrossRef] [Medline]
    53. Mueller J, Jay C, Harper S, Davies A, Vega J, Todd C. Web Use for Symptom Appraisal of Physical Health Conditions: A Systematic Review. J Med Internet Res 2017 Jun 13;19(6):e202 [FREE Full text] [CrossRef] [Medline]
    54. Ofran Y, Paltiel O, Pelleg D, Rowe JM, Yom-Tov E. Patterns of information-seeking for cancer on the internet: an analysis of real world data. PLoS One 2012;7(9):e45921 [FREE Full text] [CrossRef] [Medline]
    55. Kelly JF, Greene MC, Bergman BG. Do Drug-Dependent Patients Attending Alcoholics Anonymous Rather than Narcotics Anonymous Do As Well? A Prospective, Lagged, Matching Analysis. Alcohol Alcohol 2014 Nov;49(6):645-653 [FREE Full text] [CrossRef] [Medline]
    56. Comscore. 2018. Latest Rankings   URL: [accessed 2019-02-27] [WebCite Cache]
    57. Rosenblum S, Yom-Tov E. Seeking Web-Based Information About Attention Deficit Hyperactivity Disorder: Where, What, and When. J Med Internet Res 2017 Dec 21;19(4):e126 [FREE Full text] [CrossRef] [Medline]
    58. Dong J, Yom-Tov E, Yom-Tov GB. The Impact of Delay Announcements on Hospital Network Coordination and Waiting Times. Management Science 2018 Jul 30:1-26. [CrossRef]
    59. Yom-Tov E, Marino B, Pai J, Harris D, Wolf M. The Effect of Limited Health Literacy on How Internet Users Learn About Diabetes. J Health Commun 2016 Oct;21(10):1107-1114. [CrossRef] [Medline]
    60. Compton WM, Thomas YF, Stinson FS, Grant BF. Prevalence, correlates, disability, and comorbidity of DSM-IV drug abuse and dependence in the United States: results from the national epidemiologic survey on alcohol and related conditions. Arch Gen Psychiatry 2007 May;64(5):566-576. [CrossRef] [Medline]
    61. Hasin DS, Stinson FS, Ogburn E, Grant BF. Prevalence, correlates, disability, and comorbidity of DSM-IV alcohol abuse and dependence in the United States: results from the National Epidemiologic Survey on Alcohol and Related Conditions. Arch Gen Psychiatry 2007 Jul;64(7):830-842. [CrossRef] [Medline]
    62. Gaume J, McCambridge J, Bertholet N, Daeppen J. Mechanisms of action of brief alcohol interventions remain largely unknown - a narrative review. Front Psychiatry 2014;5:108 [FREE Full text] [CrossRef] [Medline]
    63. Simons J, Correia CJ, Carey KB. A comparison of motives for marijuana and alcohol use among experienced users. Addict Behav 2000;25(1):153-160. [Medline]
    64. Monahan J, Lannutti P. Alcohol as social lubricant. Human Comm Res 2000 Apr;26(2):175-202. [CrossRef]
    65. Prochaska JO, DiClemente CC. Transtheoretical therapy: Toward a more integrative model of change. Psychotherapy: Theory, Research & Practice 1982;19(3):276-288. [CrossRef]
    66. Yom-Tov E, Borsa D, Hayward AC, McKendry RA, Cox IJ. Automatic identification of Web-based risk markers for health events. J Med Internet Res 2015 Jan 27;17(1):e29 [FREE Full text] [CrossRef] [Medline]
    67. Cohen E, Feinn R, Arias A, Kranzler HR. Alcohol treatment utilization: Findings from the National Epidemiologic Survey on Alcohol and Related Conditions. Drug and Alcohol Dependence 2007 Jan 12;86(2-3):214-221. [CrossRef]
    68. Yom-Tov E, Muennig P, El-Sayed AM. Web-Based Antismoking Advertising to Promote Smoking Cessation: A Randomized Controlled Trial. J Med Internet Res 2016 Nov 21;18(11):e306 [FREE Full text] [CrossRef] [Medline]
    69. Pilowsky D, Wu L. Screening for alcohol and drug use disorders among adults in primary care: a review. Subst Abuse Rehabil 2012 Apr;3(1):25-34 [FREE Full text] [CrossRef] [Medline]


    AA: alcoholics anonymous
    AUD: alcohol use disorder
    BI: brief intervention
    NA: narcotics anonymous
    SBIRT: Screening, Brief Intervention, and Referral to Treatment
    SUD: substance use disorder

    Edited by G Eysenbach; submitted 02.05.18; peer-reviewed by M Meacham, R Zerbetto; comments to author 06.10.18; revised version received 28.11.18; accepted 26.01.19; published 03.05.19

    ©George Nitzburg, Ingmar Weber, Elad Yom-Tov. Originally published in the Journal of Medical Internet Research (, 04.04.2019.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on, as well as this copyright and license information must be included.