Studies of Internet search patterns provide a low cost, rapidly accessible data source across a range of disciplines. Underpinning these studies is the principle that each Internet search is a behavioral measure of an issue’s importance to an individual [1]. If individuals are concerned or interested in an issue, they are more likely to search for information related to that issue. The relative importance of an issue can thus be inferred from the volume of search queries for a specific term or terms representing that issue.

The first studies of Internet search patterns were related to medicine. These initial studies examined the quality of online information [2], searches for cancer related information [3], and influenza surveillance use diverse data sources including Google advertisements [4], and Yahoo search trends [5]. However it was the release of Google Insights For Search (GIFS) [6] and the publication of 3 influential articles in 2009 that provided an impetus to this emerging field. The 3 influential articles are Ginsberg's study of influenza surveillance using Google data in Nature [7], Brownstein's review of online surveillance in the New England Journal of Medicine [8], and Eysenbach's consolidation of infodemiology as a distinct field of medical inquiry [9]. Eysenbach describes infodemiology as “the science of distribution and determinants of information ...specifically (on) the Internet, or in a population, with the ultimate aim to inform public health and public policy...(including) data on what people browse, buy, and read”. Where infodemiology methods are used for epidemiological surveillance, Eysenbach refers to this as infoveillance. Since 2009, Google data has been the main data source for infodemiology studies, across a wide range of health problems including dengue [10], depression [11], abortion [12], tobacco control [13], and the global Google Flu Trends site [14].

Infodemiology methods have also been applied to the study of political communications and policy processes [1,2-20]. These studies are generally founded on agenda setting theory, and use Internet search patterns as a data source to complement opinion polling or traditional media (ie, television and print) coverage. Agenda setting theory suggests that issues prominently covered in traditional media are subsequently ranked as important (or salient) in public opinion polls [21,22]. The transfer of issue salience from reporting in the media to influence public opinion is an important concept in agenda setting theory [23]. Agenda setting theory appears in health advocacy studies [24], but has not yet been incorporated into studies of Internet search patterns for health policy processes.

Since the early 2000s, studies of issue salience have increasingly focused on the interplay between traditional, online media, and the public agenda. A study of online bulletin board discussions found that media reports were rapidly reflected in online discussions [25]. Rather than several weeks, themes emerged in online discussions within days of traditional media reports. More recent studies have used GIFS to measure and analyse search patterns in responses to prominent media issues. Granka suggests that issue importance can be inferred from overall changes in search query volume, and that search volumes rise and fall rapidly with public interest [1]. Similarly, Scharkow suggests search patterns are the behavioral effects of salience, and provide valid and reliable measures of the public agenda [18].

Most studies of issue salience using search have been conducted in the United States and Organization for Economic Cooperation and Development (OECD) countries. However, there is some uncertainty as to whether the transfer of salience from traditional or online media to public opinion is universally applicable in low and middle income countries with different institutional arrangements. For example McCombs suggested that agenda setting effects require a reasonably free political system and media [26]. Other authors too, have noted the lack of research into how media shape public opinion in less-than-democratic nations. For example, Moy and colleagues point to "a glaring absence of (research about)... how citizens in these states respond to specific televised messages or their attitudes regarding certain political and social issues" [27]. This means it is difficult to infer public opinion from findings of issue salience in online media outside of high income liberal democracies.

The Russian Federation is a middle income country with institutional arrangements that sharply contrast to those in the US or European Union (EU). Contemporary Russia has been described as a managed democracy [28] and an authoritarian state [29]. Several studies have pointed to a complex relationship between Russian traditional media, online media, and public opinion. A study of Russian's reactions to news broadcasts in 2005 reported marked differences between Russian and US viewer reactions to television news [30]. Russians were found to adopt a range of cognitive strategies, routinely reinterpreting the frames presented in television news stories using complex reasoning outcomes. These strategies were, the authors suggested, consistent with Soviet-era television viewing. Others have noted unique patterns of online media use. In 2009, Russians were the most engaged social media users globally [31]. Further, Russians engaged in unusually heterogeneous debates ranging across the political spectrum, as distinct from the partisan “echo chambers” that characterize online debate in the US [32]. These studies caution against a simple transposition of agenda setting and issue salience theories to traditional and online media. This suggests more complex process than that suggested by agenda setting and issue salience theories.

In 2011, Russia overtook Germany as the European country with the highest number of unique visitors online [33]. Russian Internet users grew from 43% of the population in 2010 [34] to 55% in 2012 [35]. In May 2011, Google provided 84% of Internet search queries globally [36]. The structure of the Russian-language Internet market is unique. Yandex provided 60% of Russian Internet searches in 2010-2011, compared with Google's 25% [37]. Further, Yandex offers the Wordstat (YaW) search pattern analysis tool as a direct competitor to GIFS (Table 1).

In response to political and media constraints, Russian political debate increasingly moved online during the 2000s, using platforms such as LiveJournal [38], and more recently, Twitter [39]. RuNet has been described as a catalyst for social activism [40] political mobilization [41], as well as a channel for an alternative news agenda [42]. As Internet use grew, corruption and abuses of government power emerged as important themes online. (See for example the anticorruption blog Rospil.net [43]). By 2012, the proliferation of Russian social commentary blogs prompted a Harvard study to described Russian online media as a “transparency watchdog” [39].

RuNet's rapid growth led Russian media commentators to speculate that the Internet had eclipsed television’s traditional agenda setting function in importance [44,45]. However, other observers have cautioned against overstating the importance of Russian online media, or its distance from mainstream practices. Less optimistic observers have described pro-government blogging campaigns [46], cyber attacks [47], monitoring dissent [48], and sophisticated security filtering through SORM 2 [49]. National surveys too, suggest a more modest role for online media in shaping public opinion. A 2012 survey found 63% of Russians mostly or completely believe traditional media, while 43% mostly believe online sources [50]. In summary, as elsewhere, online media provide Russians with an information source complementing traditional media.

Researchers have devoted considerable effort to establishing the validity of search pattern studies. Validity is the extent to which a test measures what it claims to measure [51]. The initial studies using GIFS established a correlation between search patterns and epidemiological surveillance data for influenza [7]. Other studies focusing on issue salience, established a correlation between search patterns, traditional media [1], and opinion polling [17]. Studies with large data sets have commonly employed ARIMA tests [52] vector auto regression and visual comparison [53], and multivariate regression [17]. Studies with smaller sample sizes have generally conducted bivariate analysis with little or no data preparation [18,54].

Several common warnings concerning validity recur in studies of search patterns. The unrepresentative demographic sampling of GIFS populations is the most frequently cited concern. The Internet user population is generally regarded as younger and wealthier than the overall population, although this cannot be elicited from search data directly [53]. Scharkow et al expand on this concern, questioning whether survey populations from traditional surveys and search pattern studies are comparable [18].

The second concern relates to the disambiguation of search terms. Individual search terms may return ambiguous search results. Care with selection of search terms, and an appropriate range of search terms is necessary to capture the breadth of potential search terms for a concept [55].

Third, several problems are associated with the limited transparency of Google's treatment of data. Google does not reveal the threshold search volume used to determine whether data is reported on a search term in GIFS [56]. This can produce unexpected zero values in time series. Further, Google provides results as relative rather than absolute data (That is, GIFS results are provided as a percentage relative to 100% during the user-defined date range-eg, 30% during June 2010). In addition, GIFS data is normalized and scaled, making comparisons between countries, regions and time spans difficult. Despite these limitations, there is a general consensus in the scholarly literature cited above that GIFS is a valid, low cost and flexible field research method. By analyzing online response to a Russian illicit drug policy episode, we hope to develop further develop the methods of search pattern analysis.

Illicit drug use is a serious social and policy problem in Russia. Russian public opinion surveys have consistently rated illicit drug use among the most serious of domestic social problems [57,58]. However, Russian policy responses to this problem are generally regarded as punitive, unsupported by scientific evidence, and ineffective [59]. The punitive aspect of Russian drug policies is exemplified in 2010 by a law prohibiting dissemination of drug related health information [60]. In this complex environment, Russian reformers have compared attempts at influencing drug policy to "throwing spaghetti against a wall, and seeing what will stick” [61].

We selected the public debate surrounding the court case against socially conservative drug policy reformer Egor Bychkov as our case study. Bychkov was the president of an NGO operating a non-medical drug rehabilitation center in the provincial Urals city of Nizhny Tagil, 1900 kilometers east of Moscow. In October 2010, a local court convicted Bychkov of holding several rehabilitation clients hostage. His subsequent imprisonment sparked widespread coverage in the Russian national media. Although Bychkov was outspoken in expressing reactionary social attitudes [62] and his belief in harsh, unscientific treatment methods, he won support from socially liberal as well as conservative commentators for bringing the fight against corrupt local courts and police into the open [63]. In November 2010, following national media support, and presidential intervention, Bychkov was released on parole [64]. In June 2011, all criminal charges against Bychkov were finally annulled [65]. In 2011, having achieved national prominence, Bychkov collaborated with leading liberal bloggers and opposition politicians in Moscow [66]. In summary, the Bychkov episode was one of several concurrent episodes of opposition to Russian government policies. In each case, traditional media sparked and spread popular outrage, leaving patterns of online search.

Google and Yandex search results were positively correlated overall. We found a consistent pattern of strong to moderate positive correlations between the two search indices for the same term, both for the Bychkov episode and concurrent political events (Table 4). However, the relationship was weaker than anticipated. For example, “illicit drugs” has a weak negative relationship (r=-0.15). This was likely the result of GIFS returning zero values over several weeks during the specified data range. For example, GIFS searches for the term “Bychkov” displayed zero values in April, June and August 2010. During this period, YaW consistently recorded 54,000-68,000 searches per month. This is an example of the GIFS “threshold value” problem identified in earlier studies [18,65]. Within any selected date range, GIFS scales search results relative to a 100% value within that range. This scaling produces different results depending on the date range selected. This effect on GIFS data is only evident when compared with YaW data. For example, GIFS search for the term “illicit drug addiction” returned 100% in October 2010 (date range March 2010-March 2012), and 100% in November 2011 (date range November-December 2011). YaW values for these peaks GIFS dates were 132,000 and 167,102, respectively (Table 4). In summary, we were able to quantify patterns of missing data identified in earlier studies based on the use of GIFS data.

Visual examination of weekly results (Figure 1) suggests weekly peaks in GIFS indices corresponded to major milestones during the Bychkov episode. This provided further non-statistical corroboration, and face validity to the relationship between media reports and GIFS searches during the Bychkov episode.

We found search volumes for the Bychkov episode were comparable to other prominent domestic political events during 2010 (Tables 4 and 5). GIFS values provide indicative comparisons of the relative importance of an event. However, YaW provides detailed raw numbers, allowing direct comparison of search patterns for an event across regions and across time. These tables provide comparative measures of the search volumes for the Bychkov episode and other concurrent events.

Google potentially provides timely search results, whereas Yandex provides more accurate geographic localization. However, both GIFS and YaW place restrictions on the data made available to the public. These restrictions are outlined in Table 1. Whereas GIFS potentially provides detailed, near real time daily data, we found gaps in actual data availability. For example, only weekly data was available for the search terms "Bychkov" and "Egor Bychkov". We described these gaps above in relation to GIFS threshold, scaling and normalization policies.

GIFS does not provide detailed subnational geographic location data in Russia. However, YaW provides disaggregated search data to the level of individual Russian provincial cities. This differentiation is important, as several of the most important domestic political events during 2010 were associated with events around the Russian capital Moscow. For example, the widely-reported Khimki forest and blue buckets protests, revealed an average of 65% and 64% of searches originating in Moscow (Table 3). This result suggests the protests were relatively more important to Moscow residents, even while Russian and international commentators ascribed national significance to these events [91,92]. By contrast, only 26% of searches for the term "Egor Bychkov" came from Moscow, and the remainder from other parts of Russia. The Bychkov episode also generated greater absolute search volumes. These two results suggest the Bychkov episode was more important across Russia than the metropolitan protests.

We found moderate to strong positive correlations between search terms representing the Bychkov episode and terms representing salient drug issues in Russian media. In YaW, we found a moderate positive correlation of the term “Egor Bychkov” with the terms "illicit drugs" (r _s = .77), "drug addiction" (r _s = .74), and a strong correlation with “illicit drug treatment” (r _s = .90). These correlations suggest searches for “Egor Bychkov” were positively correlated to salient Russian drug issues (Table 6). GIFS produced weaker or absent correlations. We attribute this to missing data as described earlier.

Previous studies have established the validity of GIFS data through relevant offline measures. This is the first study to validate GIFS results through YaW. We believe this approach has several advantages in the Russian-language context. By exploring the relationship between Google and Yandex search patterns in response to a drug policy debate, we were able to quantify several previously identified limitations of GIFS. We demonstrated shifting GIFS threshold values, and the extent of GIFS scaling and normalization of data through reference to YaW search results.

The presence of zero values in GIFS results merits additional discussion. The zeroes are artifices of censoring and that this means that variation in GIFS has been restricted, and therefore the correlation is biased downwards. The true correlation is likely to be stronger than the estimated statistic. The logic for the expected downward bias is that GIFS and YaW appear not to be substitutes, based upon the positive data observed.

Although time series in nature, the data were not adjusted for seasonality or non-stationarity, and autocorrelation was not assessed. This might have biased the results towards stronger, but spurious, correlation. However, this cannot currently be assessed due to the effect of the threshold used in reporting the GIFs data. If GIFS and YaW were substitutes this would bias the results in favor of a stronger negative correlation, which was not observed; if GIFS and YaW were complements (or if seasonality was strong), this would bias the results in favor of a stronger positive correlation. These issues will need to be explored in further analyses using larger and more transparent data.

At the same time, this result highlights some of the current limitations of publicly available search tools as a data source. Search pattern studies have emerged as an opportunistic response to the availability of GIFS and YaW marketing tools. The data Google and Yandex make available through these tools is only a small portion of that collected. Most of the limitations on data availability described in this paper are in fact constraints on data release imposed by the search providers themselves. Further, search providers routinely make changes to their services. For example, in September 2012, Google merged GIFS into the Trends online analysis service, incorporating GIFS capabilities into the latter [93].

Several studies have deployed GIFS to determine the importance of political episodes, as a low cost and rapid alternative to opinion polling [1,17]. We analyzed domestic events unrelated to the Bychkov episode to provide additional context for the Bychkov episode. We found search volumes for the Bychkov episode were broadly comparable to other concurrent domestic news events. For example, the controversy surrounding the jailed oligarch Mikhail Khodorkovsky was both reported in the domestic media, and produced high search volumes, suggesting this was an important issue. Conversely, the Khimki Forest and Blue Buckets protests produced low search volumes. While all three of these issues attracted international media coverage, our results suggest the last two of these were not important to Russians nationally. We did not set out to examine constraints on traditional media on Internet search. Future search pattern studies in Russia should account for the influence of mainstream media constraints on issue salience and Internet search.

Previous studies have described GIFS potential to complement existing public health data sources by providing timely, geographically precise [10,18], and otherwise inaccessible data [54]. Our results suggest timely GIFS data may not always be available in Russia. If GIFS data is missing, researchers will need to wait two months before YaW results are made available. While GIFS may not provide timely data useful for analyzing unfolding events, YaW is certain to provide delayed and detailed data. Geography is especially important in the Russian context. It is a large country, with many provincial cities, and considerable demographic variation. By comparing YaW raw data across specific regions, analysts may discern changes in search patterns for specific search terms across regions and across time. Based on our findings, we believe that YaW offers a potentially valuable tool to Russian drug policy researchers and advocates.

Illicit drug use has long been among one of the most important social problems troubling Russians [57]. We demonstrated a positive correlation between searches for Egor Bychkov and drug terms appearing in media reports associated with the Bychkov episode. This relationship merits further analysis. First, based on these results we are not able to distinguish between personal or sociotropic motivations for search [94]. That is, we were unable to determine whether the concurrent increase in searches for the terms "drug use", "addiction", and "drug treatment" were motivated by individual's health problems, or an interest in drug policy issues. This is consistent with Reis and Brownstein's observations concerning searches for US abortion information [12]. Second, unlike opinion polls, search patterns do not provide valency information. That is, search patterns offer no insight as to whether individuals support or oppose a specific issue. Further search terms to differentiate personal and sociotropic motivations, and to gauge valence should be considered in future search studies.

In conclusion, the Bychkov episode provides an opportunity to advance the science of search patterns. This paper investigated the relationship between Google and Yandex, and contributed to the broader methods literature by highlighting both the potential and limitations of these two search providers. We believe that YaW is a potentially valuable and underused data source for researchers working on Russian-related illicit drug policy and other public health problems. The Russian Federation, with its large, geographically dispersed, and politically engaged online population presents unique opportunities for studying the evolving influence of the Internet on politics and policy, using low cost methods resilient against potential increases in censorship. As online use grows further, primary sources of available online data will also grow. Adapting and refining research methods to best take advantage of these constantly evolving primary data sources is likely to present researchers in health policy and political communications with ongoing challenges.