JMIR Publications

Select Journals for Content Updates

When finished, please click submit.

This paper is in the following e-collection/theme issue:

    Original Paper

    Crowdsourcing Black Market Prices For Prescription Opioids

    1Epidemico, Boston, MA, United States

    2Boston Children’s Hospital, Children’s Hospital Informatics Program, Harvard Medical School, Boston, MA, United States

    3Boston Children’s Hospital, Department of Pediatrics, Harvard Medical School, Boston, MA, United States

    4Nova Southeastern University, Fort Lauderdale, FL, United States

    5RADARS System, Rocky Mountain Poison and Drug Center, Denver Health, Denver, CO, United States

    6Department of Emergency Medicine, University of Colorado School of Medicine, Aurora, CO, United States

    Corresponding Author:

    Nabarun Dasgupta, MPH, PhD

    Epidemico

    268 Newbury Street

    2nd Floor

    Boston, MA

    United States

    Phone: 1 9192603808

    Fax:1 8666571107

    Email:


    ABSTRACT

    Background: Prescription opioid diversion and abuse are major public health issues in the United States and internationally. Street prices of diverted prescription opioids can provide an indicator of drug availability, demand, and abuse potential, but these data can be difficult to collect. Crowdsourcing is a rapid and cost-effective way to gather information about sales transactions. We sought to determine whether crowdsourcing can provide accurate measurements of the street price of diverted prescription opioid medications.

    Objective: To assess the possibility of crowdsourcing black market drug price data by cross-validation with law enforcement officer reports.

    Methods: Using a crowdsourcing research website (StreetRx), we solicited data about the price that site visitors paid for diverted prescription opioid analgesics during the first half of 2012. These results were compared with a survey of law enforcement officers in the Researched Abuse, Diversion, and Addiction-Related Surveillance (RADARS) System, and actual transaction prices on a “dark Internet” marketplace (Silk Road). Geometric means and 95% confidence intervals were calculated for comparing prices per milligram of drug in US dollars. In a secondary analysis, we compared prices per milligram of morphine equivalent using standard equianalgesic dosing conversions.

    Results: A total of 954 price reports were obtained from crowdsourcing, 737 from law enforcement, and 147 from the online marketplace. Correlations between the 3 data sources were highly linear, with Spearman rho of 0.93 (P<.001) between crowdsourced and law enforcement, and 0.98 (P<.001) between crowdsourced and online marketplace. On StreetRx, the mean prices per milligram were US$3.29 hydromorphone, US$2.13 buprenorphine, US$1.57 oxymorphone, US$0.97 oxycodone, US$0.96 methadone, US$0.81 hydrocodone, US$0.52 morphine, and US$0.05 tramadol. The only significant difference between data sources was morphine, with a Drug Diversion price of US$0.67/mg (95% CI 0.59-0.75) and a Silk Road price of US$0.42/mg (95% CI 0.37-0.48). Street prices generally followed clinical equianalgesic potency.

    Conclusions: Crowdsourced data provide a valid estimate of the street price of diverted prescription opioids. The (ostensibly free) black market was able to accurately predict the relative pharmacologic potency of opioid molecules.

    J Med Internet Res 2013;15(8):e178)

    doi:10.2196/jmir.2810

    KEYWORDS



    Introduction

    The United States has a high level of concern with the diversion and public health consequences associated with the nonmedical use of prescription opioid analgesics [1-3]. Prescription opioid analgesics diverted from the pharmaceutical supply chain may end up being resold in open-air markets, in clubs and bars, or more subtly between friends and relatives [4].

    Street price data have many applications. They are routinely collected by law enforcement agencies, which rely on accurate street prices for agents to be credible buyers or sellers in undercover operations. On a policy level, the Drug Enforcement Administration cited street price data to justify assigning buprenorphine to Schedule III, a lesser category of regulation than methadone, oxycodone, and morphine [5]. Street prices for cocaine and heroin have been used as indicators of intervention impact in public policy [6,7], including as inputs for modeling the impact of policy decisions, understanding the profits and risks in the drug trade [8,9], informing debates on prohibition [10], evaluating the impact of interdiction, informing the timing of public health efforts [11,12], and understanding the impact of globalization and economic recession on drug street prices [13-15]. Behavioral economic studies in controlled settings have been used extensively in the last decade to explain and predict human behaviors associated with addictive disorders, focusing on impulse control (discounting) and the relative likeability of substances [16,17]. In pharmaceutical development, recent guidelines for pre-approval abuse liability studies for new pharmaceuticals increase reliance on laboratory behavioral economic assessments with drug users to determine differences in willingness to pay [18], but these data from controlled settings are not connected to real-world black market street price data in post-marketing surveillance. Finally, behavioral economics research has become an important tool in understanding decision making, drug dependence, and treatment choices for a variety of reinforcing substances [19-21].

    Although street price data are collected by local law enforcement, they have only occasionally been reported at a national level and are rarely made available for public health research [22]. The standard federal government source for this information, the Department of Justice’s National Drug Intelligence Center (NDIC), was closed in June 2012 as part of a broader realignment of federal drug policy with no notice of future data availability.

    An earlier study by our group suggested the Internet was an infrequent source of diverted drugs [23]. More recently, several anonymous online marketplaces operate via Tor hidden services (distributed traffic software enabling online anonymity) or using other identity-masking techniques. One such marketplace is Silk Road, where controlled substances can be purchased with a reasonable expectation of anonymity for both the purchaser and seller. We hypothesized that Silk Road could be a source of information on street prices for diverted prescription drugs.

    Given the interest but lack of scientific efforts to collect street price information, we sought to evaluate whether online crowdsourcing could be used to measure black market street prices. Crowdsourcing is a method for harnessing distributed human intelligence, where small pieces of independently derived information are systematically collected, often using electronic tools [24]. Crowdsourcing has been used in applied biomedical research to rapidly and efficiently complete tasks that would otherwise require large amounts of time, for example to evaluate medical pictograms [25] and multilanguage patient information [26], process patient narratives [27], collect soil samples across a large area [28], identify malaria parasites in slides [29], and others [30-32]. We have previously demonstrated that electronic crowdsourcing techniques can be used to produce rapid estimates in fields as varied as infectious disease incidence and illegal wildlife trade [33-35]. We hypothesized that the same could be true for street prices. Any use of the Internet as a source of information for public health research requires careful validation against established data sources and an understanding of biases that may be present in the data. Therefore, we conducted an experiment to cross-validate 2 sources of online street price data (StreetRx and Silk Road) with a more traditional survey of law enforcement officers.


    Methods

    Crowdsourced Data

    Launched on November 1, 2010, StreetRx is a collection of databases and websites, which gathers, organizes, and displays street price data on diverted pharmaceutical controlled substances for public health research purposes (see Figure 1) [36]. Site users anonymously submit prices they paid or heard were paid for diverted prescription drugs, specifying the drug formulation, dose, and the US city or state in which the transaction occurred. Date of entry is automatically collected. The system supports product codes and location information for the United States, Canada, and the United Kingdom. A visual photo identification feature for each formulation aids accuracy of reporting. In order to mitigate concerns about self-incrimination, the submitter can choose to identify the source of the information in 3 ways: personal experience, heard it from “someone who isn’t me” (SWIM), or the Internet. Links to information on drug treatment, overdose prevention, harm reduction, safe disposal, and pain management are also provided. Site visitors can query and view submitted prices at the city level using a map interface. Users have submitted links to online pharmacies, news media, government reports, and other public sources. StreetRx averages roughly 200 visitors and 20 street price submissions per day.

    Submissions for opioid analgesics that were received from the United States between January 1 and June 30, 2012, and contained data about formulation and dose strength were considered for this report. Based on previous crowdsourcing experiments, we deemed it necessary to have a systematic way to reduce noise in the data and identify less credible submissions. Outlier prices identified by site users as “cheap” or “overpriced” on a 5-point visual analog scale were excluded; approximately one quarter of all submissions were rated in these 2 categories. Because duplicate submissions for the same drug from the same IP address less than 10 seconds apart most likely indicated submission errors, only one of the dyad was retained.

    StreetRx is written in PHP programming language, with OpenLayers and jQuery user interface components. The data are stored in a MySQL relational database, on a scalable, secure hosting service with a proven track record of managing traffic spikes and high user load. Because the hosting provider also specializes in politically controversial content, the system is designed to resist attempts at being shut down due to objections to the site content. It relies on map tiles from Google Maps, but uses OpenLayers to render the map interface. The site also contains Google Analytics to track visitor volume and other statistics.

    Law Enforcement Data

    Reference data for street prices were obtained from the Researched Abuse, Diversion and Addiction Related Surveillance (RADARS) System Drug Diversion program, which collects data from approximately 280 police agencies in 49 US states on a quarterly basis. Methods of the RADARS System Drug Diversion Program, which is operated by the Center for Applied Research on Substance Use and Health Disparities, Nova Southeastern University (Miami, Florida), have been described previously [37]. For this study, a subset of 125 law enforcement reporters was selected based on prior consistency of reporting, level of diversion activity, and geographic distribution [38]. These agencies were from 46 US states, but were not sampled in a way to make them nationally representative. A standardized electronic form was developed to collect data about the prices paid for specific drug formulations and strengths. Each reporter received the survey quarterly, in April 2012 (covering January through March 2012) and July 2012 (covering April through June 2012). Respondents were instructed to provide prices for the most common dosage strength they encountered for each opioid during the time period of interest, and not to respond for opioids with no encounters during the time period.

    Dark Internet Online Marketplace Data

    Silk Road is an anonymous online marketplace structured as a Tor hidden service (see Figure 2) [39]. Prospective buyers access Silk Road through a distributed network, which provides anonymity to the IP addresses of both the buyer’s Web client and the Silk Road server [40]. Silk Road uses Bitcoin (BTC), an international peer-to-peer digital currency, for payments. Bitcoin prices were converted to US dollars using the weighted average price posted on a Bitcoin exchange website on the day the sale was posted to Silk Road [41]. The exchange rate between Bitcoin and US dollars during October 2012 was approximately 11 BTC to 1 USD. A subject matter expert (author CM) manually collected (“scraped”) prices per milligram for prescription opioids in the “Opioids” section from October 1 through October 31, 2012, and collected these data on a standardized electronic data collection form. Only posts that specified that the product would be shipped from the United States were scraped. No data cleaning steps were performed and no effort was made to purchase the drugs online. No “stealth listings” (unsearchable and unlinked listings that are accessible only by buyers who have been given the URL) were scraped.

    Figure 1. Screenshot of StreetRx - features street price data on diverted pharmaceutical controlled substances for public health research purposes.
    View this figure
    Figure 2. Screenshot of Silk Road - an anonymous online marketplace where drugs, fireworks, and stolen goods are sold.
    View this figure

    Opioids Studied

    The following drugs were initially considered for inclusion in the study: oral/sublingual dosing forms of buprenorphine, hydrocodone, hydromorphone, methadone, morphine, oxycodone, oxymorphone, tapentadol, tramadol, and transdermal patch formulations of fentanyl. Because only a small number of reports were available for tapentadol and transdermal fentanyl, these opioids were excluded from further analysis.

    Data Analysis

    Geometric means and 95% confidence intervals of the price per milligram were calculated for each opioid. Correlation between systems was assessed by comparing opioid-specific means using the nonparametric Spearman rank correlation coefficient (rho), and tested with the null hypothesis that data from each pair of systems were independent with a two-tailed significance threshold of 0.05. Data management and analysis were performed using STATA version 12. In a preplanned secondary analysis, we used a standardized equianalgesic dosing conversion table [42] to convert the strength of each formulation to milligram of morphine equivalent, and compared prices on this basis. The predicted potency was calculated by dividing the mean price per milligrams for each opioid by that of morphine.

    Ethics and Disclosure Statement

    Law enforcement data used in this study were reviewed by the Colorado Multiple Institutional Review Board (IRB), which also provides overall ethical oversight to the RADARS System. The Drug Diversion program is classified as exempt by the Nova Southeastern University IRB, as it does not constitute human subjects research.


    Results

    Data from 954 StreetRx reports, 737 Drug Diversion reports, and 147 postings on Silk Road were analyzed. The most reports were received for oxycodone and hydrocodone in each system (Table 1). Correlation between the 3 data sources was high. The Spearman correlation of prices per milligram between crowdsourced and law enforcement prices was 0.93 (P<.001; Figure 3, top frame), and the correlation between crowdsourced and online black market prices was 0.98 (P<.001; Figure 3, middle frame). Data from law enforcement and the online black market were also highly correlated (rho=0.90, P=.002; Figure 3, bottom frame).

    With the exception of morphine, there was no significant difference between the mean price per milligram of each opioid between the 3 data sources (Figure 4). The price per milligram of morphine was greater in the law enforcement survey at US$0.67/mg (95% CI 0.59-0.75), compared with $0.52/mg (95% CI 0.40-0.68) in the crowdsourced data and US$0.42/mg (95% CI 0.37-0.48) for the online black market (P=.048).

    Street prices paid for different opioids generally followed the rank order of oral equianalgesic opioid potency clinically used for rotation of opioid analgesics (Figure 4 and Table 2). The predicted potency or desirability relative to morphine was calculated by dividing the mean prices for each opioid by that of morphine. When compared to published clinical conversion guides, the predicted relative potency from crowdsourced data were similar. Oxymorphone and oxycodone had predicted potencies that were statistically indistinguishable from clinical conversion factors. Hydromorphone, hydrocodone, and methadone were valued higher on the street per milligram than in the clinic, while tramadol was valued lower on the street.

    Table 1. Mean black market street prices and equianalgesic potency, US dollars per milligram, from online and law enforcement data sources, United States, 2012.
    View this table
    Table 2. Mean street prices from crowdsourced data, adjusted for potency relative to morphine, United States, 2012.
    View this table
    Figure 3. Correlation between the data sources: StreetRx reports, Drug Diversion survey, and Silk Road postings.
    View this figure
    Figure 4. Mean price per milligram of each opioid analgesic, between the data sources. Numbers at the bottom of each bar indicate sample size.
    View this figure

    Discussion

    Principal Findings

    Data about the street price of diverted prescription opioid medications can be useful to policymakers and public health officials, but timely and accurate data are rarely available publicly. In this paper, we present findings of a national analysis of street price data for prescription opioid analgesics. Our findings show consistent prices per milligram across 3 independent sources, and that prices for different opioid active ingredients on the black market reflect their clinically established potency. We also demonstrate the feasibility of validating crowdsourced data in the absence of a “gold standard” and document the emergence of a hidden online marketplace where drugs are sold.

    Our findings are among the first to document the consistency of prices per milligram among diverted opioid analgesics. Earlier researchers, referring to heroin and cocaine, noted that “the most striking characteristics of drug prices are their high levels and extreme variability over time and space” [43]. Prescription drug prices in the United States “are affected by numerous variables, including availability, demand, law enforcement investigations, area of the country, and the relationship between the purchaser and the seller” [22]. While we observed the expected relative stability of prices during the 6 months of observation, further information is needed to understand changes over time.

    For the most part, previous research has focused on the prices and purity of illicitly manufactured drugs like heroin and cocaine [13-15,43-47]. In interpreting our results, we suggest three major differences in which prescription drugs may differ from models of illicitly manufactured drugs. First, drugs are considered by economists to be “experience goods” where buyers pay before discovering the quality of the product (such as an amusement park ride or restaurant meal). Pharmaceutical manufacturing controls create a highly uniform product where these variations are not a factor. Second, retail drug transactions for illicitly manufactured drugs often occur with standardized prices, such as “dime bags”. In these scenarios, the seller can modify the purity and weight of the product to maintain their margins. While counterfeit prescription drugs are possible, for the most part this strategy to improve profit margins is largely limited with pharmaceutical drugs. Third, urban markets for illicitly manufactured drugs are highly competitive with many small sellers creating intense competition for lower prices. Under these and other collective pressures, a seller’s reputation for quality is paramount in order to stay in business selling illicitly manufactured drugs [48]. With prescription drugs, however, these three inputs are constrained—the quality is standardized and can be assessed immediately by the buyer. Therefore, the reputation of quality of a seller may be less important than with illicit drugs. Could this create a lower threshold for initiation of selling leftover medications?

    In contrast to illicitly manufactured drugs, the different prescription opioids are nearly perfect “interchangeable goods”, from the economist’s perspective (but perhaps not the pharmaceutical industry’s); it is difficult to distinguish opioids of the same potency such as heroin (diacetylmorphine) and hydromorphone. This means that we cannot look at the prices of any single prescription opioid in isolation, but must also see what is happening with the prices of other opioid molecules. We found only 2 studies that examined street prices for opioid analgesics, neither of which focused on online sources. One study found a 10x linear association between the pharmacy price and the street price of prescription opioid analgesics in Vancouver, British Columbia [49]. A study of drug users in eastern Kentucky suggested that OxyContin may serve as a form of currency, possibly as a proxy for social capital associated with having the means to afford daily use [50].

    More studies have examined Internet pharmacies. These studies concluded that the pharmacies (whether operating legally or illicitly) were found to be rarely used sources for diverted prescription drugs [4,23,51,52]. In a fundamental shift from the itinerant Internet pharmacies of the last decade, recent social network technology has allowed the emergence of online marketplaces that allow sellers to cultivate their reputation by amassing positive feedback while keeping transactions anonymous and difficult to monitor. Often the site’s owners charge a commission or per-transaction fee. Monthly transactions on Silk Road have been estimated at US$1.9 million [40], and drugs, fireworks, and stolen goods are routinely listed. Calls from US senators to shut down the sites have resulted in broad social attention [53], but the sites continue to operate. Using these sites for academic research has been limited [40], but they offer benefits. Online black market data have less opportunity for recall bias because it is possible to quantify actual transactions instead of reports of prices. However, given the moderate difficulty in accessing hidden sites relative to the open Internet, there is likely to be selection bias in terms of who uses these sites. In addition, online black markets are under pressure from authorities and may blink in and out of existence without warning, adding additional difficulty for data collection.

    Websites designed for research-quality data collection via crowdsourcing and data mining are likely to cost less per report than traditional surveys and can be rapidly adapted to collect new information [54]. However, the credibility of law enforcement surveys is perceived to be higher. Other validation sources against which to compare street price data on prescription opioids are limited. With the closing of NDIC and the absence of a gold standard, we relied on triangulating street prices from 3 different sources, with our results showing remarkable consistency and robustness of prices across opioid molecules. In situations where a gold standard is not available and the behavior under question is illicit, crowdsourcing and data mining provide alternative strategies for collecting information.

    Limitations

    There are several limitations of this study that bear mentioning. Others have pointed out the need to take into account bulk purchasing when modeling prices of illicitly manufactured drugs [44]. To address this, StreetRx site users were asked to note if the price they were reporting was a bulk purchase of more than 10 units. Only 10% of those submitting data positively indicated bulk purchasing; however, the question was not mandatory and we cannot exclude the possibility that others did not answer the question for separate reasons. Another limitation of this analysis was the insufficient sample size to analyze fentanyl and tapentadol data. The greatest limitation, however, lies in the sample size and non-contemporaneous data collection. Our research team was not aware of Silk Road until later in the study, but we felt that it was an important data source that was worth documenting. Geographic analyses, or at least controlling for geographic variation, were deemed to be beyond the scope of the present analysis, but an area of future research. We also did not account for the site users’ history of addiction in the analyses, despite basic research suggesting that the point in the progression of addictive disease may influence willingness to pay. We hope to explore these topics in future analyses. Connecting drug prices to behaviors and health outcomes is a direction of future research.

    Finally, we note that the use of equianalgesic ratios in clinical practice should be undertaken with caution, as should the interpretation of our results using these conversion numbers. In this analysis, we do not know if the opioids were diverted for self-medication, euphoria, or preventing withdrawal. The equianalgesic conversion factors were designed with opioid rotation for pain in mind, and the relative desirability for abuse or withdrawal prevention may be different. Various equianalgesic potency tables have been proposed [42,55,56], and the predicted potency conversion factors may reflect some of this variation. However, for the most part the ostensibly free black market was able to accurately predict the relatively pharmacologic potency of opioid molecules.

    Conclusions

    Crowdsourcing and data mining are efficient ways to collect data about street prices in an era of Internet-based social networks. These data can inform pharmacoeconomic modeling and policy analysis, and may shed light on which new controlled pharmaceutical formulations have desirability relative to others when they hit the street. While this study represents an initial foray into collecting systematic economic data for modeling black markets for prescription drugs, the methodology could be extended in the future by connecting the data to health outcomes.

    Acknowledgments

    The authors wish to thank Karen Lowitz for assistance in preparing this manuscript. This research was funded by the RADARS System (Denver, Colorado, USA).

    Conflicts of Interest

    StreetRx is entirely funded by the RADARS System (Denver, Colorado, USA), an independent nonprofit operation of the Rocky Mountain Poison and Drug Center (RMPDC), a division of Denver Health. StreetRx is operated under contract by Epidemico (Boston, Massachusetts, USA), a health data collection and analytics company. Authors of this paper include employees of the RADARS System and Epidemico. All datasets used in this publication are publicly available by contacting the RADARS System.

    References

    1. McDonald DC, Carlson KE. Estimating the prevalence of opioid diversion by "doctor shoppers" in the United States. PLoS One 2013;8(7):e69241 [FREE Full text] [CrossRef] [Medline]
    2. Berge KH, Dillon KR, Sikkink KM, Taylor TK, Lanier WL. Diversion of drugs within health care facilities, a multiple-victim crime: patterns of diversion, scope, consequences, detection, and prevention. Mayo Clin Proc 2012 Jul;87(7):674-682 [FREE Full text] [CrossRef] [Medline]
    3. Rigg KK, Kurtz SP, Surratt HL. Patterns of prescription medication diversion among drug dealers. Drugs (Abingdon Engl) 2012;19(2):144-155 [FREE Full text] [CrossRef] [Medline]
    4. Substance Abuse and Mental Health Services Administration. Results from the 2010 National Survey on Drug Use and Health: Volume I (NSDUH Series H-38A, HHS Publication No. SMA Findings). In: Summary of National Findings. Rockville, MD: Health and Human Services (Office of Applied Studies); 2010:11-4856.
    5. Drug Enforcement Administration (DEA)‚ Department of Justice. Schedules of controlled substances: rescheduling of buprenorphine from schedule V to schedule III. Final rule. Fed Regist 2002 Oct 7;67(194):62354-62370. [Medline]
    6. Nordt C, Stohler R. Low-threshold methadone treatment, heroin price, police activity and incidence of heroin use: the Zurich experience. Int J Drug Policy 2009 Nov;20(6):497-501. [CrossRef] [Medline]
    7. Kleiman MA. Enforcement swamping: A positive-feedback mechanism in rates of illicit activity. Mathematical and Computer Modelling 1993 Jan;17(2):65-75. [CrossRef]
    8. Minron J. The effect of drug prohibition on drug prices: evidence from the markets for cocaine and heroin. Review of Economics and Statistics 2003;85(3):522-530.
    9. Brown G, Silverman L. The retail price of heroin: estimation and applications. Journal of the American Statistical Association 1974;69(347):595-606.
    10. Bretteville-Jensen AL. To legalize or not to legalize? Economic approaches to the decriminalization of drugs. Subst Use Misuse 2006;41(4):555-565. [CrossRef] [Medline]
    11. Bright DA, Ritter A. Retail price as an outcome measure for the effectiveness of drug law enforcement. Int J Drug Policy 2010 Sep;21(5):359-363. [CrossRef] [Medline]
    12. Caulkins JP, Tragler G, Wallner D. Optimal timing of use reduction vs. harm reduction in a drug epidemic model. Int J Drug Policy 2009 Nov;20(6):480-487. [CrossRef] [Medline]
    13. Costa Storti C, De Grauwe P. Globalization and the price decline of illicit drugs. Int J Drug Policy 2009 Jan;20(1):48-61. [CrossRef] [Medline]
    14. Caulkins JP. The global recession's effect on drug demand--diluted by inertia. Int J Drug Policy 2011 Sep;22(5):374-375. [CrossRef] [Medline]
    15. Bretteville-Jensen AL. Illegal drug use and the economic recession--what can we learn from the existing research? Int J Drug Policy 2011 Sep;22(5):353-359. [CrossRef] [Medline]
    16. Bickel WK, Miller ML, Yi R, Kowal BP, Lindquist DM, Pitcock JA. Behavioral and neuroeconomics of drug addiction: competing neural systems and temporal discounting processes. Drug Alcohol Depend 2007 Sep;90 Suppl 1:S85-S91 [FREE Full text] [CrossRef] [Medline]
    17. Bickel WK, Marsch LA. Toward a behavioral economic understanding of drug dependence: delay discounting processes. Addiction 2001 Jan;96(1):73-86. [CrossRef] [Medline]
    18. Comer SD, Zacny JP, Dworkin RH, Turk DC, Bigelow GE, Foltin RW, et al. Core outcome measures for opioid abuse liability laboratory assessment studies in humans: IMMPACT recommendations. Pain 2012 Dec;153(12):2315-2324. [CrossRef] [Medline]
    19. Bickel WK, Yi R, Mueller ET, Jones BA, Christensen DR. The behavioral economics of drug dependence: towards the consilience of economics and behavioral neuroscience. Curr Top Behav Neurosci 2010;3:319-341. [CrossRef] [Medline]
    20. Comer SD, Bickel WK, Yi R, de Wit H, Higgins ST, Wenger GR, et al. Human behavioral pharmacology, past, present, and future: symposium presented at the 50th annual meeting of the Behavioral Pharmacology Society. Behav Pharmacol 2010 Jul;21(4):251-277 [FREE Full text] [CrossRef] [Medline]
    21. Bretteville-Jensen AL. Addiction and discounting. J Health Econ 1999 Aug;18(4):393-407. [Medline]
    22. National Drug Intelligence Center. National Prescription Drug Threat Assessment 2009. Washington, DC: US Department of Justice; 2009.   URL: http://www.justice.gov/archive/ndic/pubs33/33775/33775p.pdf [WebCite Cache]
    23. Inciardi JA, Surratt HL, Cicero TJ, Rosenblum A, Ahwah C, Bailey JE, et al. Prescription drugs purchased through the Internet: who are the end users? Drug Alcohol Depend 2010 Jul 1;110(1-2):21-29 [FREE Full text] [CrossRef] [Medline]
    24. Good BM, Su AI. Crowdsourcing for bioinformatics. Bioinformatics 2013 Aug 15;29(16):1925-1933. [CrossRef] [Medline]
    25. Yu B, Willis M, Sun P, Wang J. Crowdsourcing participatory evaluation of medical pictograms using Amazon Mechanical Turk. J Med Internet Res 2013;15(6):e108 [FREE Full text] [CrossRef] [Medline]
    26. Turner AM, Kirchhoff K, Capurro D. Using crowdsourcing technology for testing multilingual public health promotion materials. J Med Internet Res 2012;14(3):e79 [FREE Full text] [CrossRef] [Medline]
    27. Maclean DL, Heer J. Identifying medical terms in patient-authored text: a crowdsourcing-based approach. J Am Med Inform Assoc 2013 May 5:1. [CrossRef] [Medline]
    28. Shelley W, Lawley R, Robinson DA. Crowd-sourced soil data for Europe. Nature 2013 Apr 18;496(7445):300. [CrossRef] [Medline]
    29. Luengo-Oroz MA, Arranz A, Frean J. Crowdsourcing malaria parasite quantification: an online game for analyzing images of infected thick blood smears. J Med Internet Res 2012;14(6):e167 [FREE Full text] [CrossRef] [Medline]
    30. Ranard BL, Ha YP, Meisel ZF, Asch DA, Hill SS, Becker LB, et al. Crowdsourcing-harnessing the masses to advance health and medicine, a systematic review. J Gen Intern Med 2013 Jul 11:1. [CrossRef] [Medline]
    31. Zhai H, Lingren T, Deleger L, Li Q, Kaiser M, Stoutenborough L, et al. Web 2.0-based crowdsourcing for high-quality gold standard development in clinical natural language processing. J Med Internet Res 2013;15(4):e73 [FREE Full text] [CrossRef] [Medline]
    32. Jarmolowicz DP, Bickel WK, Carter AE, Franck CT, Mueller ET. Using crowdsourcing to examine relations between delay and probability discounting. Behav Processes 2012 Nov;91(3):308-312. [CrossRef] [Medline]
    33. Sonricker Hansen AL, Li A, Joly D, Mekaru S, Brownstein JS. Digital surveillance: a novel approach to monitoring the illegal wildlife trade. PLoS One 2012;7(12):e51156 [FREE Full text] [CrossRef] [Medline]
    34. Chunara R, Andrews JR, Brownstein JS. Social and news media enable estimation of epidemiological patterns early in the 2010 Haitian cholera outbreak. Am J Trop Med Hyg 2012 Jan;86(1):39-45 [FREE Full text] [CrossRef] [Medline]
    35. Chan EH, Brewer TF, Madoff LC, Pollack MP, Sonricker AL, Keller M, et al. Global capacity for emerging infectious disease detection. Proc Natl Acad Sci U S A 2010 Dec 14;107(50):21701-21706 [FREE Full text] [CrossRef] [Medline]
    36. StreetRx.   URL: http://www.streetrx.com/ [accessed 2013-08-13] [WebCite Cache]
    37. Inciardi JA, Surratt HL, Cicero TJ, Kurtz SP, Martin SS, Parrino MW. The "black box" of prescription drug diversion. J Addict Dis 2009 Oct;28(4):332-347 [FREE Full text] [CrossRef] [Medline]
    38. Dart RC, Cicero TJ, Surratt HL, Rosenblum A, Bartelson BB, Adams EH. Assessment of the abuse of tapentadol immediate release: the first 24 months. J Opioid Manag 2012;8(6):395-402. [CrossRef] [Medline]
    39. Silk Road.   URL: http://silkroadvb5piz3r.onion [accessed 2013-08-13] [WebCite Cache]
    40. Christin N. Traveling the Silk Road: A measurement analysis of a large anonymous online marketplace. 2012.   URL: http://arxiv.org/abs/1207.7139 [accessed 2013-08-13] [WebCite Cache]
    41. Mount Gox.   URL: https://www.mtgox.com/ [accessed 2013-08-13] [WebCite Cache]
    42. Dasgupta N, Kramer ED, Zalman MA, Carino S, Smith MY, Haddox JD, et al. Association between non-medical and prescriptive usage of opioids. Drug Alcohol Depend 2006 Apr 28;82(2):135-142. [CrossRef] [Medline]
    43. Caulkins J, Renter P. What price data tell us about drug markets. Journal of Drug Issues 1998;28(3):593-613.
    44. Caulkins JP. What is the average price of an illicit drug? Addiction 1994 Jul;89(7):815-819. [Medline]
    45. Caulkins JP. Price and purity analysis for illicit drug: data and conceptual issues. Drug Alcohol Depend 2007 Sep;90 Suppl 1:S61-S68. [CrossRef] [Medline]
    46. Bach PB, Lantos J. Methadone dosing, heroin affordability, and the severity of addiction. Am J Public Health 1999 May;89(5):662-665. [Medline]
    47. Wood E, Tyndall MW, Spittal PM, Li K, Anis AH, Hogg RS, et al. Impact of supply-side policies for control of illicit drugs in the face of the AIDS and overdose epidemics: investigation of a massive heroin seizure. CMAJ 2003 Jan 21;168(2):165-169 [FREE Full text] [Medline]
    48. Caulkins J, Feichtinger G, Haunschmied J, Tragler G. Quality cycles and the strategic manipulation of value. Operations Research 2006;54(4):666-677.
    49. Sajan A, Corneil T, Grzybowski S. The street value of prescription drugs. CMAJ 1998 Jul 28;159(2):139-142 [FREE Full text] [Medline]
    50. Jonas AB, Young AM, Oser CB, Leukefeld CG, Havens JR. OxyContin as currency: OxyContin use and increased social capital among rural Appalachian drug users. Soc Sci Med 2012 May;74(10):1602-1609. [CrossRef] [Medline]
    51. Cicero TJ, Shores CN, Paradis AG, Ellis MS. Source of drugs for prescription opioid analgesic abusers: a role for the Internet? Pain Med 2008 Sep;9(6):718-723. [CrossRef] [Medline]
    52. Substance Abuse and Mental Health Services Administration. Results from the 2011 National Survey on Drug Use and Health: Volume I. In: Summary of National Findings (NSDUH Series H-38A, HHS Publication No. SMA Findings). Rockville, MD: Health and Human Services (Office of Applied Studies); 2011:12-4856.
    53. Chen A. The Underground Website Where You Can Buy Any Drug Imaginable. 2011.   URL: http://kotaku.com/5805928/the-underground-website-where-you-can-buy-any-drug-imaginable [accessed 2013-08-13] [WebCite Cache]
    54. Brownstein JS, Freifeld CC, Madoff LC. Digital disease detection--harnessing the Web for public health surveillance. N Engl J Med 2009 May 21;360(21):2153-2157 [FREE Full text] [CrossRef] [Medline]
    55. Haffey F, Brady RR, Maxwell S. A comparison of the reliability of smartphone apps for opioid conversion. Drug Saf 2013 Feb;36(2):111-117. [CrossRef] [Medline]
    56. Goodman LS, Gilman A, Brunton LL. Goodman & Gilman's manual of pharmacology and therapeutics. New York: McGraw-Hill Medical; 2008.


    Abbreviations

    IRB: Institutional Review Board
    NDIC: National Drug Intelligence Center
    RADARS: Researched Abuse, Diversion, and Addiction-Related Surveillance System


    Edited by G Eysenbach; submitted 01.07.13; peer-reviewed by S Schnoll, Q Zhang, G Gilmore, D Ciccarone, J Zacny; comments to author 29.07.13; accepted 09.08.13; published 16.08.13

    ©Nabarun Dasgupta, Clark Freifeld, John S Brownstein, Christopher Mark Menone, Hilary L Surratt, Luke Poppish, Jody L Green, Eric J Lavonas, Richard C Dart. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 16.08.2013.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.