Published on 21.01.19 in Vol 21, No 1 (2019): January
Preprints (earlier versions) of this paper are available at http://preprints.jmir.org/preprint/11291, first published Jun 26, 2018.
Assessing the Impact of a Social Marketing Campaign on Program Outcomes for Users of an Internet-Based Testing Service for Sexually Transmitted and Blood-Borne Infections: Observational Study
Background: While social marketing (SM) campaigns can be effective in increasing testing for sexually transmitted and blood-borne infections (STBBIs), they are seldom rigorously evaluated and often rely on process measures (eg, Web-based ad click-throughs). With Web-based campaigns for internet-based health services, there is a potential to connect campaign process measures to program outcomes, permitting the assessment of venue-specific yield based on health outcomes (eg, click-throughs per test).
Objective: This study aims to evaluate the impact of an SM campaign by the promotional venue on use and diagnostic test results of the internet-based STBBI testing service GetCheckedOnline.com (GCO).
Methods: Through GCO, clients create an account using an access code, complete a risk assessment, print a lab form, submit specimens at a lab, and get results online or by phone. From April to August 2015, a campaign promoted GCO to gay, bisexual, and other men who have sex with men in Vancouver, Canada. The campaign highlighted GCO’s convenience in 3 types of promotional venues—location advertisements in print or video displayed in gay venues or events, ads on a queer news website, and ads on geosocial websites and apps. Where feasible, individuals were tracked from campaign exposures to account creation and testing using venue-specific GCO access codes. In addition, Web-based ads were linked to alternate versions of the campaign website, which used URLs with embedded access codes to connect ad exposure to account creation. Furthermore, we examined the number of individuals creating GCO accounts, number tested, and cost per account created and test for each venue type.
Results: Over 6 months, 177 people created a GCO account because of the campaign, where 22.0% (39/177) of these completed testing; the overall cost was Can $118 per account created and Can $533 per test. Ads on geosocial websites and apps accounted for 46.9% (83/177) of all accounts; ads on the news website had the lowest testing rate and highest cost per test. We observed variation between different geosocial websites and apps with some ads having high click-through rates yet low GCO account creation rates, and vice versa.
Conclusions: Developing mechanisms to track individuals from Web-based exposure to SM campaigns to outcomes of internet-based health services permits greater evaluation of the yield and cost-effectiveness of different promotional efforts. Web-based ads with high click-through rates may not have a high conversion to service use, the ultimate outcome of SM campaigns.
J Med Internet Res 2019;21(1):e11291
Social marketing (SM) campaigns promoting testing for sexually transmitted and blood-borne infections (STBBIs) can effectively increase the uptake of testing [, ]. However, SM testing campaigns are rarely evaluated rigorously owing to pressures of real-world implementation (eg, evaluation budget and difficulty determining campaign-specific effects in an exposed population) [ , ]. Web-based elements of SM campaigns are often evaluated through monitoring the number of views (impressions) of Web-based ads and comparing the proportions of individuals clicking through to visit (click-through rate) and use a website or service (conversion rate) [ , ]; this information is used to identify promotional venues with higher yield, allowing redirection of efforts to optimize campaign reach and inform future campaigns [ ].
More robust evaluations of SM campaigns are possible for campaigns promoting internet-based health services, where users are tracked through service progression. If designed appropriately, campaign evaluations can follow individuals from their initial campaign ad view through to their program outcomes, permitting an assessment of yield of different venues based on actual health outcomes. This paper aims to describe the results of using such a design to evaluate the impact of an SM campaign on increasing the uptake of GetCheckedOnline.com (GCO) , an internet-based STBBI testing service in British Columbia (BC), Canada.
GCO is an internet-based testing service for STBBIs developed by the BC Centre for Disease Control (BCCDC), with a goal of overcoming existing testing barriers among populations with high rates of infection. We have previously published a full description of the GCO program . In brief, users go through the following 5 steps to test through GCO: (1) create an account; (2) complete a risk assessment; (3) print a laboratory requisition form; (4) provide specimens in-person at a private lab (with testing for HIV, syphilis, hepatitis C, chlamydia, and gonorrhea); and (5) receive results online if negative, or by phone if positive or indeterminate. GCO accounts are created by entering an access code on the home page unique to a specific promotion strategy or venue. In addition, individuals can be invited to use GCO by emails with a link to the account creation page.
GCO was launched in 2014, initially targeting gay, bisexual, and other men who have sex with men (GBMSM) in the Vancouver region. Most GBMSM in BC regularly test, with 64.57% (1179/1826) and 62.28% (999/1604) reporting sexually transmitted infection and HIV testing, respectively, in the past year . However, many GBMSM report delaying testing owing to barriers, including privacy concerns or inability to access clinics [ ]. In formative research, GBMSM found GCO acceptable with high intention to use, particularly among men facing testing barriers, perceiving benefits, including greater privacy, convenience, control over testing, and not needing to see a health care provider [ , ].
The SM campaign aimed to increase the awareness and uptake of GCO among GBMSM. The BCCDC partnered with the Health Initiative for Men, a community-based gay men’s health organization, which led the development and implementation of the campaign in consultation with an advisory committee of GBMSM, sexual health nurses, and a small convenience sample of Health Initiative for Men clinic clients. The campaign focused on promoting the convenience of GCO, aiming to reach GBMSM avoiding or delaying testing because of access-related barriers (eg, wait-times for appointments). The campaign concept () was “Some things just make sense online,” designed to use humor based on popular social media sites to motivate viewers to visit the JustMakesSense (JMS) campaign website [ ] which emphasized the convenience and confidentiality of the service. Campaign materials included a website, videos, Web-based ads, and print media.
Tracking Program Outcomes by the Promotional Venue
The campaign ran from April to August 2015. We used 3 venue types for promotion, each having a unique route to account creation, permitting us to track testing outcomes (and ). Location ads included the JMS website address and an access code unique to each location (ie, gay bars and clubs; sex on premises venues; community spaces; businesses; and a lesbian, gay, bisexual, and transgender film festival). Codes were short, easy-to-remember phrases, such as “TestNow” or “TestOnline,” displayed on videos, posters, or postcards (the latter could be taken home by individuals). On the JMS website, visitors entered an access code and proceeded to the account creation page on GCO. Visitors without a code could request an email invitation; visitors were not asked whether or where they had seen the campaign ( ). In addition, we used 2 types of Web-based promotional venues—advertising in a lesbian, gay, bisexual and transgender news website, and advertising on geosocial websites and apps used by GBMSM to find sex partners (Grindr, Jack’d, Manhunt, Squirt, and Scruff). To track testing outcomes for each Web-based venue, each post or ad contained a link to a unique, alternate copy of the JMS website. From each alternate site, visitors proceeded to the GCO account creation page by clicking a link containing an embedded access code unique to each Web-based venue, which could then be associated with each account created.
Our primary outcome was the number of individuals creating GCO accounts by promotional venue type; secondary outcomes included the number of individuals tested, and costs per account created and individual tested. We collected available data from website or app vendors on impressions and click-through rates and extracted GCO program data. For each type of promotional venue, we calculated the number of GCO accounts created and proportion completing testing. For Web-based venues, we described the number of campaign impressions and click- throughs to alternate JMS campaign websites. Furthermore, we described the number of visitors creating accounts through requesting an invitation on the campaign website and their testing outcomes.
Our analysis was conducted under a program evaluation mandate using data routinely collected by BCCDC or through contracts with Web-based ad vendors. The use of individual-level GCO program data is permitted for evaluation under the terms of service agreed to by all GCO users.
Overall, 177 individuals created a GCO account because of the campaign, and 22.0% (39/177) of these completed testing; all results were negative (). The highest number of accounts was from individuals viewing campaign images on geosocial apps (83/177, 46.9%) followed by individuals requesting an invitation from the campaign website (52/177, 29.4%), location ads (21/177, 11.9%), and a news website (20/177, 11.3%). The completion of testing showed little variation across venues, except the news website (1/20, 5%). We spent Can $20,801 on promotion; the average cost was Can $118 per account created and Can $533 per test ( ). The costliest approach per account created was geosocial apps (Can $211), followed by Web-based news (Can $105) and location ads (Can $53). Web-based news had the highest cost per test (Can $2104). Over 19 million impressions of the JMS campaign occurred through geosocial apps, with the highest click-through rate on Grindr (0.7%). The highest numbers of accounts were created from ads on Manhunt and Squirt, resulting in low costs per account created (Can $83 and Can $213, respectively).
|Promotional venue||Number exposed to campaign or Number of impressions||Number visited JMS websites (% of click-through)||Number created account (% of visited JustMakesSense site)||Number completed testing (% of accounts created)|
|Requested invitation||N/Aa||N/A||52b||14 (26.92)|
|Location ads||N/A||N/A||21c||5 (23.80)|
|News website||195,120||260 (0.13)||20 (7.69)||1 (5.00)|
|Geosocial apps (all)||19,232,363||41,227 (0.21)||83 (0.20)||19 (22.89)|
|Grindr||3,443,423||24,975 (0.72)||5 (0.02)||2 (40.00)|
|Jack’d||366,744||655 (0.18)||1 (0.15)||0 (0.00)|
|Manhunt||299,284||547 (0.18)||33 (6.03)||11 (33.33)|
|Squirt||1,355,044||1,822 (0.13)||37 (2.03)||6 (16.21)|
|Scruff||13,767,868||13,228 (0.10)||7 (0.05)||0 (0.00)|
|Total||19,427,483||41,487 (0.21)||177 (0.30)||39 (22.03)|
aN/A: not applicable.
bThese individuals were not asked whether they were exposed to the campaign when they requested an invitation; hence we were unable to estimate the denominator (ie, the number exposed to the campaign) and report % values.
cWe were unable to estimate the denominator (ie, the number exposed to location ads) and report % values for this.
|Promotional venue||Cost of promotion (Can $)||Cost per account (Can $)||Cost per test (Can $)|
|Geosocial apps (all)||17,582||211||925|
aN/A: not applicable.
While commonly applied in e-commerce (eg, linking ad exposures to Web-based purchases), this study demonstrated the value of using this evaluation method to understand the effects of campaign ads (in a range of promotional venues) on internet-based health services. For example, just under half of all GCO accounts were created as a result of ads on geosocial apps, where we spent the bulk of our promotional budget. While Grindr had the highest click-through rate and would, therefore, typically be considered a successful promotional venue, Grindr had the lowest proportion of GCO accounts created and a higher cost per account and per test. Conversely, Manhunt and Squirt had lower click-through rates but had higher yield in terms of GCO program outcomes. In addition, we observed the highest account creation rate among individuals exposed to our Web-based news advertisement, although a much lower proportion proceeded to test. These differences in outcomes might be explained by several factors, including the characteristics of GBMSM on these different websites and apps, such as differences in response to the JMS campaign (influencing click-through rates); demographic factors (eg, age and ethnicity); and behavioral risk or testing barriers (influencing account creation and testing rates)—all aspects worthy of further study [, ]. Furthermore, our findings demonstrate that the promotion in physical venues is important and cost-effective, as location ads had the lowest cost per account of all venues. However, we were unable to account for view through conversion, where GBMSM seeing campaign ads may have later requested a GCO invitation on the campaign website (29% of all accounts created).
We did not observe a large uptake in testing as a result of the JMS campaign. The 39 individuals testing through GCO may be “early adopters” of this intervention with the ongoing diffusion of this innovation through GBMSM networks . A shift in the message may also be needed. The feedback from GBMSM and providers following the campaign suggested convenience may not be the best selling point, given the relative availability of STBBI testing services for GBMSM in the Vancouver area (Edwards J, personal communication, November 2016); this may explain why only 1 in 5 men creating accounts tested through GCO, a measure associated with motivation to get tested in our prior evaluations [ ].
In conclusion, this study demonstrates the value of developing mechanisms for tracking individuals from their Web-based exposure to SM campaign ads about an internet-based health service to their program outcomes. In addition, this study reveals that Web-based venues with high click-through rates may not always have a high conversion to service use, which is ultimately the desired outcome of SM campaigns. We are continuing to use venue-specific access codes to evaluate promotional efforts as GCO expands to other communities across BC.
Funding for the campaign and this evaluation was provided by the Canadian Institutes of Health Research (PHE-318068).
MG, TS, DH, and MB conceived the overall evaluation design and worked with MK and JE on design, implementation, and data collection related to the JustMakeSense campaign. TS led the data analysis, and all authors provided input on the data analysis and interpretation. MG and TS led the drafting and revision of the manuscript, with input from all authors.
Conflicts of Interest
- Friedman AL, Kachur RE, Noar SM, McFarlane M. Health Communication and Social Marketing Campaigns for Sexually Transmitted Disease Prevention and Control: What Is the Evidence of their Effectiveness? Sex Transm Dis 2016 Feb;43(2 Suppl 1):S83-101. [CrossRef] [Medline]
- Cao B, Gupta S, Wang J, Hightow-Weidman LB, Muessig KE, Tang W, et al. Social Media Interventions to Promote HIV Testing, Linkage, Adherence, and Retention: Systematic Review and Meta-Analysis. J Med Internet Res 2017 Nov 24;19(11):e394 [FREE Full text] [CrossRef] [Medline]
- Wei C, Herrick A, Raymond HF, Anglemyer A, Gerbase A, Noar SM. Social marketing interventions to increase HIV/STI testing uptake among men who have sex with men and male-to-female transgender women. Cochrane Database Syst Rev 2011(9):CD009337. [CrossRef] [Medline]
- Reiter PL, Katz ML, Bauermeister JA, Shoben AB, Paskett ED, McRee AL. Recruiting Young Gay and Bisexual Men for a Human Papillomavirus Vaccination Intervention Through Social Media: The Effects of Advertisement Content. JMIR Public Health Surveill 2017 Jun 02;3(2):e33 [FREE Full text] [CrossRef] [Medline]
- Sanders A, Robinson C, Taylor SC, Post SD, Goldfarb J, Shi R, et al. Using a Media Campaign to Increase Engagement With a Mobile-Based Youth Smoking Cessation Program. Am J Health Promot 2018 Jun;32(5):1273-1279. [CrossRef] [Medline]
- GetCheckedOnline. URL: https://getcheckedonline.com/Pages/default.aspx [accessed 2018-06-12] [WebCite Cache]
- Gilbert M, Haag D, Hottes TS, Bondyra M, Elliot E, Chabot C, et al. Get Checked… Where? The Development of a Comprehensive, Integrated Internet-Based Testing Program for Sexually Transmitted and Blood-Borne Infections in British Columbia, Canada. JMIR Res Protoc 2016 Sep 20;5(3):e186 [FREE Full text] [CrossRef] [Medline]
- Trussler T, Ham D. Community-based Research Centre for Gay Men's Health. Sex Now 2015 Health Authorities: Selected results from the National Sex Now Survey comparing BC Health Authorities URL: http://cbrc.net/sites/cbrc.net/files/sn_healthauth_en_170727_RGB_0.pdf [WebCite Cache]
- Gilbert M, Hottes TS, Kerr T, Taylor D, Fairley CK, Lester R, et al. Factors associated with intention to use internet-based testing for sexually transmitted infections among men who have sex with men. J Med Internet Res 2013;15(11):e254 [FREE Full text] [CrossRef] [Medline]
- Hottes TS, Farrell J, Bondyra M, Haag D, Shoveller J, Gilbert M. Internet-based HIV and sexually transmitted infection testing in British Columbia, Canada: opinions and expectations of prospective clients. J Med Internet Res 2012;14(2):e41 [FREE Full text] [CrossRef] [Medline]
- Health Initiative for Men. Just Makes Sense campaign website URL: http://justmakessense.ca/ [accessed 2017-12-27] [WebCite Cache]
- Sun CJ, Sutfin E, Bachmann LH, Stowers J, Rhodes SD. Comparing men who have sex with men and transgender women who use Grindr, other similar social and sexual networking apps, or no social and sexual networking apps: Implications for recruitment and health promotion. J AIDS Clin Res 2018;9(2):757 [FREE Full text] [CrossRef] [Medline]
- Badal HJ, Stryker JE, DeLuca N, Purcell DW. Swipe Right: Dating Website and App Use Among Men Who Have Sex With Men. AIDS Behav 2018 Apr;22(4):1265-1272. [CrossRef] [Medline]
- Haider M, Kreps GL. Forty years of diffusion of innovations: utility and value in public health. J Health Commun 2004;9 Suppl 1:3-11. [CrossRef] [Medline]
- Gilbert M, Salway T, Haag D, Fairley CK, Wong J, Grennan T, et al. Use of GetCheckedOnline, a Comprehensive Web-based Testing Service for Sexually Transmitted and Blood-Borne Infections. J Med Internet Res 2017 Dec 20;19(3):e81 [FREE Full text] [CrossRef] [Medline]
|BC: British Columbia|
|BCCDC: British Columbia Centre for Disease Control|
|GBMSM: gay, bisexual and other men who have sex with men|
|SM: Social marketing|
|STBBI: sexually transmitted and blood-borne infection|
Edited by G Eysenbach; submitted 26.06.18; peer-reviewed by KA Nguyen, SM Goodreau; comments to author 29.08.18; revised version received 12.09.18; accepted 14.09.18; published 21.01.19
©Mark Gilbert, Travis Salway, Devon Haag, Michael Kwag, Joshua Edward, Mark Bondyra, Joseph Cox, Trevor A Hart, Daniel Grace, Troy Grennan, Gina Ogilvie, Jean Shoveller. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 21.01.2019.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.