This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.
Teens and young adults in the United States are in need of sexual and reproductive health information, as evidenced by elevated rates of sexually transmitted infections (STIs), pregnancy, and births among this population. In-person sexuality education programs are helpful, but they are unlikely to rapidly accommodate teens and young adults in a moment of crisis. Evidence suggests that technologies such as instant messaging (IM) and text messaging may be effective ways to provide teens and young adults with sexual and reproductive health information. In September 2010, Planned Parenthood Federation of America launched a text and IM program designed to provide immediate answers to urgent sexual and reproductive health questions from a reliable and confidential source and to link young people to sexual and reproductive health services if needed.
To assess whether this program is successful in reaching the target population, whether user characteristics vary by mode (IM vs text), and whether mode is associated with reaching individuals with high levels of worry or reducing worry postchat.
Data were collected from prechat and postchat surveys for all IM and text message conversations between September 2010 and August 2011. A bivariate analysis was conducted using chi-square tests for differences in the main covariates by mode of conversation. In the multivariable analysis, logistic regression was used to identify factors that were independently associated with prechat levels of worry and changes in worry postchat.
A total of 32,589 conversations occurred during the program’s first year. The odds of feeling very worried prechat were highest for IM users (adjusted odds ratio [AOR] 1.43, 95% CI 1.20-1.72), users 17 years and younger (AOR 1.62, 95% CI 1.50-1.74), Latino/Hispanic users (AOR 1.36, 95% CI 1.27-1.46), and black users (AOR 1.40, 95% CI 1.30-1.50). After controlling for the study covariates, there was no significant difference in the odds of feeling better (less worried) postchat between IM and text message users. Feeling better postchat was associated with being younger (≤17 years: AOR 1.42, 95% CI 1.17-1.72; 18-24 years: AOR 1.20, 95% CI 1.02-1.42), being Latino/Hispanic (AOR 1.31, 95% CI 1.10-1.55), reporting that the service was very helpful (AOR 3.47, 95% CI 3.24-4.32), and asking about emergency contraception (AOR 1.35, 95% CI 1.13-1.61). The odds of feeling better were lowest for users with questions about STIs (AOR 0.61, 95% CI 0.47-0.78).
The results from the process evaluation suggest that the program was able to provide informational support to vulnerable groups, such as teens and racial minorities, in moments of particular worry. Differences between the IM and text message users reveal that each mode appeals to a different population and that both are necessary to reach a diverse audience.
There are approximately 43 million people aged 15-24 years in the United States, and their need for sexual and reproductive health information is evidenced by the elevated rates of sexually transmitted infections (STIs), pregnancies, and births among this population [
Adverse sexual and reproductive health outcomes are not uniformly distributed among racial and ethnic subgroups in the United States. Recent STI surveillance efforts by the Centers for Disease Control and Prevention (CDC) have shown that rates for chlamydia, gonorrhea, and syphilis among those aged 10-24 years are significantly higher for blacks compared to whites [
The high rates of STIs and unintended pregnancies among teens and young adults in the United States point to an increased need to provide sexual and reproductive health information to this group. Sexuality education programs have traditionally been implemented in schools, health centers, and community settings, yet these venues and methods are lacking in 2 respects. First, not everyone gets sexuality education, and those that do may not receive accurate, effective, or timely sexuality education [
New approaches to sexuality education are needed that can meet the needs of those not receiving adequate sexuality education in schools and communities and may meet needs in a moment of high worry. Technology can play an important role. Two of these types of technologies are instant messaging (IM) and short message service (SMS) text messaging. In 2006, it was estimated that 69 million people in North America used IM [
Existing evidence suggests that the use of IM and SMS text messaging may be an effective way to provide teens and young adults with sexual and reproductive health information [
In light of these considerations, Planned Parenthood Federation of America launched a texting and IM program in September 2010 targeted at teens and young adults aged between 15-24 years who have an urgent sexual or reproductive health need. The goals of this program are to (1) give immediate answers to urgent sexual and reproductive health questions from a reliable and confidential source, and (2) link young people to sexual and reproductive health services if needed. The program additionally aims to reach groups experiencing health disparities, particularly black and Latino/Hispanic teens and young adults.
The program is focused on the following topic areas: emergency contraception, pregnancy tests, abortion, and STI testing. These 4 were chosen for a number of reasons. Topics were chosen based on Web analytics on the Planned Parenthood website, which included frequency of searches of these topic areas on the website, searches utilizing search engines (eg, Google, Yahoo) that led visitors to the website, and volume of page views containing this content. Topic areas were also chosen based on an analysis of questions received via the Ask the Expert email feature on the Planned Parenthood website. Further, the language used in the emails indicated high levels of worry that could be an obstacle to seeking help. Providing immediate personal interaction might alleviate this worry and allow the user to take a positive health-seeking action. Lastly, these issues are often time-sensitive and IM and texting allow a direct connection to users right when the information is needed, thereby filling a gap that sexuality education programs and health centers with limited hours are unable to meet.
The program operates as a national sexual and reproductive health hotline that, as of December 2012, operates from 9 am to midnight (ET) Monday to Thursday, 9 am to 10 pm (ET) on Fridays, 9 am to 5 pm (ET) on Saturdays, and from 2 pm to midnight (ET) on Sundays. By clicking the IM function on the Planned Parenthood Federation of America website or by sending a SMS text message, program users interact with live, trained customer service agents. Agents use a bank of more than 900 scripted responses to provide health information, correct misconceptions, and provide contact information for a Planned Parenthood health center when warranted.
To create the bank of standardized responses, likely questions were identified through archives of questions submitted to Ask the Expert on the Planned Parenthood website, Columbia University’s Go Ask Alice email feature, Ask Dr Cullins on the Planned Parenthood website, and other frequently asked question (FAQ) documents and archives from Planned Parenthood affiliates. Responses to these questions were then gathered from Planned Parenthood pamphlets, websites, and training materials. Further responses were developed by a team of external health writers with expertise in the topic areas and edited by Planned Parenthood Federation of America staff. In the first months of the program, staff actively reviewed transcripts to identify gaps and improve the existing responses. The program maintains an ongoing system for transcript review and content improvement.
Agents are trained on the substantive topics, the use of scripted responses, and how to respond compassionately and appropriately. Agents provide medical information, but do not provide medical advice or diagnoses. They use scripted messages that contain simple counseling content, but are trained to remain on the script and refer users to health centers for more specific or in-depth counseling. Agent performance is monitored and evaluated by staff with advanced sexual health training to ensure high-quality responses that meet the goals of the program. In addition, agents work in a central location and agents that are more skilled are assigned to the task of monitoring conversations as they are happening and providing guidance as needed.
For the pilot phase of the program, Planned Parenthood Federation of America promoted the service on its traditional and mobile websites, which attract 3 million visitors monthly, as well as on 2 MTV television shows:
This paper describes the results from a process evaluation of the first year of the IM and texting program. The process evaluation had 4 main research questions:
Was the program successful in reaching the target population, teens and young adults (15-24 years) and black and Latino/Hispanic teens and young adults?
Did the program reach the target population when they had high levels of worry?
Did user characteristics (sociodemographic, question topic, and level of worry) vary by mode (IM vs texting)?
Was IM or texting more likely to reach individuals when they had high levels of worry and to reduce user-reported worry postconversation?
Data for this process evaluation were collected from September 2010 until August 2011. All SMS text message and IM conversations that occurred during this period were included in the analysis (N=32,589). Data for this analysis came from 3 main sources. The first was a short prechat survey that was offered to all users prior to being connected to an agent. Because of differences between the IM and SMS text service providers, completion of the prechat survey was required for IM users whereas it was offered, but not required, for individuals communicating via texting. The second data source was a postchat survey. Technological limitations of the software program prevented the postchat survey from being offered to all users or from requiring completion by those who were offered it. Lastly, program agents were instructed to fill out a postchat survey for each interaction.
Two service providers, 1 for IM and the other for SMS text messaging, were contracted by Planned Parenthood Federation of America to provide the technological platforms for the program. Each of these providers maintained databases containing data from the prechat, postchat, and agent surveys. Neither database contained a complete list of all variables and cases. Therefore, the 2 datasets were combined to create the final dataset used for this analysis. This dataset was then cleaned to remove duplicate variables and cases.
For IM users, question topic was assessed through the agent survey and the prechat survey. To simplify the user experience for texting users, they were not asked to describe their question topic. The respondent or agent was asked to choose from the following list: abortion, STI testing, pregnancy testing, emergency contraception, and other. To construct the question topic variable, the agent-reported question categories were used as the default. In most cases (23,138/32,589, 70.99%), the responses about the question topic area from the program user and agent were in agreement. An informal qualitative analysis of a sample of transcripts from conversations with discordant responses revealed that the agent responses were more reliable than user responses. In addition, the agent-reported topic was the only data available for texting users. In the case of missing data from the agent survey (3585/32,589, 11.00%), IM user responses were used instead.
To assess whether an individual was in a moment of crisis, users were asked to report how worried they felt about their question. Level of worry was assessed twice: first in the prechat survey and again in the postchat survey. Participants were asked to report if they felt very worried, somewhat worried, or not at all worried. A dichotomous variable was created to determine changes in worry level from prechat to postchat, and users were categorized as either less worried, no change, or increased worry. Less worried could mean that someone went from feeling very worried to somewhat worried, from very worried to not at all worried, or from somewhat worried to not at all worried. Similarly, no change could represent an individual who reported any of the 3 levels of worry prechat, as long as they reported the same level of worry postchat. The no change and increased worry response options were combined due to the small number of users (329/32,589, 5.83%) who reported an increased level of worry postchat.
Users were asked in the postchat survey to rate the helpfulness of the service, with response categories of very helpful, somewhat helpful, somewhat unhelpful, and not at all helpful. In the multivariable models, these categories were collapsed to compare individuals who found the program very helpful to all other users.
Ethical clearance for this analysis was obtained from the Allendale Investigational Review Board. To investigate the main research questions, descriptive statistics for the final sample were run for all variables. Next, a bivariate analysis was conducted using Pearson chi-square test for significance for differences in the main covariates by mode of conversation. In the multivariable analysis, logistic regression was used to identify factors that were independently associated with prechat levels of worry and changes in worry postchat. Models 1 and 3 present the unadjusted odds for the relationships between conversation type and the 2 worry measures. Models 2 and 4 present the adjusted odds ratios, controlling for the other study covariates.
Questions, answer categories, associated variables, and differences by conversation for the study data sources.
Variable domain | Instant messaging | Texting | |
|
|||
|
Gender | Your gender? | Please text your age, sex, and zip code |
|
|
Women | (open ended) |
|
|
Man |
|
|
|
Transgender |
|
|
Age | Your age? | Please text your age, sex, and zip code |
|
|
11 and under | (open ended) |
|
|
12-14 |
|
|
|
15-17 |
|
|
|
18-24 |
|
|
|
25-30 |
|
|
|
31-50 |
|
|
|
51 or older |
|
|
Zip code | Your zip code? | Please text your age, sex, and zip code |
|
|
(open ended) | (open ended) |
|
Race | What best describes your race/ethnicity? | (Collected in postchat survey) |
|
|
White |
|
|
|
Black |
|
|
|
Latino/Hispanic |
|
|
|
American Indian/Alaska Native |
|
|
|
Asian/Pacific Islander |
|
|
|
Biracial/multiracial |
|
|
|
Other |
|
|
Question topic | You may have more than one question, but what is the main thing you want to chat about today? | (Collected in agent survey) |
|
|
Abortion |
|
|
|
STD testing |
|
|
|
Pregnancy tests |
|
|
|
Morning-after pill (emergency contraception) |
|
|
|
Other |
|
|
Prechat worry level | How are you feeling right now? | Before we get started, please tell us how you are feeling right now |
|
|
Very worried | Very worried |
|
|
Somewhat worried | Somewhat worried |
|
|
Not at all worried | Not at all worried |
|
|||
|
Postchat worry level | How are you feeling now? | To help us know how well we’re doing, please answer three questions. First, how are you feeling now? |
|
|
Very worried | Very worried |
|
|
Somewhat worried | Somewhat worried |
|
|
Not at all worried | Not at all worried |
|
Helpfulness | How helpful were we? | How helpful were we? |
|
|
Very helpful | Very helpful |
|
|
Somewhat helpful | Somewhat helpful |
|
|
Somewhat unhelpful | Somewhat unhelpful |
|
|
Not at all helpful | Not at all helpful |
|
Race | (Collected in prechat survey) | What best describes your race/ethnicity? |
|
|
|
American Indian |
|
|
|
Asian/Pac Islander |
|
|
|
Black |
|
|
|
Latino/Hispanic |
|
|
|
Multiracial |
|
|
|
White |
|
|
|
Other |
|
|||
|
Appointment offered | For chatter/texter in participating health center area, did you OFFER an appointment? | For chatter/texter in participating health center area, did you OFFER an appointment? |
|
|
Yes, no, N/A | Yes, no, N/A |
|
Appointment booked | For chatter/texter in participating health center area, did you BOOK an appointment? | For chatter/texter in participating health center area, did you BOOK an appointment? |
|
|
Yes, no, N/A | Yes, no, N/A |
|
Contact info offered | For chatter/texter NOT in participating health center area, did you OFFER local PP contact information? | For chatter/texter NOT in participating health center area, did you OFFER local PP contact information? |
|
|
Yes, no, N/A | Yes, no, N/A |
|
Contact info provided | For chatter/texter NOT in participating health center area, did you PROVIDE local PP contact information? | For chatter/texter NOT in participating health center area, did you PROVIDE local PP contact information? |
|
|
Yes, no, N/A | Yes, no, N/A |
|
Chatter ask for appointment | For chatter/texter NOT in participating health center area, did the chatter ask explicitly for an appointment? | For chatter/texter NOT in participating health center area, did the chatter ask explicitly for an appointment? |
|
|
Yes, no, N/A | Yes, no, N/A |
|
Question topic | What was discussed? Select all that apply. | What was discussed? Select all that apply. |
|
|
Abortion | Abortion |
|
|
Emergency contraception | Emergency contraception |
|
|
Pregnancy testing | Pregnancy testing |
|
|
STI testing | STI testing |
|
|
Other | Other |
Descriptive statistics are displayed in
The overall response rate for the postchat survey was 17.67% (5759/32,589). Among the users who completed the survey, most found this service to be very helpful (61.91%, 3559/5749). Levels of worry were lower postchat. Only 19.33% (1113/5758) reported feeling very worried after interacting with an agent, as opposed to 51.93% (2990/5758) feeling somewhat worried and 28.74% (1655/5758) feeling not at all worried. The variable that was created to capture changes in level of worry revealed that 37.90% (2140/5647) of users reported feeling less worried immediately after the chat had concluded. Although slightly more than half (56.28%, 3178/5647) reported no change, a small group (5.83%, 329/5647) reported an increase in their level of worry postchat.
Using Pearson chi-square test for significance, statistical differences between texting and IM users were found for most of the major covariates (
Overall, there was a statistically significant difference in which texting users appeared to be slightly less worried than IM users (
As shown in
Models 3 and 4 use the change in worry variable as the outcome, and the coefficients estimate the odds of feeling better (ie, less worried) postchat compared to reporting the same or increased level of worry. These models only pertain to the 17.67% of the total sample that completed the postchat survey (5759/32,589). The UOR for the relationship between conversation type and postchat worry revealed no significant difference between IM and texting users in the likelihood of feeling better postchat (UOR 0.99, 95% CI 0.84-1.16). After controlling for the other study covariates, this finding remained nonsignificant. However, there were significant relationships between other study covariates and changes in worry level postchat. Compared to the oldest users, individuals in the younger age groups were more likely to report lower levels of worry postchat (≤17 years: AOR 1.42, 95% CI 1.17-1.72; 18-24 years: AOR 1.20, 95% CI 1.02-1.42). Feeling better postchat was associated with users reporting being Latino/Hispanic (AOR 1.31, 95% CI 1.10-1.55) and reporting that the service was very helpful (AOR 3.47, 95% CI 3.24-4.32). Using abortion as a reference, users with questions about emergency contraception were most likely to report feeling less worried postchat (AOR 1.35, 95% CI 1.13-1.61). Conversely, the odds of feeling better were lowest for users with questions about STIs compared to abortion questions (AOR 0.61, 95% CI 0.47-0.78).
Characteristics of program users and conversations by mode of conversation, instant messaging (IM), and texting.
Variables | All users |
IM |
Texting |
|
||||
|
|
|
|
|
|
|
|
|
|
Abortion | 13,617 | 44.65 | 13,020 | 46.61 | 597 | 23.27 | <.001 |
|
Pregnancy testing | 5284 | 17.33 | 4761 | 17.04 | 523 | 20.39 |
|
|
Emergency contraception | 5292 | 17.35 | 4844 | 17.34 | 448 | 17.47 |
|
|
STI testing | 2810 | 9.21 | 2387 | 8.55 | 423 | 16.49 |
|
|
Other | 3495 | 11.46 | 2921 | 10.46 | 574 | 22.38 |
|
|
|
|
|
|
|
|
|
|
|
Female | 28,575 | 89.29 | 24,959 | 89.81 | 3616 | 85.85 | <.001 |
|
Male | 3427 | 10.71 | 2831 | 10.19 | 596 | 14.15 |
|
|
|
|
|
|
|
|
|
|
|
White | 12,119 | 46.17 | 11,789 | 46.30 | 330 | 41.93 | <.001 |
|
Black | 4429 | 16.87 | 4320 | 16.97 | 109 | 13.85 |
|
|
Latino/Hispanic | 4881 | 18.59 | 4687 | 18.41 | 194 | 24.65 |
|
|
Other | 4821 | 18.37 | 4667 | 18.33 | 154 | 19.57 |
|
|
|
|
|
|
|
|
|
|
|
≤17 | 7500 | 23.30 | 5839 | 20.90 | 1661 | 39.02 | <.001 |
|
18-24 | 16,485 | 51.20 | 14,361 | 51.40 | 2124 | 49.89 |
|
|
≥25 | 8210 | 25.50 | 7738 | 27.70 | 472 | 11.09 |
|
|
|
|
|
|
|
|
|
|
|
Very worried | 13,365 | 43.22 | 12,208 | 43.70 | 1157 | 38.79 | <.001 |
|
Somewhat worried | 13,953 | 45.12 | 12,590 | 45.06 | 1363 | 45.69 |
|
|
Not at all worried | 3603 | 11.65 | 3140 | 11.24 | 463 | 15.52 |
|
|
|
|
|
|
|
|
|
|
|
Yes | 13,224 | 54.57 | 11,993 | 54.48 | 1231 | 55.50 | .40 |
|
No | 11,007 | 45.43 | 10,020 | 45.52 | 987 | 44.50 |
|
|
|
|
|
|
|
|
|
|
|
Response rate | 5759 | 17.67 | 4956 | 17.74 | 802 | 17.25 | .42 |
|
|
|
|
|
|
|
|
|
|
Very worried | 1113 | 19.33 | 955 | 19.27 | 158 | 19.70 | <.001 |
|
Somewhat worried | 2990 | 51.93 | 2628 | 53.03 | 362 | 45.14 |
|
|
Not at all worried | 1655 | 28.74 | 1373 | 27.70 | 282 | 35.16 |
|
|
|
|
|
|
|
|
|
|
|
Less worried | 2140 | 37.90 | 1876 | 37.85 | 264 | 38.21 | .01 |
|
No change | 3178 | 56.28 | 2808 | 56.66 | 370 | 53.55 |
|
|
More worried | 329 | 5.83 | 272 | 5.49 | 57 | 8.25 |
|
|
|
|
|
|
|
|
|
|
|
Very helpful | 3559 | 61.91 | 2980 | 60.13 | 579 | 73.01 | <.001 |
|
Less than very helpful | 2190 | 38.09 | 1976 | 39.23 | 214 | 26.99 |
|
aAll users: n=30,498; IM: n=27,933; texting: n=2565.
bAll users: n=32,002; IM: n=27,790; texting: n=4212.
cAll users: n=26,250; IM: n=25,463; texting: n=787.
dAll users: n=32,195; IM: n=27,938; texting: n=4257.
eAll users: n=30,921; IM: n=27,938; texting: n=2983.
fAll users: n=24,231; IM: n=22,013, texting: n=2218.
gAll users: n=32,589; IM: n=27,939; texting: n=4650.
hAll users: n=5758; IM: n=4956; texting: n=802.
iAll users: n=5647; IM: n=4956; texting: n=691.
jAll users: n=5749; IM: n=4956; texting: n=793.
Differences in user-reported program helpfulness postchat by question topic (n=5626).
Helpfulness | Question topic area |
|
|||||||||
|
Abortion |
Pregnancy testing |
Emergency contraception |
STI testing |
Other |
|
|||||
|
n | % | n | % | n | % | n | % | n | % |
|
Very helpful | 1339 | 61.76 | 716 | 60.94 | 795 | 70.48 | 314 | 53.31 | 312 | 55.12 | <.001 |
Less than very helpful | 829 | 38.24 | 459 | 39.06 | 333 | 29.52 | 275 | 46.69 | 254 | 44.88 |
|
Unadjusted (UOR) and adjusted (AOR) logistic regression estimates of the relationship between conversation mode and worry measures.
Variables | Odds of reporting | ||||||||
|
Very worried prechata
|
Reduced worry postchatb
|
|||||||
|
UOR | 95% CI | AOR | 95% CI | UOR | 95% CI | AOR | 95% CI | |
|
|
|
|
|
|
|
|
|
|
|
Texting | 1.00 |
|
1.00 |
|
1.00 |
|
1.00 |
|
|
IM | 1.22 | 1.13-1.32 | 1.43 | 1.20-1.72 | 0.99 | 0.84-1.16 | 1.22 | 1.00-1.50 |
|
|
|
|
|
|
|
|
|
|
|
Male |
|
|
1.00 |
|
|
|
1.00 |
|
|
Female |
|
|
1.07 | 0.98-1.16 |
|
|
1.26 | 0.99-1.59 |
|
|
|
|
|
|
|
|
|
|
|
25 and older |
|
|
1.00 |
|
|
|
1.00 |
|
|
18-24 |
|
|
1.13 | 1.07-1.21 |
|
|
1.20 | 1.02-1.42 |
|
17 and younger |
|
|
1.62 | 1.50-1.74 |
|
|
1.42 | 1.17-1.72 |
|
|
|
|
|
|
|
|
|
|
|
White |
|
|
1.00 |
|
|
|
1.00 |
|
|
Black |
|
|
1.40 | 1.30-1.50 |
|
|
1.12 | 0.92-1.35 |
|
Latino/Hispanic |
|
|
1.36 | 1.27-1.46 |
|
|
1.31 | 1.10-1.55 |
|
Other |
|
|
1.34 | 1.25-1.44 |
|
|
1.35 | 1.13-1.61 |
|
|
|
|
|
|
|
|
|
|
|
Abortion |
|
|
1.00 |
|
|
|
1.00 |
|
|
STI testing |
|
|
0.35 | 0.32-0.39 |
|
|
0.61 | 0.47-0.78 |
|
Pregnancy testing |
|
|
0.74 | 0.69-0.80 |
|
|
0.98 | 0.82-1.17 |
|
Emergency contraception |
|
|
0.60 | 0.53-0.61 |
|
|
1.35 | 1.13-1.61 |
|
Other |
|
|
0.40 | 0.36-0.43 |
|
|
0.97 | 0.76-1.23 |
|
|
|
|
|
|
|
|
|
|
|
No |
|
|
|
|
|
|
1.00 |
|
|
Yes |
|
|
|
|
|
|
0.99 | 0.87-1.13 |
|
|
|
|
|
|
|
|
|
|
|
Less than very helpful |
|
|
|
|
|
|
1.00 |
|
|
Very helpful |
|
|
|
|
|
|
3.74 | 3.24-4.32 |
aCoefficients represent the odds of feeling very worried prechat as compared to somewhat/not at all worried.
bCoefficients represent the odds of feeling less worried after using the service as compared to reporting the same or increased worry.
The analysis of the first year of data from the IM and texting program revealed that the program was successful in reaching its target audience, with large portions of users being young (≤24 years), black, and Latino/Hispanic. A large percentage of the study population reported feeling very worried when they initiated the conversation. Because this measure was used as a proxy for whether a user was in a moment of crisis, it is reasonable to assume that the program was also successful in reaching individuals during this vulnerable time. In addition, racial minorities (including black, Latino/Hispanic, and all other groups categorized as other) and users 24 years and younger were more likely to feel very worried when they accessed the program. This may suggest that these groups are in greater need of these types of education and information services. Some evidence from the literature supports this. The 2006-2008 results of the National Survey of Family Growth reported that only 47% of females and 38% of males aged 15-19 years had received information about contraception in high school [
Results from the bivariate analysis indicate that the SMS texting service was more likely to be used by younger users and racial minorities than the IM service. This result may simply reflect technology preferences or access of individuals in these groups. This highlights the importance of offering the program through different modes of technology. Although cell phones provide a greater degree of anonymity in the moment, a text conversation, unless deleted, can be reviewed by others who have access to the phone. On the other hand, evidence of a conversation held through IM is gone from the computer once the window is closed, but the computer itself may be in a less private location and the history of visiting the Planned Parenthood website may be accessible to future users of the computer. Because there are continuing advances in technologies with mobile chat, tablets, etc, it is important to consider the implications of these new technologies and their possible impact on program utilization.
The differences that were found between texting and IM users may also be a by-product of the ways that the texting and IM portions of the program were marketed and promoted. The IM portion was only promoted on existing Planned Parenthood Web properties; therefore, it generally reached individuals who were already seeking information online. Recent research has suggested that there are demographic differences in the types of people who seek out health information online [
Although texting users did appear to be less worried overall, after controlling for the study covariates, there was no difference in the program’s effectiveness in reducing levels of worry postchat for texting versus IM users. This is an encouraging result because it indicates that both conversation modes are equally effective in reducing worry immediately after an individual has interacted with the program. However, because there were differences in the demographic profiles of the IM and texting users, it seems that both modes are needed to effectively reach the target population.
The results of this evaluation indicate that the program was more effective in reducing worry levels for the youngest age group (≤17 years) and for Latino/Hispanics. This result echoes the previous point that these groups may be more in need of this service because of inadequate access to sexual and reproductive health information at home or in school.
Whether or not a user felt better (ie, reduced level of worry) differed greatly by the topic area of their question. As compared to individuals with questions about abortion, the program was more effective in reducing worry when users had questions concerning emergency contraception. On the other hand, the odds of feeling better (ie, reducing worry) postchat were lower for individuals asking about STI testing than for abortion. There are several potential explanations for these differences. It could be that the increased odds of feeling better for individuals with questions about emergency contraception is an indication of a poor understanding of it and its uses in the general population. As such, these individuals may have benefited more from the information they received from the program and were subsequently more relieved. The fact that users with questions about STIs were less likely to report a lower level of worry postchat may be a result of the nature of STI questions. Agents are prohibited from making diagnoses during the conversations, but many individuals present with a list of symptoms and want to know what STI they may have. In these cases, agents are limited to providing information for a health center and advising the user that they will have to wait for several days or weeks after they are tested for results. As a result, users with STI-related questions are likely most frustrated with the lack of information regarding a specific diagnosis.
In an effort to test this theory after initial analysis, Pearson chi-square tests were run to determine if there were significant differences in reporting that the chat was very helpful by question topic area (
This study has several limitations. The IM services attracted far more users than the SMS texting service (IM = 27,939 vs texting = 4650). As discussed previously, this is likely partially because of differences in the way that the text and IM services were advertised. This makes it difficult to assess whether the variations in the characteristics of program users by IM or texting are an artifact of these marketing methods or an indication of underlying preferences of users. Further research is needed to more accurately address this issue. In addition, the overall sample is heavily biased toward IM users.
Technological issues also made it difficult to collect data for this evaluation. Program designers did not want to discourage participating or completing the prechat or postchat survey by presenting lengthy surveys. Therefore, gathered information was limited to basic demographics, program helpfulness, and levels of worry prechat and postchat. The worry variables were meant to capture whether or not an individual was in a moment of crisis. However, this is an imperfect proxy because some individuals could have been very worried about their sexual and reproductive health issues for several weeks. Further, as shown in
Research has shown that response rates for Internet and text-based surveys are notoriously low [
One of the major goals of the program is to increase access to health services. Program designers hypothesize that providing IM and SMS texting services will help build trust between program users and Planned Parenthood, further encouraging users to seek health services at a Planned Parenthood health center when they are needed. However, funding and technology restrictions did not allow for a more robust follow-up process. Therefore, another limitation of this process evaluation is that it only measures user attitudes immediately after interacting with the program and not behavior change. Further research is needed to systematically measure the impact of the program. This type of evaluation could determine whether individuals who used the program were more likely to utilize health care services or enact positive behavior change than those who did not.
The use of Internet and mobile technology is increasingly becoming an integral part of our everyday interactions and activities. This is especially true among teenagers and young adults. If interventions can be developed that reach young people with information and education that helps reduce worry, encourages the use of needed health services, and motivates changes in health behaviors, these technologies could be an important addition to public health practice.
The results from the process evaluation of the first year of Planned Parenthood Federation of America’s IM- and texting-based intervention offer insight into one possibility for the use of Internet and mobile technologies for sexuality education programs. Although the results are unable to describe the program’s effectiveness in affecting behavior change, they do suggest that the program was able to provide informational support to traditionally vulnerable groups, such as teens and racial minorities, in moments of particular worry.
The differences found between the IM and texting users reveal that each mode appeals to a different population of users and that both are necessary to reach a larger audience. Future research is needed to rigorously evaluate the impact of the program, including whether it increased knowledge of sexual and reproductive health issues, increased the use of health care services, and promoted safer sexual behaviors among users.
adjusted odds ratio
frequently asked question
instant messaging
short message service
sexually transmitted infections
unadjusted odds ratio
The Chat/Text program was funded by Anonymous, the Ford Foundation, The David and Lucile Packard Foundation, and The Shelley & Donald Rubin Foundation. Note: Funders played no role in the conduct of the research or preparation of the paper.
None declared.