This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.
Mobile messaging interventions have been shown to improve outcomes across a number of mental health and health-related conditions, but there are still significant gaps in our knowledge of how to construct and deliver the most effective brief messaging interventions. Little is known about the ways in which subtle linguistic variations in message content can affect user receptivity and preferences.
The aim of this study was to determine whether any global messaging preferences existed for different types of language content, and how certain characteristics moderate those preferences, in an effort to inform the development of mobile messaging interventions.
This study examined user preferences for messages within 22 content groupings. Groupings were presented online in dyads of short messages that were identical in their subject matter, but structurally or linguistically varied. Participants were 277 individuals residing in the United States who were recruited and compensated through Amazon’s Mechanical Turk (MTurk) system. Participants were instructed to select the message in each dyad that they would prefer to receive to help them achieve a personal goal of their choosing.
Results indicate global preferences of more than 75% of subjects for certain types of messages, such as those that were grammatically correct, free of
The results indicate that individuals are sensitive to variations in the linguistic content of text messages designed to help them achieve a personal goal and, in some cases, have clear preferences for one type of message over another. Global preferences were indicated for messages that contained accurate spelling and grammar, as well as messages that emphasize the positive over the negative. Research implications and a guide for developing short messages for goal-directed behaviors are presented in this paper.
Over the past decade, mental health researchers have sought to harness popular contemporary technologies, such as computers and mobile phones, in order to develop effective interventions for a range of medical and behavioral problems. The widespread availability and real-time potential of mobile phone-based short message service (SMS) has made SMS interventions an attractive and promising subject of investigation within this area. Numerous studies have shown that SMS interventions can improve outcomes across a variety of physical and mental health disorders [
The content of SMS interventions has typically been based upon prevailing global behavior change theories, such as the transtheoretical model of behavior change, social cognitive theory [
The tone and structure of a message can have an impact on user receptivity and engagement in an intervention, as each point of contact is an opportunity to engage the end user. A few pioneering studies have examined how message framing impacts intervention outcomes or adherence to interventions. For example, Bickmore and colleagues [
There is extensive literature on the benefits of tailoring computer-based intervention content, preferences, and feedback to individual users for health outcomes across conditions [
Within the general intervention development field, several development studies have used focus groups and post-pilot interviewing to examine preferences for certain types of messages. For example, participants in an SMS intervention to promote weight loss disliked the inclusion of
A useful, cost-effective method for collecting this information is rapid and iterative user preference or beta testing using quantitative methods to combat the limitations of qualitative testing. These methods have been used often in consumer research to compile data on user engagement [
This study examined preferences for a range of text messages designed to foster goal-directed behaviors. Text messages were displayed in mirrored dyads to present participants with variations in syntax and language, tone, locus of authority, and grammatical person. The aim of this study was to determine whether any global messaging preferences existed and how certain characteristics moderate those preferences in an effort to inform the development of mobile messaging interventions. In addition, we employed iterative design techniques to assess how subtle changes in messages affected preferences from one sample to the next. This study was approved by the New York State Psychiatric Institute Institutional Review Board (NYPSI IRB) and was part of the pilot intervention development work for a mobile adaptive alcohol intervention.
Participants were recruited online through Amazon.com, Inc.’s online labor market, Amazon Mechanical Turk (MTurk). MTurk is a communication platform through which
MTurk worker qualifications for this study included a HIT approval rate of 95% or greater out of at least 500 completed HITs. This ensured a sample of workers whose work on previous HITs had been consistently deemed acceptable by other requesters, as well as a sample who demonstrated an appropriate degree of computer and Internet literacy. The subject pool was further limited to participants who were located in the United States. Workers who met these qualifications could view our HIT, titled
For the purposes of maintaining anonymity, we could not link the survey to the participants’ MTurk accounts, but included several a priori validity checks for anonymous survey research in both the survey and our MTurk requester account. These validity checks were included in accordance with the Checklist for Reporting Results of Internet E-Surveys (CHERRIES) [
In total, 452 participants took one of four message preference surveys. Of those, 98 were not included in this paper because they were not located in the United States. These participants were primarily located in India and will be discussed in another paper. Of the 354 US participants, 277 were included in the final sample: 58 were excluded due to conflicting responses to identical but counterbalanced items, 9 due to missing or illegible goals, and 10 due to survey completion in under 6 minutes.
The assessment contained approximately 90 items, which were presented in groups of approximately 8 items per screen. Participants were asked to supply a personal goal they would like to achieve and to choose one of two messages in each dyad that they would prefer to receive to help them achieve that goal. These goals did not have to be health related. Participants were told that their goal could be anything from exercise to flossing more to being more assertive, and that there were no wrong goals. There were approximately 70 message dyads in 22 groupings. Each grouping typically consisted of three dyads. Message dyads were based primarily on pre-existing motivational and behavior change content and linguistic differences in message presentation derived from previous messaging studies, public health messaging campaigns, and our own experience writing messages. Based on these existing messages, we developed a corresponding, mirrored message to test a specific preference. For example, if a message included the word
Message groupings.
Organizing Principle | Dyad Grouping | Grouping Description | Dyad Example | Dyads/ Grouping |
|
||||
|
|
|||
|
|
Smiley Emoticon messages contain a smiley face to make the content gain-framed. | Don’t give up :-) | 3 |
Sad Emoticon messages contain a sad-face to make the content loss-framed. | Don’t give up :-( | |||
|
||||
|
Benefit-Oriented messages consist of language that is gain-framed. | Close your eyes – imagine the benefits of changing. | 3 | |
Consequence-Oriented messages consist of language that is loss-framed. | Close your eyes – imagine the consequences if you don’t change. | |||
|
||||
|
|
|||
|
|
Coaching messages contain a direction or recommendation with positively framed emotional emphasis. | You’ve been doing great, don’t quit now. | 3 |
Uncoached Direction messages contain a direction or recommendation with no additional emphasis. | The most important thing you can do to reach your goal is not give up. | |||
|
||||
|
|
|||
|
|
Implementation Intention messages consist of an if-then plan to trigger a specific action. | If I start to get down on myself, I will think of all my previous successes. | 3 |
General Goal messages consist of an open-ended, nonspecific if-then plan. | If I start to get down on myself, I will do something to make me feel better. | |||
|
||||
|
|
|||
|
|
Intrinsic Locus of Control messages emphasize an internal locus of control over goal attainment. | You are responsible when you don’t meet your goal. | 4 |
Extrinsic Locus of Control messages emphasize the degree to which external factors influence goal attainment. | Many different aspects of your environment play a role when you don’t meet your goal. | |||
|
||||
|
|
|||
|
|
Correct Grammar messages contain no grammatical errors. | If you accept where you are now, you’re way ahead of the pack. | 3 |
Grammatical Error messages contain grammatical errors. | If you accept where you are now you’re way ahead of the pack. | |||
|
||||
|
|
u have changed b4, u can meet ur goals today. b who u r. | 3 | |
Non- |
You have changed before, you can meet your goals today. Be who you are. | |||
|
||||
|
|
|||
|
|
Single Punctuation messages utilize only a single punctuation mark between phrases or clauses. | Reinvent yourself! | 4 |
Multiple Punctuation messages utilize multiple punctuation marks between phrases or clauses for emphasis. | Reinvent yourself!!! | |||
|
||||
|
Smiley Emoticon messages contain a smiley face to enhance a friendly or positive tone. | You are on the right track :-) just keep going! | 3 | |
No Emoticon messages contain the same language as their Smiley Emoticon counterparts, but do not include an emoticon. | You are on the right track – just keep going! | |||
|
||||
|
CAPS Emphasis messages contain at least one world that is spelled in all capital letters for emphasis. | When it comes to the negative consequences of a bad habit, you are NOT the exception. | 4 | |
No Visible Emphasis messages do not include any all-caps words. | When it comes to the negative consequences of a bad habit, you are not the exception. | |||
|
||||
|
“ |
|||
|
|
“I” Statement messages employ a singular first person point of view. | Changing can be hard: I promise it will get better. | 4 |
“We” Statement messages employ a plural first person (or collectivist) point of view. | Changing can be hard: we promise it will get better. | |||
“ |
||||
|
“You” Statement messages employ a singular second person point of view. | Your past should motivate you to change – not paralyze you! |
|
|
“We” Statement messages employ a plural first person (or collectivist) point of view. | Our pasts should motivate us to change – not paralyze us! | |||
|
||||
|
Male Quote messages consist of a quote from a famous man. | “When it is darkest, men see the stars.” |
2 | |
Female Quote messages consist of a quote from a famous woman. | “I like the night. Without the dark, we’d never see the stars.” |
|||
|
||||
|
Cited messages refer to a source/sources of the information presented. | Studies show that simply visualizing your future actions makes them more likely to come true! | 3 | |
Uncited messages provide no point of reference for the information presented. | Simply visualizing your future actions makes them more likely to come true! | |||
|
||||
|
|
|||
|
|
Direction messages express a command. | Think about what you will lose if you give up on your goals. | 3 |
Passive messages express a suggestion in a passive or non-urgent tone. | It could be helpful to think about what you will lose if you give up on your goals. | |||
|
||||
|
Statement messages utilize declarative language. | Committing to your goals today will help you in the long-run. | 4 | |
Question messages utilize interrogative language. | How will committing to your goals today help you in the long-run? | |||
|
||||
|
Aggression messages utilize a confrontational or shaming tone. | Do you seriously think that blaming others will help you change for the better? | 3 | |
Nonaggression messages utilize a non-confrontational tone. | Blaming others probably won’t help you change for the better. | |||
|
||||
|
Polite messages include words such as |
Please text us to let us know if you received this message. | 2 | |
Non-Polite messages do not include words such as |
Text us to let us know if you received this message. | |||
|
||||
|
Directive messages contain an imperative statement within the context of a time frame. | Call a friend to help you feel better as soon as you have a free moment. | 3 | |
Nondirective Statement messages offer suggestions with no direction or time-sensitive context. | Going out with friends is a good idea to help you feel better. | |||
|
||||
|
Humor messages include a joke or playful tone to suggest levity. | Why did the chicken cross the road? Because it knew that action creates change. | 2 | |
Gravity messages are serious in tone and do not contain playful or jocular language. | Action creates change. | |||
|
||||
|
|
|||
|
|
Metaphor messages contain symbolic imagery. | When you reach the end of your rope, tie a knot and hang on. | 5 |
Literal messages present content in plain terms. | When you feel like giving up, keep going until it passes. | |||
|
||||
|
|
|||
|
|
Short messages contain as little content as possible to convey meaning. | Your actions define you. | 3 |
Long messages are designed to convey additional meaning. | Your actions define you: the world looks at you differently when you act differently. |
The survey was published on MTurk a total of 4 times. After the data from the first 2 survey publications was downloaded and analyzed, a number of message dyads and groupings were removed if there appeared to be a clear consensus in preference among participants (eg, the Smiley Emoticon vs Sad Emoticon grouping). New dyads and categories were then added to the survey for publication on MTurk the third and fourth time. These revisions account for the differences in the sample size for many of the message groupings examined. The content of 3 message dyads within 2 groupings was slightly altered over the course of the study in order to correct for vagueness, disproportionately weighted language, or language that did not accurately reflect the general profile of a message grouping. Specifically, dyad #3 in the Directive vs Passive grouping and dyads #2 and #3 in the Statistic vs Anecdote grouping were altered. Ultimately, the Statistic vs Anecdote grouping was excluded from the main findings due to the researchers’ concern that the grouping as a whole was unsound. Therefore, only differences based on the alterations made in dyads #2 and 3 in this grouping are reported. Goals were coded into three broad categories based on their subject matter: physical health and well-being, competence and mastery, and personal fulfillment. Goals within these categories were then subcoded into more specific groupings as follows. In the physical health and well-being category, goals were subcoded as weight loss, fitness, nutrition, smoking cessation, sleep health, or personal hygiene goals. In the competence and mastery category, goals were subcoded as professional, academic, financial, or personal goals. In the personal fulfillment category, goals were subcoded as emotional, social, or spiritual goals. Finally, we included process rulers related to one’s self-selected goal such as goal importance, benefits of meeting that goal, and goal efficacy, which have been used in previous research [
A dichotomous variable for preferences within each dyad grouping was created based on a participant’s majority preference for messages in that dyad (ie, at least 2/3 or 3/4 messages chosen). If a category included four messages, individuals who chose two messages of each type (50/50 preference) were removed from analysis. Moderator analysis was conducted using chi-square analysis and comparative percentages are reported.
Demographics are presented in
Demographics (n=277).
Variable | n (%) | |
|
||
|
18-30 | 113 (40.8) |
|
31-40 | 90 (32.5) |
|
41-older | 74 (26.7) |
Gender (% female) |
|
156 (56.5) |
|
||
|
Black | 19 (6.8) |
|
White | 225 (81.1) |
|
Asian | 20 (7.1) |
|
Other | 12 (5.0) |
|
||
|
Hispanic | 22 (8.2) |
|
||
|
High School or GED | 33 (12.1) |
|
Some College | 77 (28.2) |
|
College Degree | 123 (45.1) |
|
Graduate Degree | 40 (14.7) |
Employment status (% employed full-time) | 130 (47.3) | |
Phone plan includes text messaging |
|
266 (96.1) |
Type of text messaging plan (% unlimited) | 197 (71.2) |
Participants generated a variety of personal goals to refer to while choosing their preferred messages. In total, 137 out of 277 participants (49.5%) generated personal goals related to physical health and well-being. Within this broad category, 52 out of 137 participants (38.0%) generated fitness goals (eg, “I want to go to the gym more often”) and 44 out of 137 participants (32.1%) generated weight loss goals. Also, 103 out of 277 (37.2%) participants generated personal goals related to competence and mastery. Within this category, 33 of 103 participants (29.9%) generated financial goals (eg, “I want to save more money this year”), 25 of 103 participants (24.3%) generated professional goals (eg, “I want to advance in my company”), and 18 of 103 participants (17.4%) generated personal mastery goals (eg, “I want to build my own house”). In addition, 33 out of 277 participants (11.9%) generated goals related to personal fulfillment (eg, “I want to communicate more effectively with my spouse” or “I want to have more fun”).
Results of messaging preferences are presented in
Message grouping preferencesa (n=277).
Message Type: |
% | Message Type: |
% | nc |
Smiley emoticon | 97.6 | Sad emoticon | 2.4 | 213b |
Correct grammar | 96.7 | Grammatical errors | 3.3 | 211b |
Non-textese | 95.8 | Textese | 4.2 | 216b |
Locus of control: intrinsic | 93.5 | Locus of control: extrinsic | 6.5 | 46b |
Benefit-oriented | 89.2 | Consequence-oriented | 10.8 | 195b |
Polite | 86.5 | Impolite | 13.5 | 208b |
Nonaggression | 82.9 | Aggression | 17.1 | 269b |
Direction | 82.3 | Passive | 17.7 | 211b |
Statement | 82.0 | Question | 18.0 | 245b |
No Humor | 77.9 | Humor | 22.1 | 188b |
Male quote | 71.9 | Female quote | 28.1 | 146b |
“I” statement | 66.7 | “We” statement | 33.3 | 264b |
Single punctuation | 64.9 | Multiple punctuation | 35.1 | 174b |
“You” statement | 62.9 | “We” statement | 37.1 | 272b |
Uncited | 62.9 | Cited | 37.1 | 272b |
Nondirective | 61.0 | Command | 39.0 | 272b |
Coaching | 57.4 | Uncoached direction | 42.6 | 61 |
Literal | 56.6 | Metaphorical | 43.4 | 272b |
Smiley emoticon | 53.6 | No emoticon | 46.4 | 274 |
CAPS (capitalization) emphasis | 53.1 | No visible emphasis | 46.9 | 213 |
General goal | 52.6 | Implementation intention | 47.4 | 57 |
Short | 51.1 | Long | 48.9 | 272 |
aSee
bIndicates that a message preference is not the result of chance using a non-parametric binomial test to ensure that the difference between groups was greater than a 50% chance (
cThe n applies to both message types.
Because this was an intervention development study, we also created several messages in which we manipulated specific language components from sample to sample. Slightly altering the wording of the passive message within Directive vs Passive dyad #3 from “Every time you feel down, try to change your thoughts to something positive about change” to “Every time you feel down, you might want to try to change your thoughts to something positive about change” resulted in an increase in participants’ overall preferences for the directive message: 127 out of 208 participants (61.1%) preferred the directive message prior to the dyad’s change, but 51 out of 57 participants (89.5%) preferred the directive message post-change. Within a message grouping that examined preferences for Statistics vs Anecdotes, changing the statistic within message dyad #3 from “93% of people who monitor their food intake reduce their calorie intake” to “44% of people who monitor their food intake reduce their calorie intake” caused overall preferences for the statistic to diminish; while 156 out of 212 participants (73.6%) preferred the message containing a statistic in the first version, only 37 out of 60 participants (61.7%) preferred it after the statistic was changed. Conversely, changing the statistics in message dyad #4 from “People who report doing nice things for other people are 44% happier than those who do not” to “People who report doing nice things for other people are 93% happier than those who do not” caused the overall preferences for the statistic to increase: 97 out of 210 participants (46.2%) preferred the message containing a statistic prior to its change, while 37 out of 57 participants (64.9%) preferred the message after the statistic was changed.
We assessed differences in preferences based on several demographic variables, including gender, age, and education. A significant difference existed between male and female participants’ preferences for messages in only one message grouping, with female participants being more likely than male participants to prefer correct grammar to incorrect grammar (χ2
212=5.334,
Significant differences in preference existed between participants with different levels of education for several message groupings as well. Participants with less than a college degree were more likely than participants with a college degree or greater level of education to prefer directions to suggestions (χ2
210=6.061,
We also assessed differences in preference based upon personality or trait variables. Participants who reported being generally sad were significantly more likely than participants who reported being generally happy to prefer commands to nondirective general statements (χ2
267=4.037,
Despite the fact that participants reported radically different goals, the only differences that existed between participants with different higher order goals were that those with personal fulfillment goals were significantly more likely to prefer consequence messages than either those with physical health and well-being or competence and mastery goals (χ2
189 =6.829,
Finally, we assessed differences in preferences based on three process rulers: the participants’ perceived benefits of changing, confidence about their ability to change, and perceived importance of changing. A preference for coaching messages was significantly associated with perceiving greater benefits of change (F1,31=4.33,
To our knowledge, this is the first study to quantitatively examine messaging preferences for a range of message types to help guide text-based mobile intervention development. Results of this study indicate that there are clear user preferences for certain types of message characteristics over others, underscoring the importance of attention to message structure, linguistic content, and overall tone in the development of messages for goal-directed behaviors. This is particularly true of accurate spelling and grammar, as well as messages that emphasize the positive over the negative. While there has been little quantitative research on this topic, the findings of the present study are generally reflective of past qualitative research on messaging development, and are further supported by the Centers for Disease Control and Prevention’s
Participants indicated an overwhelming preference for messages that were accurately spelled and grammatically correct over messages that included
The third and fourth message groupings to examine message syntax compared messages with single punctuation (eg, “.” or “!”) to messages with multiple punctuation marks (eg, “…” or “!!!”) and capitalization of a whole word or phrase to no capitalization (eg, “When it comes to the negative consequences of a bad habit, you are NOT the exception”). Multiple punctuation marks or capitalization can be utilized to add emphasis or to create a pause, and thus constitute visual substitutions for verbal cues. Participants’ preferences for single over multiple punctuation marks was much less pronounced than their preferences for proper spelling and grammar, and there was no clear preference between messages with caps emphasis versus messages with no visible emphasis. However, participants who reported that meeting their goal was very important and would benefit them immensely were more likely to prefer messages with some or all capitalized words for emphasis, and there was a trend toward a significant preference for multiple punctuation marks in the high benefits of change group. This emphasizes that understanding the end user’s state is a crucial component of intervention development.
We examined variations in preferences for two different message groupings that contained emoticons: one that compared messages with a smiley face to identical messages with a sad face, and another that compared messages with a smiley to messages with no emoticons. Of all of the message groupings we examined, preferences were strongest in the smiley versus sad emoticon message grouping, with participants vastly preferring messages with the smiley. This finding resonates with past research that suggests that users vastly prefer positive messages to negative messages, as does our finding that participants generally preferred benefit-oriented to consequence-oriented messages. By contrast, no clear preference existed for messages that contained a smiley versus messages that contained no emoticon. It is possible that some participants found the inclusion of an emoticon too informal within messages designed to help users achieve a personal goal, while others perceived the inclusion of a smiley face as encouraging or rewarding when compared to a message with no emoticon. While virtually all participants seem to prefer a positive to a negative image, the fact that preferences within the two emoticon groupings varied so extremely suggests that the inclusion of an emoticon can communicate very different things depending on the context.
In concordance with previous literature suggesting that visual cues are more effective than text for those with lower need for cognition [
The clear user preferences for statements over questions have particularly interesting implications, as self-evaluative questions are often used in order to integrate motivational interviewing techniques into messaging interventions. Based on our findings, the inclusion of such messages without a fuller understanding of the preferences of the end-user requires some reconsideration. For example, will an individual who drinks too much be motivated to contemplate and evaluate the consequences of drinking simply because a text message asks him or her to do so? Further, questions that do not require interactivity may be disregarded by the individual because they will receive no feedback on their response. On the other hand, Muench and colleagues [
There were clear preferences for more directive language over passive or suggestive language. Moderator analysis revealed that the preference for directive language was especially pronounced in older adults and individuals with more education. This could be a result of several factors, but may simply indicate that these users have been taught to avoid passive or suggestive language when communicating. While participants’ overall preference for directive over passive messages was clear, they were more averse to commands for immediate action (eg, “Do x right now…”), indicating that while individuals may want instruction, they may not want to feel commanded to behave a certain way in the moment.
As mentioned above, there was also a general preference for messages that did not include humor. It could be that humor minimizes one’s struggle to achieve a goal and should be used sparingly and possibly only after an alliance is built. Messages that were presented in first person singular (“I” statements) and second person (“You” statements) were preferred to messages that were presented in first person plural (“We” statements). This may indicate that participants generally prefer to be identified as individuals as opposed to one of a number of people, and prefer to identify the message originator as an individual as well.
We found no clear preferences for several other types of messages within groupings, including short vs long messages. This grouping is of particular interest because intervention developers have often been encouraged to break down messages to their smallest component pieces [
Differential preference analysis was designed to help distinguish preferences among different groups. In our case, overall analyses did not reveal dramatic shifts in preferences, but rather subtle differences between certain groups on certain variables. For example, while younger participants were significantly more likely than older participants to prefer “We” messages to “You” messages, the two groups differed by only 14%. In fact, few moderators shifted one group’s overall preferences for a dyad from one message type to the other. However, the differences reported are significant and future research should examine these subtle variations in message preferences based on these differences. For example, Muench and colleagues [
Interestingly, some individuals preferred more negative messaging. As there is ample evidence that aggression or a demeaning tone decreases long-term adherence (as opposed to constructive negative feedback or consequence-oriented messaging, which can be useful), this finding highlights the downside of preference research in guiding intervention design. Communications that contain negative components like shaming or punishment are contraindicated in interventions to promote long-term behavior change [
It was surprising to us that there were few differences in preferences between those with different scores on process rulers, with the only differences being that those with higher confidence, importance, and benefits to changing preferred caps emphasis in messages to no emphasis and those who saw greater benefits of meeting their goal preferred coaching messages when compared to those who perceived few benefits in meeting their goal. Both messaging types are designed to illicit some sort of emotion in the individual and it could be that adding a positive emotional emphasis is useful for individuals who see greater benefits or importance for change or have greater confidence in their ability to change. While understanding these processes certainly has important implications for message development, it was more striking to us that there were so few differences and results should be interpreted with caution.
Despite the promise of this line of research, there are limitations. The most salient is that preferences do not necessarily translate into improved outcomes [
There were population limitations as well. Namely, we did not restrict the availability of the survey only to people who might be the target of a health intervention, but left it open to a wider population with a broad range of goals—some of which were completely unrelated to health. It is possible that this wider population may have different message preferences than a sample of people who are struggling specifically with a health or mental health problem. Because some of the message groupings were added during later cohorts, there were smaller sample sizes for these preference findings, reducing the strength of the effects. Therefore, future studies should replicate these findings with larger samples. Finally, we limited this analysis to US populations. As a larger part of the study, we are comparing US and Indian populations on messaging preferences, as there is good evidence that linguistic styles and communications differ dramatically between cultures [
When taken together, understanding preferences for intervention presentation may improve engagement, regardless of the content or theory upon which an intervention is based [
Tips for writing intervention messages based on user preferences.
Checklist for Reporting Results of Internet E-Surveys
human intelligence task
Amazon Mechanical Turk
short message service
This study was supported with funding from the National Institute on Alcohol Abuse and Alcoholism (R34 AA021502-01).
Dr Muench consults with several mobile health companies on the development of mobile messaging platforms for health concerns. There were no other conflicts of interest.