Background: Little is known about how individuals engage over time with smartphone app interventions and whether this engagement predicts health outcomes.
Objective: In the context of a randomized trial comparing 2 smartphone apps for smoking cessation, this study aimed to determine distinct groups of smartphone app log-in trajectories over a 6-month period, their association with smoking cessation outcomes at 12 months, and baseline user characteristics that predict data-driven trajectory group membership.
Methods: Functional clustering of 182 consecutive days of smoothed log-in data from both arms of a large (N=2415) randomized trial of 2 smartphone apps for smoking cessation (iCanQuit and QuitGuide) was used to identify distinct trajectory groups. Logistic regression was used to determine the association of group membership with the primary outcome of 30-day point prevalence of smoking abstinence at 12 months. Finally, the baseline characteristics associated with group membership were examined using logistic and multinomial logistic regression. The analyses were conducted separately for each app.
Results: For iCanQuit, participants were clustered into 3 groups: “1-week users” (610/1069, 57.06%), “4-week users” (303/1069, 28.34%), and “26-week users” (156/1069, 14.59%). For smoking cessation rates at the 12-month follow-up, compared with 1-week users, 4-week users had 50% higher odds of cessation (30% vs 23%; odds ratio [OR] 1.50, 95% CI 1.05-2.14; P=.03), whereas 26-week users had 397% higher odds (56% vs 23%; OR 4.97, 95% CI 3.31-7.52; P<.001). For QuitGuide, participants were clustered into 2 groups: “1-week users” (695/1064, 65.32%) and “3-week users” (369/1064, 34.68%). The difference in the odds of being abstinent at 12 months for 3-week users versus 1-week users was minimal (23% vs 21%; OR 1.16, 95% CI 0.84-1.62; P=.37). Different baseline characteristics predicted the trajectory group membership for each app.
Conclusions: Patterns of 1-, 3-, and 4-week smartphone app use for smoking cessation may be common in how people engage in digital health interventions. There were significantly higher odds of quitting smoking among 4-week users and especially among 26-week users of the iCanQuit app. To improve study outcomes, strategies for detecting users who disengage early from these interventions (1-week users) and proactively offering them a more intensive intervention could be fruitful.
User engagement in digital behavior change interventions has predicted improved treatment outcomes across a wide variety of domains, including mental health, physical activity, dietary change, weight loss, alcohol use, and smoking cessation [- ]. A central challenge in creating effective digital health behavior change interventions is that a large proportion of users disengage early from these interventions, thereby contributing to low treatment success rates [ , , ]. Given the importance of user engagement, designing strategies to increase engagement has been a priority of digital behavioral health interventions [ , , , , ].
Within the domain of cigarette smoking, smartphone apps for smoking cessation have become a ubiquitous intervention approach for which user engagement can be readily measured. Nearly 500 English language smartphone apps for smoking cessation have been downloaded more than 33 million times since 2012 (R Nelson, SensorTower.com, personal communication, April 15, 2020). Higher user engagement in smartphone interventions for smoking cessation is predictive of cessation outcomes [, - ]. Although this positive association may partly be driven by self-selection bias (or reverse causation), as users have not been randomized to different levels of engagement, a certain level of exposure to the intervention’s active ingredients is logically necessary for successful behavior change [ ]. This logic has led to a body of work focusing on identifying design strategies that promote user engagement. For example, tailoring content to individuals’ characteristics or unique situations, interactivity (eg, through conversational agents) [ ], and credibility have been found across a range of studies using mixed methods to be important for engagement with smoking cessation interventions.
Although smartphone intervention engagement is usually measured by the number of log-ins, little is known about how users engage with smartphone interventions over time and whether those temporal patterns predict higher odds of smoking cessation. In the educational literature, a well-documented finding is that learning new material becomes more effective when it occurs over a longer period . This process, called spaced practice, increases the variability in learning and remembering new information [ ]. The purpose of this study was to determine, in the domain of smartphone apps for smoking cessation, whether user engagement over time leads to improved cessation outcomes.
Smartphone apps for smoking cessation are commonly available for participants to use at will, resulting in high variations in use trajectories over time. For example, some users may follow a trajectory of logging in several times within the first few days of starting an intervention and then never return. Others may follow a trajectory in which they log in consistently and then gradually taper off. Other users may follow a trajectory in which they consistently log in over the course of several months. Conceivably, some groups of individuals might follow unique use trajectories over time that are associated with differential health outcomes. For example, people who log in consistently over the course of many months might have higher cessation rates, because they have consistently benefited from the information and skills presented in the app.
There is a dearth of studies analyzing use trajectories for digital smoking cessation interventions. We are aware of 3 publications for SMS text messaging interventions [- ], 4 publications for website interventions [ , - ], and none for smartphone interventions. Regarding SMS text messaging interventions, a study identified 5 distinct use trajectories of an SMS text messaging–based smoking cessation program over the 6 weeks after quit date, namely high engagement, increasing engagement, rapid decrease, delayed decrease, and low engagement [ ]. The study found that the high engagement and increasing engagement groups were more likely than the other groups to quit smoking over the course of 6 weeks. Within the context of smoking cessation websites, our group conducted a functional clustering analysis of log-in data from both arms of a large (N=2637) randomized trial of 2 website interventions for smoking cessation (WebQuit and Smokefree), with a primary outcome of 30-day point prevalence smoking abstinence at 12 months [ ]. Compared with 1-week WebQuit users, 5- and 52-week users had 57% higher odds (odds ratio [OR] 1.57, 95% CI 1.13-2.17; P=.007) and 124% higher odds (OR 2.24, 95% CI 1.45-3.43; P<.001), respectively, of being abstinent at 12 months. The 5-week use of either website predicted higher odds of quitting smoking, with the highest odds for 52-week WebQuit users. These results suggest that experimental testing strategies to increase digital intervention engagement for 4 more weeks (ie, from 1 week to 5 weeks) would be valuable. Studying distinct groups of use trajectories can help identify which use patterns are beneficial and thereby make recommendations for future program use. These results will help inform the design of more engaging digital interventions for smoking cessation, with the ultimate goal of a higher likelihood of cessation.
If we can identify smartphone intervention use trajectories that predict cessation, understanding the sociodemographic characteristics of individuals who tend to follow more or less successful trajectories is important. Knowing the characteristics of individuals who are likely to have certain engagement patterns might allow researchers and intervention designers to tailor smartphone interventions according to users’ unique baseline characteristics. There is an emerging literature of randomized trial designs that algorithmically use baseline characteristics predictive of treatment outcomes in the design of tailored interventions [, ]. Although studies have found that being female, being older, and having a higher education are generally consistent predictors of greater digital intervention use [ - ], very little is known about the user characteristics that are associated with different patterns of use over time [ , ]. For example, a study found that being female and having higher baseline motivation were associated with more consistent log-in trajectories [ ]. For the WebQuit website intervention, we found that smoking for at least the past 10 years and screening negative for anxiety predicted a 90% higher odds (OR 1.90, 95% CI 1.14-3.14) and a 56% higher odds (OR 1.56, 95% CI 1.06-2.33), respectively, of being a 52-week user (compared with being a 1-week user) [ ]. Regarding smoking cessation smartphone apps, we are aware of no literature on baseline predictors of their use patterns over time.
We recently developed and tested iCanQuit, a smartphone app for smoking cessation based on acceptance and commitment therapy (ACT), a behavioral approach that teaches skills for allowing cravings to smoke to pass without smoking, which is conceptually distinct from the US Clinical Practice Guidelines (USCPG)–based approaches that teach avoidance of urges . In a large 2-arm randomized trial, iCanQuit was compared with QuitGuide, a USCPG-based smartphone app. At the 12-month follow-up, iCanQuit was 1.5 times more efficacious than QuitGuide for smoking cessation among 2415 adults who smoked (36% racial or ethnic minority groups) from all 50 US states [ ]. The iCanQuit study was the first full-scale randomized controlled trial with long-term follow-up to show that a smartphone app was efficacious for smoking cessation.
Using data from the iCanQuit parent randomized trial, this study identified the following: (1) distinct groups of smartphone app log-in trajectories, (2) their association with the 12-month smoking cessation outcome, and (3) baseline sociodemographic user characteristics that are associated with different use trajectory groups. Log-in trajectories (ie, log-ins over time) are a generalizable metric that can be useful for other digital intervention researchers—agnostic of the intervention-specific content contained in any one app. The overall goal of this study is to inform the design of future smartphone health interventions that could be more efficacious by identifying trajectory groups in need of further intervention. To accomplish these aims, in this study we analyzed 182 consecutive days of log-in data from both arms of the large (N=2415), 2-arm randomized trial of iCanQuit versus QuitGuide smartphone app interventions for smoking cessation (NCT02724462).
A total of 2415 individuals were enrolled in the 2-arm iCanQuit randomized controlled trial for smoking cessation, with full protocol details previously described . In brief, a racially and ethnically diverse sample of 2415 adult daily smokers from all 50 US states was randomized 1:1 to receive access to an ACT-based smartphone app (iCanQuit) or a USCPG-based smartphone app (QuitGuide) for smoking cessation. Data for this analysis were from the 2133 individuals who logged into their assigned app at least once and had a complete 182 days of engagement data available.
Eligibility criteria included individuals who (1) were aged ≥18 years; (2) smoked 5 or more cigarettes per day for the past year; (3) wanted to quit smoking within the next 30 days; (4) if concurrently using any other tobacco products, wanted to quit all tobacco products within 30 days; (5) were interested in learning skills to quit smoking and willing to be randomized to either treatment condition; (6) had daily access to their own smartphone; (7) knew how to download smartphone apps; (8) were willing and able to read in English; (9) had never used QuitGuide and not currently using another smoking cessation treatment; (10) had never participated in our prior studies; (11) had no household members already enrolled; (12) were willing to complete outcome surveys, and (13) could provide contact information for themselves and 2 relatives.
Recruitment, Enrollment, and Follow-up
Adults were recruited nationally via Facebook advertisements, a survey sampling company, search engine results, and friend or family referrals. Participants completed an encrypted web-based screening survey and were notified of their eligibility via email. They then clicked on their secured emailed link to the study website where they provided consent and completed the baseline survey. At each enrollment step, the study was presented as a comparison of 2 smartphone apps for smoking cessation.
Participants were randomized (1:1) to iCanQuit or QuitGuide using randomly permuted blocks of sizes of 2, 4, and 6, stratified by smoking frequency (≤20 vs ≥21 cigarettes per day), education (≤high school vs ≥some college), race or ethnicity (minority race or ethnicity vs non-Hispanic White), and depression screening (Center for Epidemiologic Studies Depression score ≤15 vs ≥16) . Random assignments were concealed from participants throughout the trial. The random allocation sequence was generated by a database manager and implemented automatically by the study website. Neither research staff nor study participants had access to the upcoming randomized study arm assignments. In both arms, participants could access their interventions from the moment of randomization and beyond (ie, after the end of the 12-month follow-up period). All participants provided their consent on the web and were compensated with up to US $105 for completing study data collection. The data retention rate was 88% (1886/2133) and differed slightly between arms (90% in QuitGuide vs 87% in iCanQuit; P=.01).
Participants randomized to the iCanQuit arm received access to download the iCanQuit smartphone app (version 1.2.1). iCanQuit intervenes on the ACT-focused processes of acceptance of internal cues to smoke and enacting one’s values that guide smoking cessation . The acceptance component of the app teaches skills to accept physical sensations, emotions, and thoughts that trigger smoking by distancing oneself from thoughts about smoking (“cognitive defusion”), mindfulness skills, and flexible perspective taking. The values component of the app teaches skills for determining the core life domains that motivate quitting smoking (eg, family, health, and spirituality) and taking repeated small actions within these domains (eg, playing with grandchildren) to develop a smoke-free life. The program is self-paced, and the content is sequentially unlocked across 8 levels. Each of the first 4 levels is made accessible immediately after the prior level is completed, whereas each of the last 4 levels is only unlocked upon recording 7 consecutive days without smoking. If a participant lapses, the program encourages (but does not require) them to set a new quit date and return to the first 4 levels for preparation. The program also includes on-demand tools to help in coping with smoking urges, tracking the daily number of cigarettes smoked, and urges passed without smoking. Content was presented in a sequenced format with short paragraphs of text and some audio or visual content for experiencing ACT concepts.
Participants randomized to the QuitGuide arm received access to download the QuitGuide smartphone app (version 1.2.2). QuitGuide content is delivered in four main sections: (1) “Thinking about quitting,” which focuses on motivations to quit by using reason and logic such as identifying reasons to quit and providing information on the health consequences of smoking and quitting; (2) “Preparing to Quit,” which helps users develop a quit plan, identify smoking behaviors, triggers, and reasons for being smoke free, and social support for quitting; (3) “Quitting,” which teaches skills for avoiding cravings to smoke; and (4) “Staying Quit,” which presents tips, motivations, and actions to stay smoke free and skills for coping with slips. No smoking cessation medications or coaching was provided in either intervention arm . Content was presented in a sequenced format with short paragraphs of text.
Baseline Characteristics and Covariates
Data collected at baseline included age, gender, race, ethnicity, education, employment, income, marital status, and sexual orientation. Study participants completed validated positive screening tools to assess mental health, including depression , panic [ ], and posttraumatic stress disorder [ ]. Alcohol consumption and heavy drinking were assessed using the Quick Drinking Screen [ ]. Smoking behavior variables included nicotine dependence (measured using the Fagerström Test for Nicotine Dependence) [ ], number of cigarettes smoked per day, years of smoking, use of e-cigarettes, quit attempts, and relationships with other smokers. Acceptance of internal cues to smoke was measured via the Avoidance and Inflexibility Scale (adapted from the study by Gifford et al [ ]), using means of the three 9-item subscales that assess one’s willingness to experience physical sensations, emotions, and thoughts that cue smoking. The items are rated on a 5-point scale from 1=“not at all” to 5=“very willing” and averaged, with higher scores indicating greater acceptance. A sample physical sensation item was “How willing are you to notice these bodily sensations without smoking?” and items from the emotions and thoughts subscales were similar, substituting “feelings” or “thoughts” for “bodily sensations.” Valued living was measured using the 10-item Valuing Questionnaire [ ], designed to assess the extent of personal values enactment. Each item is rated on a 7-point scale ranging from 0=“not at all true” to 6=“completely true.” Scores were averaged, and 2 distinct factors were derived: progress and obstruction, with higher scores indicating either greater progress or greater obstruction toward valued living, respectively. A sample progress item was “I worked toward my goals even if I didn’t feel motivated to” and a sample obstruction item was “I was basically on auto-pilot most of the time.”
Engagement: Baseline to Day 182 Log-ins
Engagement with the assigned app was measured objectively using Google Analytics. The measure of engagement was the number of days each application was opened, which was consistent with other digital interventions’ measures of engagement [, , ]. For each participant, time- and date-stamped log file records of each page opening were recorded. For this analysis, we used a binary measure indicating whether each participant logged in at least once each day (ie, had at least one log-in recorded in the log file data). Using this method, each participant had a 0/1 code for each day for 182 days from the date of randomization. Owing to a technical error in the Google Analytics system, only the first 182 days of engagement data were available for the study sample.
Smoking Cessation Outcome: 12 Months
The parent trial’s primary smoking cessation outcome was specified a priori as self-reported complete case 30-day point prevalence abstinence (PPA) at the 12-month follow-up. As reported in the parent trial, for the primary outcome of 30-day PPA at the 12-month follow-up, iCanQuit participants had 1.49 times higher odds of quitting smoking as compared with QuitGuide participants (293/1040, 28.17% abstinent vs 225/1067, 21.08% abstinent; OR 1.49, 95% CI 1.22-1.83; P<.001). Note that when missing data were coded as smokers, 12-month 30-day PPA results were very similar (293/1214, 24.13% abstinent for iCanQuit vs 225/1201, 18.73% abstinent for QuitGuide; OR 1.40, 95% CI 1.14-1.71; P<.001).
The analyses were conducted separately for each app. As mentioned in the engagement measurement, log-in data were summarized as a binary time series indicating log-in occurrence on each day of the first 6 months (ie, 182 days) of using the application, from the date of randomization of each participant. Next, log-in time series were presmoothed as the average number of days logged in over a window of 7 previous days [, ]. This type of dense trajectories data is known as functional data. We applied functional clustering based on functional principal component (FPC) analysis. Specifically, we conducted an FPC analysis [ ] by smoothed covariance to summarize each participant’s log-in trajectory using a set of low-dimensional FPC scores. We chose to retain the first 3 and 4 FPC scores for clustering in the iCanQuit and QuitGuide arms, respectively, based on a minimum threshold of 90% for the percentage of variance explained. Trajectories were clustered using the Clustering for Large Applications algorithm [ ] into k=2 and 3 groups in each arm, which met a minimum prediction strength [ ] threshold of 0.6. The Clustering for Large Applications procedure does not rely on parametric assumptions on the shapes of trajectories and is capable of handling densely recorded longitudinal data and complex missing data patterns. We then examined the cluster solutions in each arm for a minimum group size ≥5% of the sample and reasonable separation of the mean log-in trajectories among groups to determine the optimal number of groups for each.
After determining distinct trajectory groups, smoking cessation rates were compared among the groups using logistic regression, with the lowest use group as the reference group. Baseline characteristics with significant univariate associations with cessation were considered as covariates, to control for characteristics that may confound the association between trajectory group and cessation . Thus, the aim of the analysis, as guided by the study’s scientific questions, was to understand the unique prediction of the 12-month cessation outcome by trajectory group membership. A shared set of covariates (ie, all baseline characteristics) was considered for each treatment arm. We conducted stepwise logistic regression using Akaike Information Criterion to determine the subset of covariates to be included in the final adjusted model [ ]. Finally, baseline characteristics were compared among the groups. Those with significant univariate association with trajectory group membership were considered in stepwise selection, using Akaike Information Criterion, of an adjusted multinomial logistic regression model (iCanQuit arm) or logistic regression model (QuitGuide arm) to determine the best baseline predictors of group membership. All statistical tests were 2-sided, with α=.05, and analyses were conducted in R (version 4.0.3; R Foundation for Statistical Computing [ ]), using the R packages “refund” [ ] for FPC analysis, “fpc” [ ] for prediction strength, and “nnet” [ ] for multinomial logistic regression.
All study activities were approved by the institutional review board at the Fred Hutchinson Cancer Research Center (approval number IR-8317).
Description of Sample
shows the baseline demographics and participant characteristics in both the iCanQuit and QuitGuide arms. The mean (SD) age at enrollment was 37.8 (10.8) years. Furthermore, 70.28% (1499/2133) of the participants were female and 35.72% (762/2133) of the participants reported racial and ethnic minority backgrounds. There were 40.46% (863/2133) of the participants with a high school or less education. Regarding smoking, 74.54% (1590/2133) of the participants smoked more than half a pack (at least 11 cigarettes) per day. Less than half (785/2030, 38.67%) of the participants had made a quit attempt in the last year, and 82.47% (1759/2133) of the sample had been smoking for >10 years, with an average Fagerström Test for Nicotine Dependence score of 5.86 (moderate nicotine dependence; SD 2.04). There were no statistically significant differences between the 2 arms for any baseline variable (all P>.05).
Description of Distinct Groups of Trajectories
The functional clustering analysis of 26 weeks of log-ins revealed 3 distinct groups of trajectories for iCanQuit versus 2 distinct groups for QuitGuide. Log-in patterns are shown in(for iCanQuit) and (for QuitGuide). For iCanQuit ( ), the first trajectory group (610/1069, 57.06%) logged in a mean of 2.0 days in the first week and then had <1 mean log-in day in weeks 2 and beyond. They were termed “1-week users.” The second trajectory group (303/1069, 28.34%) logged in a mean of 4.6 days in week 1, a mean of 3.1 days in week 2, a mean of 2.0 days in week 3, a mean of 1.2 days in week 4, and then had <1 mean log-in day in weeks 5 and beyond. They were termed “4-week users.” The third trajectory group (156/1069, 14.59%) logged in a mean of 5.0 to 5.4 days per week in weeks 1 through 5, a mean of 3.1 to 4.7 days per week in weeks 6 through 10, tapering to a mean of twice every week starting week 17, and continuing in this pattern until week 26. They were termed “26-week users.”
For QuitGuide (), the first trajectory group (695/1064, 65.32%) logged in a mean of 1.4 days in the first week and then had <1 mean log-in day in weeks 2 and beyond. As with iCanQuit, they were termed “1-week users.” The second trajectory group (369/1064, 34.68%) logged in a mean of 2.8 times in week 1, a mean of 1.7 times in week 2, a mean of 1.1 times in week 3, and then had few log-ins after that. They were termed “3-week users.”
Trajectory Membership Prediction of Smoking Cessation Outcome
shows each intervention arm’s trajectory group membership as a predictor of 30-day PPA at the 12-month follow-up, after controlling for all baseline covariates included in the statistical model. For iCanQuit, abstinence rates for the 3 trajectory groups were 23% for 1-week users, 30% for 4-week users, and 56% for 26-week users. Compared with the 1-week users, the 4-week users had 50% higher odds (OR 1.50, 95% CI 1.05-2.14; P=.03), whereas 26-week users had 397% higher odds (OR 4.97, 95% CI 3.31-7.52; P<.001), respectively, of being abstinent at 12 months. Descriptively, for QuitGuide, abstinence rates for the 2 trajectory groups were 21% for 1-week users and 23% for 3-week users. There was no significant difference in the odds of being abstinent at 12 months for 3-week versus 1-week users (OR 1.16, 95% CI 0.84-1.62; P=.37). The above models adjusted for the baseline covariates selected, as outlined in the statistical methods, and are shown in .
|Treatment arm and covariate||Odds ratio (95% CI)||P value|
|4-week users||1.50 (1.05-2.14)||.03|
|26-week users||4.97 (3.31-7.52)||<.001|
|Gender (male)||1.87 (1.33-2.62)||<.001|
|High school or lower education||1.42 (1.03-1.95)||.03|
|Depression screen positive||0.69 (0.50-0.97)||.03|
|Panic disorder screen positive||0.74 (0.50-1.09)||.14|
|Used e-cigarettes at least once in past month||0.66 (0.45-0.96)||.03|
|Confidence in being smoke free||1.01 (1.01-1.02)||<.001|
|Drinks per day on a typical drinking day||0.95 (0.91-1.00)||.048|
|3-week users||1.16 (0.84-1.62)||.37|
|Used e-cigarettes at least once in past month||1.44 (0.99-2.08)||.05|
|Confidence in being smoke free||1.01 (1.00-1.02)||.005|
|Close friends who smoke||0.89 (0.81-0.97)||.01|
|Heavy drinkerb||0.64 (0.37-1.07)||.10|
|Valuing questionnaire—progress||1.03 (1.01-1.06)||.01|
aThe reference group is 1-week users for both arms.
bHeavy drinkers are defined as women who had 4 or more drinks and men who had 5 or more drinks on a typical drinking day.
Baseline Characteristics Predicting Trajectory Membership
Because the trajectory groups were different across the 2 arms,presents the results for baseline characteristics predicting membership in the groups for the 2 arms separately. For iCanQuit, the baseline characteristics significantly associated with more engaged group membership, as compared with 1-week user group membership, were age in years (OR 1.05, 95% CI 1.03-1.06 for 26-week users), smoking up to one-half pack per day (OR 1.90, 95% CI 1.25-2.87 for 26-week users), smoking first cigarette >5 minutes after waking (OR 1.42, 95% CI 1.05-1.92 for 4-week users), and higher mean acceptance of internal physical sensations (OR 1.82, 95% CI 1.41-2.35 for each 1-point increase for 4-week users).
For QuitGuide, the baseline characteristics significantly associated with 3-week user group membership, as compared with 1-week user group membership, were female gender (OR 1.46, 95% CI 1.10-1.95), minority race (people of color) or ethnicity (Hispanic; OR 1.40, 95% CI 1.08-1.83), and smoked for 10 or more years (OR 1.56, 95% CI 1.04-2.35).
|Arm, trajectory group, and characteristic||Odds ratio (95% CI)|
|Age (years)||1.01 (0.99-1.02)|
|Smokes ≤10 cigarettes per day||1.25 (0.89-1.76)|
|First cigarette >5 minutes after waking||1.42 (1.05-1.92)|
|Number of quit attempts in previous year||1.03 (0.99-1.08)|
|Each point increase in acceptance of physical sensations||1.82 (1.41-2.35)|
|Age (years)||1.05 (1.03-1.06)|
|Smokes ≤10 cigarettes per day||1.90 (1.25-2.87)|
|First cigarette >5 minutes after waking||1.42 (0.96-2.08)|
|Number of quit attempts in previous year||0.92 (0.83-1.02)|
|Each point increase in acceptance of physical sensations||1.23 (0.88-1.71)|
|QuitGuide: 3-week users|
|Gender (female)||1.46 (1.10-1.95)|
|Minority race or ethnicity||1.40 (1.08-1.83)|
|Anxiety screen positive||0.75 (0.55-1.02)|
|Smoked for ≥10 years||1.56 (1.04-2.35)|
aThe reference group is 1-week users for both treatment arms.
This study contributes to the nascent literature on longitudinal use trajectories of digital health interventions and their prediction of health outcomes [- ]. The study found (1) 1-, 4-, and 26-week trajectories for iCanQuit versus 1- and 3-week trajectories for the QuitGuide smoking cessation apps; (2) that these trajectory groups differentially predicted smoking outcomes at 12 months for iCanQuit but not for QuitGuide; and (3) that certain user characteristics were associated with membership in certain trajectory groups. Notably, compared with the 1-week iCanQuit users, the 4-week users had 50% higher odds, whereas 26-week users had 397% higher odds, respectively, of being abstinent at 12 months. The results are discussed in the following sections.
Use Trajectories and Health Outcomes
At least half of the participants in both arms were 1-week users. Similarly, our 2018 study examining log-in trajectories of 2 smoking cessation websites found that half of the participants were 1-week users, and similar to this study, that study showed that these participants were the least likely to have quit smoking at the 12-month follow-up . Thus, there are now 4 separate digital interventions (2 in this study and 2 in the 2018 study [ ]) showing large proportions of users (645/1309, 49.27% to 610/1069, 57.06%) who are 1-week users, suggesting overall that 1-week use may be a common engagement pattern of digital health interventions. Thus, it is imperative to identify early who would likely become a 1-week user. For example, the baseline characteristics results of this study suggest that a younger age, smoking at least one-half pack per day, smoking the first cigarette within 5 minutes of waking (a marker of nicotine dependence), and scoring lower on acceptance of internal physical sensations that trigger smoking (a marker of avoidance of cigarette cravings) predicted membership in the iCanQuit 1-week trajectory group. Measuring these factors at baseline might allow for the early identification of individuals who would be more likely to disengage from iCanQuit in the first week. Another approach that might be worth testing in future research is investigating use patterns within the first week (eg, number of log-ins per day and time spent on the app per day) to predict whether a participant would become a 1-week user. Once identified, more intensive intervention strategies could be used with this group, which might range from push notification communications or proactive intervention. Beyond this study, it would be worthwhile to determine whether 1-week use is a common pattern across multiple digital platforms (eg, websites and smartphone apps) and health domains (eg, tobacco, exercise, and diet) and to what extent this use pattern affects health outcomes.
The second trajectory group for each arm was the 4-week users for iCanQuit (303/1069, 28.34%) and 3-week users for QuitGuide (369/1064, 34.68%). Although the length of each group was similar (3-4 weeks) and the proportion of each group was somewhat smaller in iCanQuit, only the iCanQuit 4-week users had significantly higher quit rates than their 1-week comparators (ie, 50% higher odds of quitting). Two potential reasons why iCanQuit’s, but not QuitGuide’s, second trajectory group had higher quit rates are most likely due to the content and structure of the iCanQuit app. Regarding content, we have published multiple studies showing that the effect of iCanQuit (but not QuitGuide) on smoking cessation was mediated by ACT-based processes of acceptance of internal cues to smoke (ie, sensations, thoughts, and emotions) [, - ]. The differences in content, with iCanQuit focused on ACT versus QuitGuide focused on standard USCPG content [ ], suggest that 4 weeks of engaging with ACT content that targets acceptance of internal cues is effective at improving quit rates. Regarding structure, the iCanQuit app presented content in a sequenced interactive format (eg, content is unlocked in a sequential manner) with short paragraphs of text and some audio or visual for experiencing ACT concepts, whereas the QuitGuide app presented content in a sequenced format with short paragraphs of text [ ]. Thus, the extent to which intervention engagement predicts behavior change might depend on the content and structure of the intervention, which is a valuable topic for future research. For iCanQuit, these results suggest that strategies to increase engagement 3 more weeks (ie, from 1 to 4) could be an effective approach to improving quit rates. Example strategies worth testing include (1) proactive check-ins (via SMS text messages or phone calls) from staff about progress with the iCanQuit app, (2) rewarding each day’s use of iCanQuit, and (3) a “four-week challenge,” which shows other users’ daily log-in progress toward the goal of 4 weeks of use.
Only iCanQuit had a third trajectory group, namely 26-week users (156/1069, 14.59% of the iCanQuit arm sample). The 26-week users’ group had nearly 400% higher odds of quitting smoking (as compared with 1-week users). The 12-month 56% quit rates observed in this group are the highest we have ever observed in a digital smoking cessation intervention and suggest that iCanQuit could be a highly effective and scalable intervention for this group of users. Our 2018 paper found a similar group of long-term users on the WebQuit website (159/1240, 12.82% of WebQuit arm sample) who had high 12-month quit rates (34.2% ) but not as high as those found here for iCanQuit. The iCanQuit ACT-based content and structure may have encouraged long-term, spaced skills practice [ ]. Taken together, the findings for both the iCanQuit and WebQuit third trajectory groups suggest that consistent use of each program over time is prognostic of a better health outcome, which is contrary to the notion that consistent log-ins may be a marker of ongoing challenges and struggles to change a health behavior. Instead, consistent log-ins over time may be a marker of a participant’s commitment to changing a health behavior. Digital intervention designs could focus on methods to encourage commitment and prevent lapses over time, which may include strategies similar to those suggested above, in addition to just-in-time adaptive interventions that aim to provide the right type of lapse-prevention support to smokers at the right time [ ].
Personal Characteristics and Use Trajectories
The impact of personal characteristics on use trajectories appears to vary according to the intervention. For example, among iCanQuit participants, smoking the first cigarette >5 minutes after waking and higher levels of acceptance of physical cues to smoke predicted being a 4-week user, whereas older age and smoking ≤10 cigarettes per day predicted being a 26-week user. The findings generally suggest that less dependence and greater acceptance of cravings predict long-term engagement with iCanQuit. The results on increasing age predicting iCanQuit’s 26-week use trajectory membership are consistent with past research showing that older age is a predictor of higher digital health intervention use [- ], including our 2018 WebQuit trajectories paper [ ]. In contrast to the view that as people age, their willingness to use technology decreases [ ], this study suggests that increasing age may actually indicate which one is more likely to remain long-term users of iCanQuit, and in turn, have very high quit rates. In contrast, although there were baseline factors that predicted QuitGuide’s use trajectory membership, neither of the 2 trajectories predicted smoking cessation, so the value of these baseline prediction results is unclear. Nonetheless, we recommend that future research explore a variety of baseline subgroup differences (eg, sex, race, and age) in digital intervention trajectories to better understand who is most or least likely to engage over time. Overall, these analyses suggest a need for further research on which baseline factors might predict different use trajectories and therefore inform the development of tailored interventions that facilitate long-term, consistent engagement based on an individual’s specific baseline characteristics.
This study has several key limitations. First, only 2 smartphone apps were tested, and both were focused on smoking cessation; thus, future research should examine the extent to which the results generalize to other behaviors and to other types of digital interventions. Cessation outcome data were self-reported for the reasons stated in the Methods section. Remote biochemical validation of smoking cessation would have introduced biases including low response rates, prohibitive cost, challenges with confirming the identity of the person providing the sample, and inability to confirm abstinence beyond 24 hours [- ]. Owing to a technical error, log-ins were recorded for the first 6 months of the trial. Finally, as users self-select to different app use patterns (rather than being randomized), the associations observed in this study may not be causal, and care should be taken in their interpretation.
Patterns of 1-, 3-, and 4-week use of smartphone apps may be common for how people engage in digital health interventions. In addition, 4-week users, and especially 26-week users of iCanQuit, have higher odds of quitting smoking. Strategies to detect potential 1-week iCanQuit users and proactively offer them more intensive intervention could be fruitful.
The funders, National Cancer Institute (R01 CA192849) and Cancer Research UK (C1417/A22962), had no role in the trial conduct or interpretation of results.
Conflicts of Interest
OP is an unpaid member of the scientific advisory board for the Smoke Free app.
Summary of baseline characteristics of participants from the iCanQuit and QuitGuide arms by log-in trajectories.DOCX File , 25 KB
- Torous J, Nicholas J, Larsen ME, Firth J, Christensen H. Clinical review of user engagement with mental health smartphone apps: evidence, theory and improvements. Evid Based Ment Health 2018 Aug;21(3):116-119. [CrossRef] [Medline]
- Perski O, Blandford A, West R, Michie S. Conceptualising engagement with digital behaviour change interventions: a systematic review using principles from critical interpretive synthesis. Transl Behav Med 2017 Jun;7(2):254-267 [FREE Full text] [CrossRef] [Medline]
- Kohl LF, Crutzen R, de Vries NK. Online prevention aimed at lifestyle behaviors: a systematic review of reviews. J Med Internet Res 2013 Jul 16;15(7):e146 [FREE Full text] [CrossRef] [Medline]
- Graham AL, Papandonatos GD, Cha S, Erar B, Amato MS. Improving adherence to smoking cessation treatment: smoking outcomes in a Web-based randomized trial. Ann Behav Med 2018 Mar 15;52(4):331-341 [FREE Full text] [CrossRef] [Medline]
- Graham AL, Zhao K, Papandonatos GD, Erar B, Wang X, Amato MS, et al. A prospective examination of online social network dynamics and smoking cessation. PLoS One 2017 Aug 23;12(8):e0183655 [FREE Full text] [CrossRef] [Medline]
- Danaher BG, Smolkowski K, Seeley JR, Severson HH. Mediators of a successful Web-based smokeless tobacco cessation program. Addiction 2008 Oct;103(10):1706-1712 [FREE Full text] [CrossRef] [Medline]
- Alkhaldi G, Hamilton FL, Lau R, Webster R, Michie S, Murray E. The effectiveness of prompts to promote engagement with digital interventions: a systematic review. J Med Internet Res 2016 Jan 08;18(1):e6 [FREE Full text] [CrossRef] [Medline]
- Brouwer W, Kroeze W, Crutzen R, de Nooijer J, de Vries NK, Brug J, et al. Which intervention characteristics are related to more exposure to Internet-delivered healthy lifestyle promotion interventions? A systematic review. J Med Internet Res 2011 Jan 06;13(1):e2 [FREE Full text] [CrossRef] [Medline]
- Spaulding EM, Marvel FA, Piasecki RJ, Martin SS, Allen JK. User engagement with smartphone apps and cardiovascular disease risk factor outcomes: systematic review. JMIR Cardio 2021 Feb 03;5(1):e18834 [FREE Full text] [CrossRef] [Medline]
- Mclaughlin M, Delaney T, Hall A, Byaruhanga J, Mackie P, Grady A, et al. Associations between digital health intervention engagement, physical activity, and sedentary behavior: systematic review and meta-analysis. J Med Internet Res 2021 Feb 19;23(2):e23180 [FREE Full text] [CrossRef] [Medline]
- Browne J, Halverson TF, Vilardaga R. Engagement with a digital therapeutic for smoking cessation designed for persons with psychiatric illness fully mediates smoking outcomes in a pilot randomized controlled trial. Transl Behav Med 2021 Sep 15;11(9):1717-1725. [CrossRef] [Medline]
- Ubhi HK, Michie S, Kotz D, van Schayck OC, Selladurai A, West R. Characterising smoking cessation smartphone applications in terms of behaviour change techniques, engagement and ease-of-use features. Transl Behav Med 2016 Sep;6(3):410-417 [FREE Full text] [CrossRef] [Medline]
- Bendotti H, Lawler S, Ireland D, Gartner C, Hides L, Marshall H. What do people want in a smoking cessation app? An analysis of user reviews and app quality. Nicotine Tob Res 2022 Feb 01;24(2):169-177. [CrossRef] [Medline]
- Bricker JB, Mull KE, Kientz JA, Vilardaga R, Mercer LD, Akioka KJ, et al. Randomized, controlled pilot trial of a smartphone app for smoking cessation using acceptance and commitment therapy. Drug Alcohol Depend 2014 Oct 01;143:87-94 [FREE Full text] [CrossRef] [Medline]
- Sridharan V, Shoda Y, Heffner J, Bricker J. A pilot randomized controlled trial of a Web-based growth mindset intervention to enhance the effectiveness of a smartphone app for smoking cessation. JMIR Mhealth Uhealth 2019 Jul 09;7(7):e14602 [FREE Full text] [CrossRef] [Medline]
- Zeng EY, Heffner JL, Copeland WK, Mull KE, Bricker JB. Get with the program: adherence to a smartphone app for smoking cessation. Addict Behav 2016 Dec;63:120-124 [FREE Full text] [CrossRef] [Medline]
- Perski O, Crane D, Beard E, Brown J. Does the addition of a supportive chatbot promote user engagement with a smoking cessation app? An experimental study. Digit Health 2019 Sep 30;5:2055207619880676 [FREE Full text] [CrossRef] [Medline]
- Shaughnessy JJ. Long-term retention and the spacing effect in free-recall and frequency judgments. Am J Psychol 1977 Dec;90(4):587-598. [CrossRef]
- Pyc MA, Rawson KA. Testing the retrieval effort hypothesis: does greater difficulty correctly recalling information lead to higher levels of memory? J Mem Lang 2009 May;60(4):437-447. [CrossRef]
- Christofferson DE, Hertzberg JS, Beckham JC, Dennis PA, Hamlett-Berry K. Engagement and abstinence among users of a smoking cessation text message program for veterans. Addict Behav 2016 Nov;62:47-53 [FREE Full text] [CrossRef] [Medline]
- Heminger CL, Boal AL, Zumer M, Abroms LC. Text2Quit: an analysis of participant engagement in the mobile smoking cessation program. Am J Drug Alcohol Abuse 2016 Jul;42(4):450-458. [CrossRef] [Medline]
- Coa KI, Wiseman KP, Higgins B, Augustson E. Associations between engagement and outcomes in the SmokefreeTXT program: a growth mixture modeling analysis. Nicotine Tob Res 2019 Apr 17;21(5):663-669 [FREE Full text] [CrossRef] [Medline]
- Zhao K, Wang X, Cha S, Cohn AM, Papandonatos GD, Amato MS, et al. A multirelational social network analysis of an online health community for smoking cessation. J Med Internet Res 2016 Aug 25;18(8):e233 [FREE Full text] [CrossRef] [Medline]
- Bricker JB, Sridharan V, Zhu Y, Mull KE, Heffner JL, Watson NL, et al. Trajectories of 12-month usage patterns for two smoking cessation websites: exploring how users engage over time. J Med Internet Res 2018 Apr 20;20(4):e10143 [FREE Full text] [CrossRef] [Medline]
- Perski O, Watson NL, Mull KE, Bricker JB. Identifying content-based engagement patterns in a smoking cessation website and associations with user characteristics and cessation outcomes: a sequence and cluster analysis. Nicotine Tob Res 2021 Jun 08;23(7):1103-1112 [FREE Full text] [CrossRef] [Medline]
- Golas SB, Nikolova-Simons M, Palacholla R, Op den Buijs J, Garberg G, Orenstein A, et al. Predictive analytics and tailored interventions improve clinical outcomes in older adults: a randomized controlled trial. NPJ Digit Med 2021 Jun 10;4(1):97 [FREE Full text] [CrossRef] [Medline]
- Nahum-Shani I, Qian M, Almirall D, Pelham WE, Gnagy B, Fabiano GA, et al. Q-learning: a data analysis method for constructing adaptive interventions. Psychol Methods 2012 Dec;17(4):478-494 [FREE Full text] [CrossRef] [Medline]
- Wangberg SC, Bergmo TS, Johnsen JA. Adherence in Internet-based interventions. Patient Prefer Adherence 2008 Feb 02;2:57-65 [FREE Full text] [Medline]
- Strecher VJ, McClure J, Alexander G, Chakraborty B, Nair V, Konkel J, et al. The role of engagement in a tailored Web-based smoking cessation program: randomized controlled trial. J Med Internet Res 2008 Nov 04;10(5):e36 [FREE Full text] [CrossRef] [Medline]
- Balmford J, Borland R, Benda P. Patterns of use of an automated interactive personalized coaching program for smoking cessation. J Med Internet Res 2008 Dec 17;10(5):e54 [FREE Full text] [CrossRef] [Medline]
- Zbikowski SM, Hapgood J, Smucker Barnwell S, McAfee T. Phone and Web-based tobacco cessation treatment: real-world utilization patterns and outcomes for 11,000 tobacco users. J Med Internet Res 2008 Nov 14;10(5):e41 [FREE Full text] [CrossRef] [Medline]
- Rahman QA, Janmohamed T, Pirbaglou M, Ritvo P, Heffernan JM, Clarke H, et al. Patterns of user engagement with the mobile app, manage my pain: results of a data mining investigation. JMIR Mhealth Uhealth 2017 Jul 12;5(7):e96 [FREE Full text] [CrossRef] [Medline]
- Cillessen L, van de Ven MO, Compen FR, Bisseling EM, van der Lee ML, Speckens AE. Predictors and effects of usage of an online mindfulness intervention for distressed cancer patients: usability study. J Med Internet Res 2020 Oct 02;22(10):e17526 [FREE Full text] [CrossRef] [Medline]
- Goh G, Tan NC, Malhotra R, Padmanabhan U, Barbier S, Allen JC, et al. Short-term trajectories of use of a caloric-monitoring mobile phone app among patients with type 2 diabetes mellitus in a primary care setting. J Med Internet Res 2015 Feb 03;17(2):e33 [FREE Full text] [CrossRef] [Medline]
- Bricker JB, Watson NL, Mull KE, Sullivan BM, Heffner JL. Efficacy of smartphone applications for smoking cessation: a randomized clinical trial. JAMA Intern Med 2020 Nov 01;180(11):1472-1480 [FREE Full text] [CrossRef] [Medline]
- Gifford EV, Kohlenberg BS, Hayes SC, Antonuccio DO, Piasecki MM, Rasmussen-Hall ML, et al. Acceptance-based treatment for smoking cessation. Behav Ther 2004;35(4):689-705 [FREE Full text] [CrossRef]
- Ramsay JO, Silverman BW. Applied Functional Data Analysis: Methods and Case Studies. New York, NY, USA: Springer; 2002.
- Kaufman L, Rousseeuw PJ. Finding Groups in Data: An Introduction to Cluster Analysis. Hoboken, NJ, USA: Wiley Online Library; 1990.
- Tibshirani R, Walther G. Cluster validation by prediction strength. J Comput Graph Stat 2005;14(3):511-528. [CrossRef]
- Pocock SJ, Assmann SE, Enos LE, Kasten LE. Subgroup analysis, covariate adjustment and baseline comparisons in clinical trial reporting: current practice and problems. Stat Med 2002 Oct 15;21(19):2917-2930. [CrossRef] [Medline]
- Venables WN, Ripley BD. Modern Applied Statistics with S. 4th edition. New York, NY, USA: Springer; 2002.
- R Core Team. R: a language and environment for statistical computing. R Foundation for Statistical Computing. 2020. URL: https://www.R-project.org/ [accessed 2022-03-28]
- Goldsmith J, Scheipl F, Huang L, Wrobel J, Di C, Gellar J, et al. refund: Regression with Functional Data. The Comprehensive R Archive Network. 2022 Apr 16. URL: https://CRAN.R-project.org/package=refund [accessed 2022-03-28]
- Hennig C. fpc: Flexible Procedures for Clustering. The Comprehensive R Archive Network. 2020. URL: https://CRAN.R-project.org/package=fpc [accessed 2022-03-28]
- Bricker JB, Levin M, Lappalainen R, Mull K, Sullivan B, Santiago-Torres M. Mechanisms of smartphone apps for cigarette smoking cessation: results of a serial mediation model from the iCanQuit randomized trial. JMIR Mhealth Uhealth 2021 Nov 09;9(11):e32847 [FREE Full text] [CrossRef] [Medline]
- Santiago-Torres M, Mull KE, Sullivan BM, Kendzor DE, Bricker JB. Efficacy and utilization of smartphone applications for smoking cessation among low-income adults: secondary analysis of the iCanQuit randomized trial. Drug Alcohol Depend 2022 Feb 01;231:109258. [CrossRef] [Medline]
- Santiago-Torres M, Mull KE, Sullivan BM, Kwon D, Nollen NL, Zvolensky MJ, et al. Efficacy and utilization of an acceptance and commitment therapy-based smartphone application for smoking cessation among Black adults: secondary analysis of the iCanQuit randomized trial. Addiction 2022 Mar;117(3):760-771. [CrossRef] [Medline]
- Santiago-Torres M, Mull K, Sullivan B, Kwon DM, Nez Henderson P, Nelson LA, et al. Efficacy and utilization of smartphone applications for smoking cessation among American Indians and Alaska Natives: results from the iCanQuit trial. Nicotine Tob Res 2022 Mar 01;24(4):544-554. [CrossRef] [Medline]
- Fiore MC. Treating tobacco use and dependence: an introduction to the US Public Health Service Clinical Practice Guideline. Respir Care 2000 Oct;45(10):1196-1199. [Medline]
- Nahum-Shani I, Smith SN, Spring BJ, Collins LM, Witkiewitz K, Tewari A, et al. Just-in-Time Adaptive Interventions (JITAIs) in mobile health: key components and design principles for ongoing health behavior support. Ann Behav Med 2018 May 18;52(6):446-462 [FREE Full text] [CrossRef] [Medline]
- Wandke H, Sengpiel M, Sönksen M. Myths about older people's use of information and communication technology. Gerontology 2012;58(6):564-570 [FREE Full text] [CrossRef] [Medline]
- Cha S, Ganz O, Cohn AM, Ehlke SJ, Graham AL. Feasibility of biochemical verification in a Web-based smoking cessation study. Addict Behav 2017 Oct;73:204-208 [FREE Full text] [CrossRef] [Medline]
- Herbec A, Brown J, Shahab L, West R. Lessons learned from unsuccessful use of personal carbon monoxide monitors to remotely assess abstinence in a pragmatic trial of a smartphone stop smoking app - a secondary analysis. Addict Behav Rep 2019 Jun;9:100122 [FREE Full text] [CrossRef] [Medline]
- Thrul J, Meacham MC, Ramo DE. A novel and remote biochemical verification method of smoking abstinence: predictors of participant compliance. Tob Prev Cessat 2018 May;4:20 [FREE Full text] [CrossRef] [Medline]
- Garrison KA, Pal P, O'Malley SS, Pittman BP, Gueorguieva R, Rojiani R, et al. Craving to quit: a randomized controlled trial of smartphone app-based mindfulness training for smoking cessation. Nicotine Tob Res 2020 Mar 16;22(3):324-331 [FREE Full text] [CrossRef] [Medline]
- van der Aalst CM, de Koning HJ. Biochemical verification of the self-reported smoking status of screened male smokers of the Dutch-Belgian randomized controlled lung cancer screening trial. Lung Cancer 2016 Apr;94:96-101. [CrossRef] [Medline]
- Wong SL, Shields M, Leatherdale S, Malaison E, Hammond D. Assessment of validity of self-reported smoking status. Health Rep 2012 Mar;23(1):47-53 [FREE Full text] [Medline]
- Benowitz NL, Bernert JT, Foulds J, Hecht SS, Jacob P, Jarvis MJ, et al. Biochemical verification of tobacco use and abstinence: 2019 update. Nicotine Tob Res 2020 Jun 12;22(7):1086-1097 [FREE Full text] [CrossRef] [Medline]
- SRNT Subcommittee on Biochemical Verification. Biochemical verification of tobacco use and cessation. Nicotine Tob Res 2002 May;4(2):149-159. [CrossRef] [Medline]
|ACT: acceptance and commitment therapy|
|FPC: functional principal component|
|OR: odds ratio|
|PPA: point prevalence abstinence|
|USCPG: US Clinical Practice Guidelines|
Edited by G Eysenbach; submitted 02.05.22; peer-reviewed by N Maglaveras, K Wiseman; comments to author 24.05.22; revised version received 03.06.22; accepted 13.07.22; published 18.08.22Copyright
©Jonathan B Bricker, Kristin E Mull, Margarita Santiago-Torres, Zhen Miao, Olga Perski, Chongzhi Di. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 18.08.2022.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.