This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.
Despite the near ubiquity of mobile phones, little research has been conducted on the implementation of mobile health (mHealth) apps to treat patients in primary care. Although primary care clinicians routinely treat chronic conditions such as asthma and diabetes, they rarely treat addiction, a common chronic condition. Instead, addiction is most often treated in the US health care system, if it is treated at all, in a separate behavioral health system. mHealth could help integrate addiction treatment in primary care.
The objective of this paper was to report the effects of implementing an mHealth system for addiction in primary care on both patients and clinicians.
In this implementation research trial, an evidence-based mHealth system named Seva was introduced sequentially over 36 months to a maximum of 100 patients with substance use disorders (SUDs) in each of three federally qualified health centers (FQHCs; primary care clinics that serve patients regardless of their ability to pay). This paper reports on patient and clinician outcomes organized according to the Reach, Effectiveness, Adoption, Implementation, and Maintenance (RE-AIM) framework.
The outcomes according to the RE-AIM framework are as follows: Reach—Seva reached 8.31% (268/3226) of appropriate patients. Reach was limited by our ability to pay for phones and data plans for a maximum of 100 patients per clinic. Effectiveness—Patients who were given Seva had significant improvements in their risky drinking days (44% reduction, (0.7-1.25)/1.25,
Implementing an mHealth system can improve care among primary care patients with SUDs, and patients using the system can support one another in their recovery. Among clinicians, however, implementation requires figuring out how information from the mHealth system will be used and making mHealth data available in the electronic health (eHealth) record. In addition, paying for an mHealth system remains a challenge.
Despite the near ubiquity of mobile phones, little systematic research has been conducted on the use or implementation of mobile health (mHealth) technology in managing chronic health conditions in primary care. Although primary care patients use mHealth apps, their use is generally haphazard and self-selected [
Although hundreds of mobile phone apps for addiction treatment are available commercially, most of these apps have not been evaluated in the peer-reviewed literature [
In this paper, we report the results of an implementation research trial funded by the National Institutes of Health–National Institute on Drug Abuse. The trial aimed to integrate behavioral health treatment into primary care. We focused on one aspect of behavioral health—addiction—that presents considerable barriers to integration, such as the inability to bill for services, mental health stigma, and primary care physicians being ill-prepared to treat behavioral health problems [
In this research, our premise was that mHealth could ease the integration of addiction treatment into primary care. We proposed to examine how implementing an evidence-based mHealth system for addiction could be useful to both patients and clinical staff in real-world primary care settings. The mHealth system used in the study is named
The study reported here sought answers to 3 broad research questions:
How can Seva be implemented in primary care settings efficiently and effectively?
To what extent do patients and staff accept and use Seva?
How does Seva affect clinical care for patients and staff?
Because the study tested both clinical and implementation interventions, it is considered to be a hybrid type 2 effectiveness-implementation study [
We made Seva available to up to 100 patients at each of 3 federally qualified health centers (FQHCs) across the United States of America. FQHCs are federally funded primary care clinics that serve mainly low-income patients. As a condition of funding, FQHCs must provide access to behavioral health services. Thus, FQHCs are in the vanguard of clinics in the United States trying to integrate behavioral health into primary care. FQHCs also serve very vulnerable patients—many in poverty and suffering from addiction. At each site, patients were enrolled over a 12-month period. After enrollment, patients had access to Seva for 12 months.
Because the focus of the study was implementation and not patient outcomes
The study protocol was approved by the Medical Sciences Institutional Review Board at the University of Wisconsin–Madison (2012-0937-CP020) and is registered at ClinicalTrials.gov (NCT01963234).
Clinics were recruited in partnership with the National Association of Community Health Centers. We recruited FQHCs with established electronic health (eHealth) records to understand how Seva relates to existing clinic technology. In selecting sites, we aimed to achieve geographic reach, diversity in patient populations, and differences in organizational structures to better understand how environmental and structural factors might affect implementation. From a pool of approximately 1100 FQHCs nationally, we selected an FQHC affiliated with the University of Wisconsin as a pilot site; a relatively small, rural, freestanding FQHC with integrated behavioral health services (including addiction treatment) as a second site; and an urban FQHC that largely serves a minority population as our third site.
A limited number of clinician subjects consented at each site. These staff subjects worked with the research team to integrate Seva into clinical workflows during the preimplementation phase and subsequently identified, recruited, enrolled, and trained patients to use Seva and monitored patients’ use of the system.
Authorized clinicians identified potential subjects through the electronic health record (EHR). On the basis of their clinical judgment, clinicians were free to enroll patients from their substance-using populations whom they thought might benefit from Seva. The patients had to meet the following inclusion criteria: (1) aged 18 years or older, (2) meet the criteria for substance use disorder (SUD) as per the Severity of Dependence Scale, (3) have no current psychotic disorder severe enough to prevent participation, (4) have no acute medical problem requiring immediate inpatient treatment, (5) are willing to use Seva, and (6) could understand and sign a consent form in English. If a patient was incarcerated during the study, his or her participation was stopped. If he or she was still interested in participating in the study when released from jail, the person was able to rejoin the study. Patient participation was voluntary. Patient subjects were excluded if (1) their condition warranted inpatient detoxification until they were well enough to participate and (2) they were unable to understand and complete the informed consent. Couples were not recruited to the study at the same time to avoid dyadic conflict.
During a clinic appointment, the clinician asked an eligible patient if he or she was interested in learning more about the study. If the patient was interested and gave permission, the clinician notified the site coordinator that a patient was interested in hearing about the project. In the privacy of a clinic exam room or office, the site coordinator explained the study, its benefits, and potential risks of participation. The site coordinator also answered any questions the patient (subject) had. If the patient was interested in participating, he or she was asked to complete the informed consent.
The flow of patients through the study is shown in
The site coordinator trained participants in person. Participants could download Seva if they had their own mobile phone. If they did not, the site coordinator gave participants a phone and basic instructions on how to use it. Then he or she explained and showed participants how to use the different services of Seva. Participants were encouraged to ask questions and were given a toll-free number to call the Seva technical support line if they had any questions. Each participant also received a user guide that showed all Seva services and explained how to use them with easy text and graphic instructions.
Participant flowchart.
Each site designated a change leader, a clinical leader who was the point of contact with the implementation coach and coordinated implementation activities; the site coordinator; and a change team of 4 to 8 clinical and/or administrative staff members who helped make the organizational changes that were necessary to implement Seva. The coach made an initial site visit during the 4-month period of implementation preparation to create a welcoming environment for Seva. During this visit, the coach conducted a walk-through exercise with the change team members (an exercise in which employees experience clinic processes as patients do); a workflow assessment using flowcharting; and a technical assessment of data to be gathered and procedures needed to conduct the study. Through this preparation period, the coach worked with the change leader to ensure that pretest data were collected, Seva was demonstrated to clinical teams who might serve as referral sources, and barriers to implementation were identified and rectified.
Seva offered patients a discussion board used by the patients in the study; interactive modules to teach problem-solving, self-regulation, and other skills; tools for coping with cravings and high-risk situations (eg, relaxation exercises, strategies from cognitive behavioral therapy, links to local 12-step meetings); and health tracking. For clinicians, Seva provided a Web portal with a Clinician Report containing longitudinal information generated by patients’ self-reported data about their substance use and well-being (eg, sleep, depression).
We created a detailed implementation plan tailored to each clinic. The implementation strategy involved 4 phases at each site, which were (1) initiate (bring key clinical staff together for training), (2) prepare (assure a welcoming environment for Seva), (3) improve (conduct rapid-cycle tests of ideas from the previous stages), and (4) implement (use, monitor, and sustain the technology). An organizational coach (a member of the research team experienced in coaching) helped clinics implement Seva, starting with an in-person planning and kick-off meeting. This meeting was to be followed by monthly phone calls. Implementation plans consisted of a list of scheduled activities, such as when recruitment would start and end, when patients would be trained to use Seva, and so on. The implementation plans were informed by baseline assessments of readiness for implementation, sustainability potential, and technology acceptance from the perspective of clinic staff members who worked on the implementation plan [
We expected that Seva would enable primary care physicians to better manage addiction in their patients, much as they manage chronic conditions such as diabetes and asthma, by making information about a patient’s recovery readily available. Physicians and other clinicians who referred a patient to behavioral health could see through Seva how these referrals worked out. Seva would also improve outcomes by making support, information, and skills training available to patients almost anywhere and anytime.
The measures used in this study are summarized in
List of measures.
Domain and measure | Data source(s) | |
Number of Seva patients (eligible, enrolled) | EHRa, patient surveys | |
Characteristics of participating patients | Patient surveys | |
Substance use | Patient surveys | |
QoLb | Patient surveys | |
Health care utilization (hospitalizations, ERc visits, specialty addiction treatment) | Patient surveys | |
HIV testing rates | Patient surveys | |
HIV risk behaviors | Patient surveys | |
Characteristics of participating clinics | Clinic administrative data | |
Use of Seva by staff (including referrals) | Seva server files, referral tracking logs maintained by clinic staff | |
Use of Seva by patients | Seva server files | |
Stages of Implementation Completion | Project tracking logs maintained by research team | |
Implementation and operating costs | Observation and interviews of clinic staff; project administrative data | |
6-month follow-up on effectiveness measures | Patient surveys | |
Patient use of Seva at 12 months | Seva server files |
aEHR: electronic health record.
bQoL: quality of life.
cER: emergency room.
To assess
To assess
Effectiveness outcomes are reported for differences between baseline and 6 months. To handle skew distributions, nonparametric related sample tests (Wilcoxon signed rank tests for continuous and ordinal variables and McNemar tests for binary outcomes) were conducted.
We assessed
We selected an FQHC in Madison, WI, affiliated with the University of Wisconsin-Madison as our pilot implementation site; a relatively small, rural, freestanding FQHC with integrated behavioral health services, including addiction treatment, in Missoula, MT, was our second site; and an urban FQHC in the Bronx, NY, that serves a largely minority population was our third site (
In the substance use domain, significant reductions were observed for the number of risky drinking days (the primary outcome in the clinical trial preceding this implementation study), which declined by 44% [(0.7-1.25)/1.25] from baseline to 6 months and illicit drug-use days, which declined by 34% [(2.14-3.22)/3.22]. Two of the three abstinence outcomes also showed significant improvements (any illicit drug use and/or any drink or drug). Significant effects were found for two of the three QoL scores (overall QoL and mental health).
Post hoc analyses assessed the relationship between the extent of Seva use and our study outcomes at 6 months. For each outcome, we used linear or logistic regression, controlling for the value of the outcome at baseline. System use was operationalized as (1) the total number of calendar days during the first 6 months on which individuals used Seva (going beyond the main menu), and (2) the total number of Seva pages viewed in the first 6 months (excluding the main menu). These system use measures were natural log transformed to reduce skewness. We found that participants who used Seva on more days in the first 6 months showed a significant increase in alcohol abstinence (
Whereas Seva’s quantitative results show promise in helping primary care patients remain abstinent, other important effects can be best appreciated by directly observing how mHealth affected patient care. An exchange on the Seva discussion board in the Bronx (Site 3) illustrates how the system helped patients support one another.
Mobile health apps generally have low levels of continued use—approximately 80% are abandoned after only 2 weeks [
Clinician adoption of Seva was less robust than patient adoption for two main reasons. At each clinic, clinicians worried about being responsible for data available from Seva. For example, clinicians were concerned that a patient might express suicidal thoughts in a discussion post and they would miss it. To address this concern, the clinical staff at each clinic wanted one staff member to lead the implementation and operation of Seva for the clinic. This job included monitoring patient Seva activity for the clinic and alerting clinicians of significant changes in patient status. At 2 of the 3 sites, members of the behavioral health department led the implementation and operation of Seva. At the other site, the director of innovation led implementation. Not all FQHCs have such a position. At one of the sites, the behavioral health provider who led the implementation of Seva left the clinic near the end of the patient enrollment period. Members of the research team increased their involvement at the site to pick up the slack, but staff turnover remained a vexing issue.
Baseline characteristics of participating clinics and patients.
Characteristics | Site 1 (Madison, WI) | Site 2 (Missoula, MT) | Site 3 (Bronx, NY) | |||
Medicare | 8.7 | 11.6 | 11.1 | |||
Medicaid | 56.3 | 23.1 | 46.5 | |||
Private/other | 18.5 | 19.4 | 28.9 | |||
Not insured | 16.5 | 45.9 | 13.5 | |||
eHealtha records | Epic | eClinicalworks | Epic | |||
Services offered | Primary care and mental health | Primary care, mental health, and addiction | Primary care and mental health | |||
PCMHb designation | Level 3 (2011) | Level 3 (2014) | Level 3 (2014) | |||
Number of eligible SUDc patients | 1189 | 961 | 1076 | |||
Patients enrolled in Seva | 97 | 100 | 71 | |||
Range | 21-64 | 21-66 | 22-64 | |||
Mean (standard deviation) | 41.61 (10.95) | 42.53 (10.24) | 42.66 (11.78) | |||
Female | 52 (54) | 40 (40) | 36 (51) | |||
Alcohol | 34 (35) | 44 (44) | 27 (38) | |||
Opiates | 31 (32) | 14 (14) | 8 (11) | |||
Crack cocaine | 9 (9) | 3 (3) | 11 (16) | |||
Marijuana | 1 (1) | 4 (4) | 16 (23) | |||
Methamphetamine | 0 (0) | 15 (15) | 1 (1) | |||
Multiple drugs | 22 (23) | 20 (20) | 8 (11) | |||
Hispanic/Latino | 1 (1) | 2 (2) | 35 (49) | |||
White | 68 (70) | 90 (90) | 24 (33) | |||
African American/Black | 30 (31) | 2 (2) | 35 (50) | |||
American Indian or Alaskan Native | 4 (4) | 8 (8) | 1 (2) | |||
Asian or Pacific Islander | 0 (0) | 1 (1) | 0 (0) | |||
Other | 0 (0) | 2 (2) | 15 (21) |
aeHealth: electronic health.
bPCMH: patient-centered medical home. Three levels of recognition exist, based on practice sites meeting six standards. Level 3 clinics have the best adherence to the standards.
cSUD: substance use disorder.
dPercentages do not add to 100 because patients could select more than one race.
Effectiveness results, continuous patient outcomes
Measures | Sample size | Baseline | 6 months | Za,b ( |
Effect size |
|
N | Mean (SD) | Mean (SD) | ||||
Any drinking days | 207 | 2.53 (6.01) | 1.67 (4.69) | -2.304 (.02) | -0.228 | |
Risky drinking daysd | 207 | 1.25 (3.78) | 0.70 (2.58) | -2.008 (.4) | -0.199 | |
Illicit drug-use days | 206 | 3.22 (7.57) | 2.14 (6.55) | -2.499 (.01) | -0.248 | |
Overall QoL | 202 | 28.47 (6.46) | 30.03 (7.11) | 3.653 (<.001) | 0.370 | |
Physical subscale QoL | 206 | 13.20 (3.01) | 13.48 (3.11) | 1.682 (.09) | 0.167 | |
Mental subscale QoL | 204 | 9.75 (2.99) | 10.77 (3.50) | 3.892 (<.001) | 0.393 | |
No. of hospitalizationsd | 207 | 0.43 (1.03) | 0.22 (0.65) | -3.357 (.001) | -0.335 | |
No. of ERf visitsd | 207 | 1.10 (2.79) | 0.75 (1.31) | -1.911 (.06) | -0.189 |
aZ, provided in the Wilcoxon sign test, is the standard normal distributed
bFor
cCalculated from effect size
dRisky drinking days, hospitalizations, and ER visits: Those who reported no such events were coded with zero in the number of days of these events.
eQoL: quality of life.
fER: emergency room.
Effectiveness results, dichotomized patient outcomes
Measures | Sample size | Baseline | 6 months | Chi-square ( |
Odds ratioa | |
N | n (%) | n (%) | ||||
Any drink (Yes) | 207 | 64 (30.9) | 51 (24.6) | 3.2 (.07) | 0.552 | |
Illicit drug use (Yes) | 206 | 63 (30.6) | 36 (17.5) | 14.38 (<.001) | 0.270 | |
Any drink or drug (Yes) | 206 | 97 (47.1) | 69 (33.5) | 12.57 (<.001) | 0.349 | |
Currently receive other addiction treatments (Yes) | 207 | 89 (43) | 78 (37.7) | 1.639 (.20) | 0.694 | |
HIV risky behavior (Yes) | 207 | 76 (36.7) | 65 (31.4) | 1.818 (.18) | 0.667 | |
HIV testing (Yes) | 206 | 81 (39.3) | 116 (56.3) | 33.03 (<.001) | —b |
aNumbers lower than 1 mean reductions of the events from baseline to 6 months.
bThe odds ratio for this variable cannot be calculated because patients’ HIV testing status was considered current at 6 months if they had been tested at baseline; that is, there were zero patients considered tested at pretest and not tested at 6 months.
Exchange among Seva patients in the Bronx.
A second unexpected development affected clinician adoption: We never succeeded in incorporating Seva data into the EHR at each clinic. The interoperability of EHRs is a widely recognized problem. In fact, the same system (eg, Epic Systems, developed by Epic Systems Corporation) may function very differently at one clinic versus another. The technical challenge in getting Seva data into the EHR of 3 health systems proved insurmountable in the context of this implementation research project. Instead, clinicians who wanted to review Seva data had to go to a website outside the EHR.
At each site, the leaders of Seva implementation and operation logged into Seva regularly during the 24 months of enrollment and implementation. At Site 1, 2 other staff clinicians besides the clinic Seva leader from behavioral health were most involved with Seva. The 3 clinicians at Site 1—one primary care physician and 2 behavioral health staff members—logged in an average of 0.6, 0.5, and 1.0 days per week. The Seva clinic leaders at Sites 2 and 3 were also the main users of Seva. The director of innovation at Site 2 logged in an average of 1.7 days per week, and a member of the behavioral health group at Site 3 logged in an average of 4.1 days per week. At two sites, behavioral health providers made most referrals of patients to Seva: 92% (89/97) of referrals at Site 1 and 76% (54/71) at Site 3. At Site 2, on the other hand, where we observed especially strong leadership and operational support for integrating behavioral with medical health, medical providers made 92% (92/100) of referrals (see
Percentage of patients who logged onto Seva at least once per study week. Patients were excluded from analysis at the point when they were removed from the study (eg, if they lost their phone, died, or were incarcerated).
Adoption and implementation outcomes.
Measures | Site 1 |
Site 2 |
Site 3 |
|
Total number of Seva patients | 97 | 100 | 71 | |
Total number of primary care clinical staff at sitea | 70 | 74 | 27 | |
Number of patients referred to Seva by primary care clinical staff out of total number of Seva patients, n (%) | 8 (8) | 92 (92) | 17 (24) | |
Total number of behavioral health providers at siteb | 10 | 8 | 11 | |
Number of patients referred to Seva by behavioral health providers out of total number of Seva patients, n (%) | 89 (92) | 8 (8) | 54 (76) | |
Preimplementation (12 months planned) | 12 | 12 | 18 | |
Implementation (12 months planned) | 18 | 13 | 16 | |
Maintenance (12 months planned) | 2 | 4 | 1 | |
Preimplementation (% of monthly implementation plan milestones reached) | 12 (100) | 12 (100) | 12 (100) | |
Implementation (% of 100 patient enrollment goal at each site) | 97 (97) | 100 (100) | 71 (71) | |
Maintenance (% of 12 monthly coaching follow up calls completed) | 2 (17) | 4 (33) | 1 (8) |
aClinical staff members were physicians, residents, physician assistants, nurse practitioners, registered nurses, licensed practical nurses, and medical assistants.
bBehavioral health providers were licensed medical social workers, licensed mental health counselors, licensed clinical social workers, doctorate-level psychologists, and physicians.
Cost analysis (US $).
Type of costs | Site 1 (Madison, WI) | Site 2 (Missoula, MT) | Site 3 (Bronx, NY) |
Operating costs (patient and clinic) | 113,636 | 117,150 | 83,177 |
Implementation costs (clinic) | 9948 | 23,345 | 28,121 |
Total costs for clinic | 123,584 | 140,495 | 111,298 |
Cost/patient | 1274 | 1405 | 1568 |
Our implementation plans and their execution arose from baseline assessments of readiness for implementation, sustainability potential, and technology acceptance by clinic staff members who worked on the implementation plan, as well as the information that researchers gained from directly interacting with clinic staff. The plans for implementation involved using an organizational coach to help clinics through the 4 phases (initiate, prepare, improve, and implement), using an initial in-person visit and then monthly phone calls between the coach and the clinic change team. Although the organizational coach was heavily involved in the first two phases, another member of the research team who was most familiar with operating Seva became the primary contact for the clinics starting with the third phase. This researcher monitored the Seva discussion groups and gave hands-on, practical advice in response to questions from clinicians. Instead of having monthly coaching calls, clinician-researcher contact consisted mostly of short, frequent, ad hoc phone and email communications between the main clinician users and the research team’s site coordinator in response to specific patient issues and technical questions. These were the types of problems clinics wanted help in solving, rather than helping make organizational change, which was the primary type of help offered by the coach. Thus, we adapted our implementation plan at each site to remove monthly coaching calls during the implementation phase and instead focused on the enrollment of patients as our primary implementation goal.
Maintenance, defined as continued use of Seva after the 12 months during which patient phones and data plans were paid for, was low at all 3 sites. Patients were allowed to keep their phones and could continue accessing Seva by using Wi-Fi or paying for service themselves with a new phone number. Although we did not track whether patients paid for new data plans independently, we did track Seva use after the 12 months of paid use. Use after 12 months declined gradually to zero once the last recruited patient reached the end of the 12-month period of paid phone use. The decline in use limited our ability to collect follow-up patient surveys and led us to abandon our attempts to collect the phone survey we planned to collect 6 months after each patient’s 12-month intervention period to gauge maintenance. The decline in clinician use of Seva mirrored patient decline. That is, as fewer patients used the system, or if only a few later-enrolled patients were using it, clinicians logged in less because Seva had little patient activity for them to see.
Sites did not continue to use Seva after the study for at least two reasons. First, we could not resolve issues related to transitioning from a research study to ongoing use of an mHealth system. In particular, establishing procedures for consenting patients who wanted to use their own phones to access Seva outside the research protocol, and commingling these patients with patients from the research study, proved challenging. Second, the National Institute of Health grant funding ended, and none of the 3 clinics made arrangements to pay for mobile phones and data plans afterwards. Offering Seva only to patients who can cover their own mobile phone costs could have been a condition of eligibility, but this choice would have shifted the cost to patients and restricted access for low-income patients. It may also have reduced patients’ motivation to use the system, because patients reported that receiving a mobile phone was a strong incentive to use Seva.
Without ongoing funding for patients to use the system, use of Seva declined significantly and patients ultimately became unreachable to the research staff (note that patient surveys were administered over the phone, using the number associated with the patient’s mobile phone). This led us to abandon our attempts to collect the phone survey we planned to collect 6 months after each patient’s 12-month intervention period to gauge maintenance.
This analysis addresses cost from the perspective of an FQHC clinic administrator—a comprehensive economic analysis of the study will be reported separately. This analysis includes operating costs (eg, mobile phones, data plans, clinician time for monitoring Seva use, information technology [IT] staff time) and the costs of executing the implementation strategy (eg, site visits, coaching calls). Costs were tracked for 36 months across each project phase (preimplementation, implementation, and maintenance). Costs are broken down as follows: system operating costs; implementation costs per clinic; and overall cost per patient and per clinic (see
Total cost per clinic averaged approximately US $124,000 across the 3 clinics; cost per patient averaged approximately US $1,400. For comparison, the average cost for an episode of outpatient addiction treatment among 21 addiction treatment programs surveyed in 2008 (the year with the most recent available cost data) was US $2325 [
To summarize our findings related to the three research questions that were the focus of the study:
The study offers the following 4 lessons about implementing an mHealth system in primary care:
First, plan and have a budget for working extensively with clinic IT staff to integrate mHealth data into the EHR. This very challenging task is essential for integrating mHealth into primary care because it makes the mHealth data part of rather than separate from the data clinicians expect to see as they treat patients. Second, work with clinic staff to figure out how the mHealth system will fit into the clinic’s existing workflow. All 3 clinics in this study chose to appoint one clinician to monitor Seva data and alert fellow clinicians as needed about important changes in a patient’s recovery. Wider and deeper integration would result from each clinician routinely monitoring data from the mHealth system for his or her patients, just as he or she monitors data related to other chronic conditions such as diabetes. This level of involvement may be unrealistic, however, given the time pressures on primary care staff, and assigning routine monitoring to other clinical staff has been done effectively in other studies we have conducted [
Patient use was exceptionally high compared with continued use of most mHealth apps, although patient use declined steeply after funding for the phones and data plans ended. Clinician use was low compared with patient use because, as stated above, clinicians worried about being responsible for data from Seva and they had to view Seva data in a separate website rather than in the EHR. Of the two aspects of integration we examined—the integration of behavioral health into primary care, and the integration of mHealth into the treatment of addiction in primary care—the second was more successful than the first. Although the treatment of patients suffering from addiction and the use of Seva remained mainly the province of behavioral health providers in this study, the integration of mHealth into addiction treatment was successful if judged by the high levels of patient use.
This study showed the potential of patient peer support in encouraging treatment adherence. Patient peer support is unusual in primary care. It is also a type of care that does not add to, and may reduce, clinician burden. Clinicians who used Seva were generally enthusiastic about it, as demonstrated by 2 clinics wanting to continue using the system after grant funding ended, even though these intentions have thus far gone unfulfilled. The number of clinicians involved, though, is too small to warrant generalizations.
This study—which involved 3 unaffiliated primary care clinics enrolling nearly 300 patients in the same mHealth system—is the most comprehensive implementation research trial focused on the use of mHealth in primary care yet conducted in the US health care system. Implementation research on mHealth has been focused almost entirely on developing countries in Africa, where mHealth is usually used as a replacement for standard health care and operates independently of any health care system [
Prior studies focused on providing addiction treatment to primary care patients have also been rare; we found only 2 clinical trials for primary care SUD interventions [
We observed significant reductions in drinking and substance use among patients using Seva. These are promising reductions compared with other outpatient AUD treatments. For example, Project MATCH showed that psychosocial treatments for alcoholism are not particularly effective [
Granted, this study was an implementation research trial not specifically designed to retest the effectiveness of Seva. A rigorously designed randomized controlled trial would be required to definitively demonstrate the effectiveness of Seva within a substance-using primary care population.
The study has some limitations. (1) The primary outcome, risky drinking days, is a self-reported measure. (2) Selection bias could have affected which patients participated in the study. Clinicians were advised to enroll any eligible patients they thought might benefit from Seva, based on their clinical judgment. It was not possible for us to tell if this led clinicians to favor some patients over others (eg, those with more education). (3) We could not retrieve racial and ethnic data for SUD patients who were eligible to participate in the study but were not enrolled, and we did not collect data about education and socioeconomic status on Seva patient surveys, limiting our understanding of the representativeness of the patient sample. (4) We lost an opportunity to learn about sustainability from the patient’s perspective because we could not reach patients for the survey we planned to collect 6 months after each patient’s 12-month intervention period. (5) The study would be hard to replicate because it reports on a mobile phone app that, like almost all such apps, changes over time rather than remaining stable. For example, since the start of this study, the app has been rebuilt to update it to current accessibility and security standards, add features (My Motivation), improve features (the automatic tailoring of feedback in response to weekly surveys), and add content for opioid addiction.
The current findings suggest that mHealth faces several formidable challenges to widespread implementation. The first is integrating mHealth with EHRs (eg, Epic Systems), described above. Clinicians in this study had to view Seva data by going to a website outside the EHR, requiring already overburdened clinicians to make extra mouse-clicks. In addition, logistical and patient privacy concerns limited our ability to use EHR data for evaluation purposes. For example, we could not obtain participants’ attendance at primary care visits, which we planned to include in the health care utilization analyses. The poor interoperability and accessibility of EHRs are likely to generalize to other researchers and mHealth developers, making research challenging to conduct and limiting the usefulness of mHealth systems for clinicians.
Another key challenge relates to the difficulty of enrolling patients with SUDs. Clinics reported that it often took multiple follow-up calls to get patients into the clinic to get informed consent and conduct training, even though patients were often excited about getting a mobile phone. The effort required to enroll patients might have dampened enthusiasm for enrollment beyond the first 100 patients that the budget provided for.
FQHCs are required by law to provide behavioral health services. We do not know how Seva might function in a primary care clinic without designated behavioral health staff. Although Seva was somewhat integrated into behavioral health, it did not deeply penetrate physicians' treatment of patients, as reported previously about Site 1 [
Finally, our experience has revealed several fundamental questions about the role of mHealth in primary care: Is the value of patient peer support for behavioral health sufficient to make the costs of embedding an mHealth system such as Seva into a clinic’s operations worthwhile? Might it instead suffice for patients to use mHealth systems on their own, based upon the recommendation of primary care clinicians? If integrating mHealth into clinic systems is deemed worthwhile, who bears the costs? Certain costs, such as those for mobile phones and data plans, could be borne by patients, some of whom already pay for phones and data plans out of pocket. In this study, patient use of Seva and survey follow-up rates dropped significantly when research funding stopped paying for data plans. Yet, these costs account for only about half the total cost of the system, and lower-income patients often have pay-as-you-go data plans that may not work with the data requirements of an mHealth system such as Seva. Volunteer peer mentors could potentially act as monitors, as clinicians and researchers did in this project, thereby reducing the cost to clinics. Indeed, similar volunteer roles are essential to the Alcoholics Anonymous model (eg, “sponsors”).
mHealth has the potential to transform health care, and given the enormous cost of health care, we need to make effective use of every available resource. In contrast to the seemingly inexorable rise in health care costs over time, the cost of technology tends to decrease in accordance with Moore’s Law, which posits that computing capability roughly doubles every 2 years. Our experience illustrates that mHealth can engage patients suffering from addiction in ways that benefit patients without adding substantial burden on health care providers. Although challenges remain, thoughtful deployment of mHealth could improve the treatment of addiction in primary care and might also improve the treatment of other chronic conditions that have prominent behavioral components (eg, diabetes). In so doing, mHealth could transcend the physically local and professionally controlled systems that characterize the US health care system.
electronic health record
emergency room
federally qualified health center
information technology
mobile health
quality of life
Reach, Effectiveness, Adoption, Implementation, and Maintenance
This research was funded by the National Institute on Drug Abuse grant number R01-DA034279. The funder had no role in any aspect of the development, conduct, analysis, or reporting of the study. The authors wish to thank the dedicated clinicians and staff members at the 3 clinics where the study was conducted, in particular, Chantelle Thomas from Access Community Health Center in Madison, WI; Virna Little, Victoria Ward, and Tom McCarry from the Institute for Family Health, Center for Counseling, at Walton, Bronx, NY; and Mary Jane Nealon from Partnership Health Center in Missoula, MT. The project could not have succeeded without their dedication and support. We also thank Judith Ganch, Joseph Glass, Andrew Isham, Adam Maus, Helene McDowell, Klaren Pe-Romashko, and Victoria Ward for their important contributions to the project.
Authors DG, FM, RJ, and AQ have a shareholder interest in CHESS Mobile Health, a public benefit corporation that disseminates technology to the specialty addiction treatment system. CHESS Mobile Health did not develop Seva. The relationship between the authors and CHESS Mobile Health is managed by the authors and the University of Wisconsin–Madison’s Conflict of Interest Committee. LM is affiliated with HealthSim, LLC—a small business that developed the Web-based Therapeutic Education System (TES). The relationship is extensively managed by LM and her academic institution. All other authors declare that they have no competing interests.