This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.
Knowledge about user experiences may lead to insights about how to improve treatment activity in Internet-based cognitive behavioral therapy (iCBT) to reduce symptoms of depression and anxiety among people with a somatic disease. There is a need for studies conducted alongside randomized trials, to explore treatment activity and user experiences related to such interventions, especially among people with older age who are recruited in routine care.
The aim of the study was to explore treatment activity, user satisfaction, and usability experiences among patients allocated to treatment in the U-CARE Heart study, a randomized clinical trial of an iCBT intervention for treatment of depression and anxiety following a recent myocardial infarction.
This was a mixed methods study where quantitative and qualitative approaches were used. Patients were recruited consecutively from 25 cardiac clinics in Sweden. The study included 117 patients allocated to 14 weeks of an iCBT intervention in the U-CARE Heart study. Quantitative data about treatment activity and therapist communication were collected through logged user patterns, which were analyzed with descriptive statistics. Qualitative data with regard to positive and negative experiences, and suggestions for improvements concerning the intervention, were collected through semistructured interviews with 21 patients in the treatment arm after follow-up. The interviews were analyzed with qualitative manifest content analysis.
Treatment activity was low with regard to number of completed modules (mean 0.76, SD 0.93, range 0-5) and completed assignments (mean 3.09, SD 4.05, range 0-29). Most of the participants initiated the introduction module (113/117, 96.6%), and about half (63/117, 53.9%) of all participants completed the introductory module, but only 18 (15.4%, 18/117) continued to work with any of the remaining 10 modules, and each of the remaining modules was completed by 7 or less of the participants. On average, patients sent less than 2 internal messages to their therapist during the intervention (mean 1.42, SD 2.56, range 0-16). Interviews revealed different preferences with regard to the internet-based portal, the content of the treatment program, and the therapist communication. Aspects related to the personal situation and required skills included unpleasant emotions evoked by the intervention, lack of time, and technical difficulties.
Patients with a recent myocardial infarction and symptoms of depression and anxiety showed low treatment activity in this guided iCBT intervention with regard to completed modules, completed assignments, and internal messages sent to their therapist. The findings call attention to the need for researchers to carefully consider the preferences, personal situation, and technical skills of the end users during the development of these interventions. The study indicates several challenges that need to be addressed to improve treatment activity, user satisfaction, and usability in internet-based interventions in this population.
Symptoms of depression and anxiety are common following a myocardial infarction [
Typically, guided iCBT uses a written treatment material and internet-based synchronous or asynchronous communication with a therapist [
Treatment acceptability may be defined as the extent intended users perceive a given intervention as reasonable, justified, fair, and palatable [
The overall aim of this study was to explore treatment activity, user satisfaction, and usability experiences among patients allocated to treatment in the U-CARE Heart study, a randomized clinical trial of an iCBT intervention for treatment of depression and anxiety following a recent myocardial infarction (unpublished data, 2018; [
What was the treatment activity with regard to completed modules, completed assignments, and therapist communication initiated by the participants?
What positive and negative experiences of the intervention, as well as suggestions for improvement, did the participants describe?
This study was conducted alongside the U-CARE Heart study. The results from the randomized controlled trial (RCT) indicate no differences between the groups in symptoms of depression and anxiety after intervention [
The U-CARE Heart study used an internet-based portal to deliver an iCBT intervention tailored for patients with a recent myocardial infarction. A two-factor authentication solution with a password and numerical short message service (SMS) verification was required to log on to the portal. The design of the portal included a side bar and a menu bar, accessible from all pages. A short presentation and pictures of the therapist who worked in the program was provided in the “About us” section.
Sitemap of the internet-based portal.
The treatment program consisted of 11 modules. Each module consisted of 2 to 4 steps. Each step contained 1 or 2 assignments, such as self-monitoring or registration of skills training. The treatment material consisted of PDF files with psychoeducation. The average word count per module was 6739.91 (SD 2786.79). Participants were encouraged to work with one step per week during the 14-week treatment period. The first introductory module was mandatory and oriented the user to the portal and the treatment program through an instructional video, psychoeducation about CBT, and common reactions post myocardial infarction. Thereafter, participants were invited to read a short description of the available modules, before choosing which modules to work with. Participants were limited to work with 2 active modules simultaneously. Supplementary material and video clips of interviews conducted with patients about their experience of depression and anxiety after a myocardial infarction were available throughout the course of treatment in an additional module called the Library. Participants also had access to a discussion board where they could communicate with other participants.
Each patient was assigned 1 of 3 therapists, who could be contacted any time. Therapists provided asynchronous written feedback on assignment via an internal message function within 24 hours. After completing all steps in a module, approval from a therapist was needed to activate a new module. Participants inactive for more than 1 week were reminded to stay active via phone calls. Participants unable to be reached were reminded by prompts sent via SMS.
Clinical psychologists and experts in IT solutions developed a preliminary version of the intervention. This version was evaluated through face-to-face think-aloud sessions [
Patients were consecutively recruited from 25 cardiac clinics in Sweden. To be eligible, patients needed to: (1) be younger than 75 years, (2) have a medical history of a recent myocardial infarction less than 3 months prior, and (3) report a score >7 on either the depression or anxiety subscale in the Hospital Anxiety and Depression Scale (HADS) [
Participants (n=69) allocated to the treatment arm, between June 2015 and October 2016, were eligible to participate in a follow-up telephone interview (
The majority of the participants in the randomized trial were males, employed, living in a relationship, born in Sweden, had no children in their household, and did not receive any current counseling (
User activity was automatically registered through the U-CARE internet portal. Number of completed modules and assignments, and internal messages sent from patients to therapists, were used as quantitative measures of treatment activity.
The fourth author (GB) conducted individual telephone interviews with the aid of a semistructured interview guide (
Quantitative data regarding number of completed modules, assignments, and therapist communication initiated by participants were analyzed with descriptive statistics using R version 3.2.2 (R Foundation for Statistical Computing).
The interviews were analyzed with inductive qualitative manifest content analysis, inspired by the outline presented by Graneheim and Lundman [
Recruitment of participants. HADS-A: Hospital Anxiety and Depression Scale-Anxiety; HADS-D: Hospital Anxiety and Depression Scale-Depression.
Baseline demographic and clinical characteristics of participants. Between-group comparisons are conducted between participants not interviewed and interviewed. Categorical data is analyzed with Fisher exact test and continuous data is analyzed with Welsh
Characteristics | Allocated to intervention (n=117) | Not interviewed (n=96) | Interviewed (n=21) | ||
Age in years, mean (SD) | 58.37 (8.98) | 58.68 (8.67) | 56.95 (10.38) | .48 | |
Female | 44 (37.6) | 37 (39) | 7 (33) | .80 | |
Male | 73 (62.4) | 59 (62) | 14 (68) | ||
Employed | 78 (66.7) | 62 (65) | 16 (76) | .44a | |
Unemployed | 4 (3.4) | 4 (4) | 0 (0) | ||
Retired | 33 (28.2) | 28 (29) | 5 (24) | ||
Sick leave | 2 (1.7) | 2 (2) | 0 (0) | ||
Elementary | 22 (18.8) | 19 (20) | 3 (14) | .03b | |
High-school | 45 (38.5) | 41 (43) | 4 (19) | ||
University <3 years | 24 (20.5) | 18 (19) | 6 (29) | ||
University >3 years | 26 (22.2) | 18 (19) | 8 (38) | ||
Single | 18 (15.4) | 15 (16) | 3 (14) | >.99 | |
In relationship | 99 (84.6) | 81 (84) | 18 (86) | ||
Sweden | 96 (82.1) | 81 (84) | 15 (71) | .21 | |
Other | 21 (17.9) | 15 (16) | 6 (29) | ||
Yes | 43 (36.8) | 33 (34) | 10 (48) | .32 | |
No | 74 (63.2) | 63 (66) | 11 (52) | ||
Yes | 30 (25.6) | 24 (25) | 6 (29) | .78 | |
No | 87 (74.4) | 72 (75) | 15 (71) | ||
HADS-Ac | 10.27 (2.94) | 10.39 (3.11) | 9.76 (2.00) | .25 | |
HADS-Dd | 7.97 (3.15) | 8.20 (3.26) | 6.95 (2.42) | .05 |
aEmployed versus other.
bStudied at university versus didn't study at university.
cHADS-A: Hospital Anxiety and Depression Scale-Anxiety.
dHADS-D: Hospital Anxiety and Depression Scale-Depression.
Of all participants allocated to intervention, 113 (96.6%, 113/117) initiated the introduction module, which was completed by 63 (53.9%, 63/117). Each of the remaining modules was completed by 7 or less of the participants.
Number of participants in the randomized controlled trial (n=117) who initiated and completed the respective modules in the treatment program.
Module | Initiated, n (%) | Completed, n (%) |
Introduction | 113 (96.6) | 63 (53.9) |
Managing worry | 23 (19.7) | 7 (6.0) |
Applied relaxation training | 28 (24.0) | 5 (4.3) |
Behavioral activation | 16 (13.7) | 4 (3.4) |
Fear and avoidance post myocardial infarction | 7 (6.0) | 3 (2.6) |
Cognitive restructuring | 11 (9.4) | 2 (1.7) |
Coping with insomnia | 6 (5.1) | 2(1.7) |
Problem solving | 4 (3.4) | 2 (1.7) |
Relapse prevention depression and anxiety | 3 (2.6) | 1 (0.9) |
Communication skills | 7 (6.0) | 0 (0.0) |
Values in life | 3 (2.6) | 0 (0.0) |
Total number of completed modules, completed assignments, and messages sent to therapist among the participants allocated to the intervention in the randomized controlled trial (n=117).
Number of completed modules, assignments, and sent internal messages at end of treatment period | Number of participants who completed modules, n (%) | Number of participants who completed assignments, n (%) | Number of participants who sent messages to therapist, n (%) |
0 | 54 (46.2) | 30 (25.6) | 66 (56.4) |
1 | 45 (38.5) | 21 (17.9) | 21 (17.9) |
2 | 14 (12.0) | 23 (20.5) | 7 (6.0) |
3 | 1 (0.9) | 8 (6.8) | 6 (5.1) |
4 | 2 (1.7) | 2 (3.4) | 4 (3.4) |
5 | 1 (0.9) | 14 (12.0) | 3 (2.6) |
> 5 | 0 (0.0) | 19 (16.2) | 10 (8.5) |
A minority of participants completed additional modules beyond the introductory module (18/117, 15.4%), completed more than 5 assignments (19/117, 16.2%), and sent more than 5 messages to the therapist (10/117, 8.5%; see
The mean number of completed modules, completed assignments, and messages sent to therapist did not reach above 0.6 at any of the 14 treatment weeks. The total summed range for all 14 weeks was 0 to 5 for completed modules, 0 to 29 completed assignments, and 0 to 16 messages sent to therapist (
We identified 4 main categories: (1) the portal, (2) the treatment program, (3) the therapist communication, and (4) the personal situation and required skills (
Total number of completed modules, assignments, and messages sent to therapist during the 14-week treatment period.
Summary of positive and negative experiences described in interviews.
Category and subcategory | Findings | ||
Positive experiences | Negative experiences | ||
Design | Appealing interface with easy navigation |
Navigational difficulties, unfamiliar interface |
|
Usability | Easy and secure log-in procedure |
Complicated log-in procedure with technical failures Required desktop or laptop, issues when using mobile device Cumbersome to open PDF files |
|
Content of treatment material | Relevant, well-written, and useful information |
Irrelevant outdated material and posts in discussion board Repetitive material with poor readability |
|
Working with the material | Manageable difficulty, approach gave time to reflect Time flexibility, possibility to select modules |
Strenuous, tedious, difficult, and time-consuming work Too intensive work, restrictions in active modules felt rigid |
|
Treatment period | Deadline promoted activity toward end of treatment |
Treatment duration and time to work with modules was too short |
|
Therapist feedback | Tailored, available, and rapid feedback Telephone conversations with therapist Reminders were useful prompt to log in |
Lack of and irrelevant therapist feedback Aversive and stressful reminders |
|
Internet-based communication | Preference for verbal and synchronous communication Communication felt impersonal and involved a risk of misunderstanding |
||
Unpleasant emotions evoked by the intervention | Bad conscience and guilt for being inactive Treatment rekindled difficult memories and emotions Fear of making mistakes |
||
Lack of time | Lack of time because of everyday life Poor timing of treatment |
||
Responding to outcome measures in questionnaires | Questionnaires were difficult to understand, felt repetitive, and irrelevant Strenuous work with questionnaires |
||
Technical aspects | Insufficient computer literacy Intervention required time in front of a computer Lack of Internet connection |
Portal
Remove the completed modules to facilitate navigation
Include the possibility to have several windows open at the same time
Make the portal available via CD-ROM and as an app for mobile devices
Treatment program
Have less focus on depression and anxiety following a myocardial infarction
Include the possibility to ask medical questions to health professionals and other participants in the portal
Include information concerning how to communicate with children
Use easy-to-read language
Use closed-ended questions with predetermined alternatives in the treatment program
Prolong the treatment period and allow longer time for work with modules that feel relevant for the patient
Make the program feel more fun for the intended users
Therapist communication
Offer synchronous verbal therapist communication, via telephone calls
Offer more therapist feedback in decision making concerning which modules to work with
Use audio or video recordings of therapist feedback
Entitle the patient with their name instead of username
Include picture of the therapist in all conversations
Personal situation and required skills
Individualize the outcome questionnaires
Make the outcome questionnaires easier to understand
Allow participants to access previous responses in the outcome questionnaires
Offer access to treatment closer in time to the infarction
Our study focused on treatment activity and user experiences of an iCBT intervention to reduce symptoms of depression and anxiety among adults with a recent myocardial infarction. The results show that treatment activity was low with regard to completed modules and assignments and submitted internal messages to therapists. Various positive experiences, negative experiences, and suggestions for improvements were described in follow-up interviews related to the internet-based portal, treatment program, therapist communication, as well as the personal situation and required skills of the participants. Previous research shows inconclusive and variable results concerning treatment activity and user satisfaction of iCBT. Although some studies report high levels of adherence and sufficient treatment satisfaction [
Although iCBT shows promise as a mode of treatment for symptoms of depression and anxiety [
The observed low treatment activity and described negative experiences related to design and usability call attention to what has been described as a risk of distress and frustration when faced with technological difficulties [
Tailored interventions have the potential to successfully meet patient preferences by providing them with the choice of which treatment modules to work with [
In this study, there are methodological limitations that should be taken into consideration. The sample may not fully represent the population of patients with symptoms of depression and anxiety after a recent myocardial infarction. Patients were recruited in routine care at 25 Swedish cardiac clinics. Only patients below 75 years of age were invited to participate in the randomized trial. This may limit the generalizability and transferability with regard to older patients. The majority of the participants in the trial were males, employed, living in a relationship, and born in Sweden. Furthermore, only a subsample of those who took part in the intervention was interviewed. The reason for this was mainly practical, as we lacked necessary resources to collect qualitative data in the early stages of the study. This may imply a source of selection bias that may impact the results. We acknowledge that the qualitative results only reflect the experiences of a proportion of the whole sample in the RCT. Although the sample characteristics for participants in the RCT were represented in the sample of participants included in the follow-up telephone interviews, none of the participants who were unemployed or on sick leave were included in the follow-up interviews. Furthermore, a higher proportion of the interviewed participants had studied at a university compared with those who were not interviewed. This may imply a limited transferability to participants with lower levels of education. For example, it is possible that participants with experience of university studies may be more comfortable with text-based material and communication. Moreover, we did not collect any quantitative measure of computer literacy. Thus, we cannot make any claims about the actual computer literacy among the participants in our sample.
The data collection and analysis of the qualitative material may not fully represent the experiences of the interviewees. One psychologist who was not involved as a therapist in the treatment program conducted telephones interviews. Telephone interviews reduce the risk for socially desirable answers, may lead to increased sense of anonymity, and have the potential to make participants feel more comfortable [
The findings indicate a need for rigorous preparations before conducting iCBT interventions for adults with depression or anxiety after a recent myocardial infarction. There is a need for future research that investigates ways to ensure that development of these interventions is more adapted to the intended end users. The low treatment activity and negative experiences related to the use of the internet platform and the treatment content call attention to the importance of usability and feasibility trials. Future research should investigate patient, therapist, and treatment-related factors to improve treatment activity in internet-based interventions implemented in this population.
Patients with symptoms of depression and anxiety after a recent myocardial infarction showed low treatment activity in guided iCBT with regard to completed modules, assignments, and messages sent to their therapist. They describe various negative experiences and suggestions for improvement, calling attention to the need for researchers to carefully consider the preferences, personal situation, and required skills of the end users during the development of these interventions. The findings indicate several challenges that need to be addressed to improve treatment activity, user satisfaction, and usability of internet interventions in this population.
Screenshot of the U-CARE Heart portal.
Description of development process.
Interview guide.
Examples of the steps in the qualitative analysis.
Backgrounds of the researchers involved in data collection and qualitative analysis.
Completed modules, completed assignments, and messages sent to therapist for each treatment week of the intervention.
An expanded presentation of the qualitative findings with illustrative quotes from interviews.
Hospital Anxiety and Depression Scale
Hospital Anxiety and Depression Scale-Anxiety
Hospital Anxiety and Depression Scale-Depression.
Internet-based cognitive behavioral therapy
Montgomery Asberg Depression Rating Scale Short form
randomized controlled trial
short message service
This study is part of U-CARE strategic research environment funded by the Swedish Research Council (dnr 2009–1093). U-CARE Heart is also funded by the Swedish Heart and Lung Association, the Uppsala-Örebro Regional Research Council, Swedish Research Council for Health, Working Life, and Welfare (dnr 2014–4947), and the Vårdal foundation (dnr 2014–0114). The funders had no involvement in the study conception, data collection, analysis, and manuscript production.
EW, FN, and GB conceived and designed the study. GB collected the data. EM and TC analyzed the data and drafted the manuscript. FN, EO, CH, and GB read and revised the draft. All authors approved the final version of the manuscript.
None declared.