The study was a multisite, assessor-masked, parallel-group RCT with a 1:1 allocation ratio.

A detailed description of the study protocol has been reported elsewhere [40] and is attached as Multimedia Appendix 1. The study, which was approved by the ethics committees of the participating centers and registered in the Japanese clinical trials registry (UMIN CTR 000013693), took place in 20 psychiatric clinics and hospitals across Japan between September 2014 and October 2016. Multimedia Appendix 2 provides the complete list of trial sites and investigators.

Eligible participants (1) were aged between 25 and 59 years, (2) had a primary diagnosis of major depressive disorder without psychotic features according to the Diagnostic and Statistical Manual of Mental Disorders (Fifth Edition) [41] as ascertained by using the Primary Care Evaluation of Mental Disorders procedure [42], (3) were antidepressant-resistant, defined as scoring 10 or more on the Beck Depression Inventory-II (BDI-II) [37,43] after taking one or more antidepressants at an adequate dosage for four or more weeks (stage I, II, or III according to the criteria by Thase and Rush [44]), (4) had not been prescribed escitalopram or sertraline, or received CBT or interpersonal therapy for the index episode.

The study psychiatrists introduced the trial to the potentially eligible patients from among the patients they were seeing and invited them to participate voluntarily. After full disclosure of the trial contents and procedures, all participants provided written informed consent. No advertisement through the media was used.

On entry, all participants started switching their antidepressant, had the Kokoro-app installed onto their iPhone, and had access to the welcome session, which mainly aimed at, after a brief description of CBT, training the participants in the use of the smartphone and its speech recognition. (When the participants did not own an iPhone, we lent one to them for the duration of the trial.) After this 1-week run-in, they had a telephone interview with the Patient Health Questionnaire-9 (PHQ-9) and the Frequency, Intensity, and Burden of Side Effects Ratings (FIBSER) and were classified into one of two samples. The per-protocol sample comprised those who had not or only partially responded since week 0 (PHQ-9 scores ≥5 [45] at week 1), who tolerated and adhered to the new antidepressant, and had no problem using the smartphone and Kokoro-app. The auxiliary sample included all others. The total sample (the per-protocol sample plus the auxiliary sample) would answer the real-world question of the value of smartphone CBT among the patients for whom the clinicians would initially consider prescribing the smartphone CBT, including those who would not tolerate the new medication or who may have some difficulty using the smartphone app, whereas the per-protocol sample would answer the question of the value of smartphone CBT under the narrower circumstances (ie, among patients who were able to follow the protocol and for whom the app would be expected to demonstrate its full effects).

After stratification by group, the participants were randomized 1:1 to the combined antidepressant switch plus smartphone CBT arm (intervention arm) or the antidepressant switch alone arm (control arm) using an automated Web program implementing the method of minimization. Therefore, the randomization was concealed. Clinics, number of antidepressants previously prescribed for the index episode (≥3 vs <3), and a total score of the PHQ-9 at week 1 (≥10 vs <10) were used as minimization variables.

During the telephone interview for the PHQ-9, if participants reported suicidal ideation for more than half the days or nearly every day of the previous 2 weeks, the interviewer immediately notified the staff in the central office, who then notified the responsible psychiatrist.

The intervention group received both the antidepressant switch and the smartphone CBT, whereas the control group received only the antidepressant switch during the intervention period up to week 9. Details of each intervention are described subsequently.

The masked assessors interviewed the participants by telephone with PHQ-9 and FIBSER at weeks 0, 1, 5, 9, and 17.

The primary outcome was the PHQ-9 at week 9. It consists of the nine diagnostic criteria items of major depression [47]. Each item is rated between 0=“not at all” through 3=“nearly every day.” The total score ranges between 0 and 27. The instrument has excellent reliability, validity, and responsiveness [48]. Remission was defined as scoring four or less on PHQ-9, and response as 50% or greater reduction from baseline.

Secondary outcomes included the FIBSER, which assesses the frequency, intensity, and burden of side effects, each on a 7-point scale, with a total score between 3 and 27, [49] and the BDI-II [43], a self-report measure of depression severity that asks about 21 symptoms of depression, each on a scale between 0 and 3, with the total score between 0 and 63.

The study participants and the psychiatrists in charge of medication change were aware of the treatment allocation. The outcome assessors conducting telephone interviews were unaware of allocation. The success of this masking was evaluated by calculating the Bang Index [50] of assessors’ treatment guesses after each telephone assessment. The Bang Index is scaled to an interval of –1 to 1, 1 being complete lack of blinding, 0 being consistent with perfect random guessing, and –1 indicating opposite guessing.

The study was powered to detect a moderate effect size of 0.5 in terms of standardized mean difference between the two treatments for the primary outcome at week 9, with 80% power at two-sided alpha level of .05. Assuming that 30% would leave the study or be classified into the auxiliary group at week 1, the required total sample size was 164 participants.

All analyses were undertaken according to the intention-to-treat principle, including all the randomized participants for the total sample as well as for the per-protocol sample.

For each continuous outcome up to week 9, we used a linear mixed model including sites and patients as random effects and time (5 and 9 weeks), treatment, and time*treatment interaction, adjusting for its baseline score and the stratification variables, as fixed effects. The primary endpoint was the estimate of the least squares mean difference along with the 95% confidence interval (95% CI) at week 9. For the continuous outcome at week 17 follow-up, we used the similar linear mixed model but without the time*treatment interaction because this was a one-time comparison after all the participants received the smartphone CBT both in the intervention and the control groups. For the dichotomous outcomes, we used a generalized linear mixed model with the same random effects and the fixed effects. We chose odds ratios as the measure of effect. We used SAS version 9.4 (SAS Institute Inc, Cary, NC, USA). Multimedia Appendix 1 provides the statistical analysis plan.

The statistician (QZ), blinded to allocation, conducted the statistical analyses. The writing committee reviewed a statistical report in which the two treatment arms were designated A and B, and developed interpretation of the results and associated conclusions under two different scenarios, one assuming A to be the smartphone CBT plus medication change arm and B to be the medication change alone arm, and another alternative scenario. The code was broken only after the writing committee signed off on the agreed-on interpretations (see Multimedia Appendix 4).

Figure 2 shows the flow of participants through the study. Between September 2, 2014, and July 1, 2016, 323 patients were assessed for eligibility; 166 patients provided informed consent and started medication change. Two withdrew consent before randomization at week 1. Therefore, we recruited 164 patients, of whom 117 found no difficulty with the smartphone, were adherent to the protocol treatment, remained at least moderately symptomatic, and constituted the per-protocol sample; 60 were allocated to the smartphone CBT and 57 to medication change alone. Of the remaining 47 participants in the auxiliary group, 21 were allocated to the intervention group and 26 to control. Primary outcome data at 9 weeks were obtained from all but one randomized participant (163/164, 99.4%).

Table 1 shows that the baseline demographic and clinical characteristics of the treatment groups were well balanced. Typically, patients were in their thirties to forties, had three previous episodes, were in the current episode for nearly 2 years, and were severely to moderately depressed.

In the total sample, 16 of 81 participants (20%) stopped the protocol antidepressant treatment by escitalopram or sertraline in the intervention arm, as did 14 of 83 (17%) in the medication only arm (Table 2). Some received augmentation drugs, such as antipsychotics or lithium, which were not allowed in the protocol for the antidepressant switch (n=15 in the total sample), a few had deterioration or side effects and could not continue the protocol treatment (n=5), and a few others got so well and did not want to continue with the protocol medication (n=2). The medication dosages were comparable between the arms.

For the smartphone CBT, all but one participant (80/81, 99%) completed at least half the sessions of the program, and 71 of 81 (88%) completed at least six of eight sessions. It took the patients, on average, 10.8 (SD 4.2) days to complete one session. The patients filled in a mean 11.2 (SD 11.4) “mind maps” for self-monitoring, conducted a mean 14.4 (SD 17.1) behavioral activation “personal experiments,” and generated a mean 6.1 (SD 6.0) alternative thoughts for cognitive restructuring (Table 2).

In the intention-to-treat analysis of the total sample, patients who were allocated the CBT app (n=81) scored 2.48 points (95% CI 1.23-3.72, P<.001; standardized mean difference [SMD] 0.40) lower on PHQ-9 than those who were not (n=83) at week 9 (Table 3, Figure 3). The former group also scored 4.1 points (95% CI 1.5-6.6, P=.002) lower on BDI-II. Significantly more participants showed response (OR 2.73, 95% CI 1.35-5.53, P=.005). However, the increase in remission did not reach statistical significance (OR 2.02, 95% CI 0.93-4.42, P=.08).

With regard to the harm outcomes, patients using the smartphone CBT reported somewhat less overall burden of side effects, but the difference was not statistically significant (FIBSER mean difference=–0.76, 95% CI –1.58 to 0.05, P=.07). There was one report of suicidality (self-injurious behavior without suicidal intent) in the combined treatment arm and one report of a serious adverse event in the control arm (brief hospital admission for examination of preexisting spinal canal stenosis).

In the per-protocol sample, who were comfortable with the smartphone app, were still symptomatic, and were adherent to medication with mild or less side effects after the run-in, the patients who received smartphone CBT in addition to medication change (n=60) scored mean 1.72 (95% CI 0.25-3.18, P=.02; SMD 0.28) points lower on PHQ-9 than those undergoing medication change alone (n=57) at week 9. The combined treatment arm was superior to the control arm in terms of BDI-II (difference=–3.2, 95% CI –6.3 to 0.0, P=.05, not statistically significant), but not in terms of remission (OR 1.99, 95% CI 0.74-5.38, P=.17) or response (OR 2.11, 95% CI 0.92-4.85, P=.08).

In terms of the harm outcome, the combination treatment arm reported significantly less overall burden of side effects (FIBSER mean difference=–0.75, 95% CI –1.47 to –0.03, P=.04) (Table 4, Figure 3).

The participants who had access to the smartphone app in the first 9 weeks maintained their gains for a further 8 weeks. At week 17, when the participants in the control arm also had access to the smartphone app, the results were comparable between the two groups in terms of PHQ-9, BDI-II, and FIBSER, both for the total sample and for the per-protocol sample (Tables 3 and 4).

The Bang Index of the treatment guesses by the masked assessors at weeks 5, 9, and 17 was 0.10 (95% CI –0.13 to 0.33), 0.29 (95% CI 0.06-0.51), and 0.30 (95% CI 0.08-0.52) for the intervention arm and –0.21 (95% CI –0.45 to 0.03), –0.18 (95% CI –0.41 to 0.06), and –0.30 (95% CI –0.53 to –0.07) for the control arm, respectively. The observed patterns indicate that the raters guessed the treatment allocation haphazardly or guessed it to be the smartphone CBT arm more often regardless of the actual allocation, resulting in ideally unbiased assessment of outcomes in the trial [51].