Background: The number of smartphone apps that focus on the prevention, diagnosis, and treatment of depression is increasing. A promising approach to increase the effectiveness of the apps while reducing the individual’s burden is the use of just-in-time adaptive intervention (JITAI) mechanisms. JITAIs are designed to improve the effectiveness of the intervention and reduce the burden on the person using the intervention by providing the right type of support at the right time. The right type of support and the right time are determined by measuring the state of vulnerability and the state of receptivity, respectively.
Objective: The aim of this study is to systematically assess the use of JITAI mechanisms in popular apps for individuals with depression.
Methods: We systematically searched for apps addressing depression in the Apple App Store and Google Play Store, as well as in curated lists from the Anxiety and Depression Association of America, the United Kingdom National Health Service, and the American Psychological Association in August 2020. The relevant apps were ranked according to the number of reviews (Apple App Store) or downloads (Google Play Store). For each app, 2 authors separately reviewed all publications concerning the app found within scientific databases (PubMed, Cochrane Register of Controlled Trials, PsycINFO, Google Scholar, IEEE Xplore, Web of Science, ACM Portal, and Science Direct), publications cited on the app’s website, information on the app’s website, and the app itself. All types of measurements (eg, open questions, closed questions, and device analytics) found in the apps were recorded and reviewed.
Results: None of the 28 reviewed apps used JITAI mechanisms to tailor content to situations, states, or individuals. Of the 28 apps, 3 (11%) did not use any measurements, 20 (71%) exclusively used self-reports that were insufficient to leverage the full potential of the JITAIs, and the 5 (18%) apps using self-reports and passive measurements used them as progress or task indicators only. Although 34% (23/68) of the reviewed publications investigated the effectiveness of the apps and 21% (14/68) investigated their efficacy, no publication mentioned or evaluated JITAI mechanisms.
Conclusions: Promising JITAI mechanisms have not yet been translated into mainstream depression apps. Although the wide range of passive measurements available from smartphones were rarely used, self-reported outcomes were used by 71% (20/28) of the apps. However, in both cases, the measured outcomes were not used to tailor content and timing along a state of vulnerability or receptivity. Owing to this lack of tailoring to individual, state, or situation, we argue that the apps cannot be considered JITAIs. The lack of publications investigating whether JITAI mechanisms lead to an increase in the effectiveness or efficacy of the apps highlights the need for further research, especially in real-world apps.
Globally, each year, at least 246 million people are affected by depression , and depression is the leading cause of years lived with disability [ ]. Although effective treatments for depression exist [ - ], most individuals in need still do not receive effective treatment [ ], or those obtaining treatment do not benefit. People seeking help often face barriers such as the high costs of treatment, a shortage of trained clinicians, the stigma associated with seeking help, and accessibility difficulties [ - ].
Mobile apps may have the potential to address the rising prevalence of depression and the insufficient resources available for treatment [, ]. Apps are already an integral part of most people’s everyday lives [ ], and the threshold for engagement with apps is assumed to be low, resulting in prompt, flexible, portable, and anonymous treatment for people with depression [ ]. Individuals otherwise not reachable could receive treatment [ ], and the interventions could be delivered in economies that have limited resources for mental health treatment [ ]. Small to large effect sizes have been reported by several systematic reviews that show that apps and other digital interventions reduce symptoms of mental health problems, including depression [ - ]. Finally, apps can be used in real-life situations, where behavior change is most desirable and clinicians are unable to intervene [ ].
This study aims to complement the existing assessment of apps that address depression by focusing on the use of just-in-time adaptive intervention (JITAI) mechanisms [, ]. JITAIs aim to deliver an adaptive treatment (ie, personalized or tailored) at a time of vulnerability (ie, “person’s transient tendency to experience adverse health outcomes or to engage in maladaptive behaviors” [ ]) and receptivity (ie, “the person’s transient tendency to receive, process, and use the support provided” [ ]). To be considered a JITAI, an intervention needs to deliver the content when a state of vulnerability and state of receptivity are detected. Vulnerability refers to the right time, that is, a moment of heightened exposure or receptiveness for a negative health outcome [ ]. For example, a JITAI that addresses depression should deliver an intervention component when changes in mood are detected. Ideally, the component delivered should aim to improve the symptom that has been detected. The treatment must not only be delivered at the right time (vulnerability) but the recipient also needs to be receptive (receptivity). Receptivity refers to the availability, that is, the detection of the time when an individual can receive, process, and use the support provided [ ]. For a JITAI that addresses depression, this could be the time when the individual is at home because delivery of a behavior activation component needs a safe space.
The measured outcomes used to tailor content and timing along a state of vulnerability and state of receptivity are referred to as tailoring variables [, ]. Although ecological momentary assessments may facilitate the detection of these tailoring variables, passive measurements (eg, using the location derived from a smartphone’s GPS data) are regarded as the gold standard of tailoring variables for JITAIs. These passive measurements have the advantage of enabling unobtrusive, continuous observation [ ]. Although tailoring content and timing to the state of receptivity and state of vulnerability may be possible by asking ecological momentary assessments (eg, asking through message, “Would now be a good time to start an activity?”), the benefit of passive measurements may be that the content and timing are tailored without deliberate interaction with the intervention itself (eg, detecting a state of vulnerability and state of receptivity and sending a notification, “It seems that you are at home and in a bad mood. Would you like to start an exercise?”). Therefore, it has been proposed that JITAIs that tailor the content to the person, situation, and time by using these passive measurements reduce the burden on users and increase the effectiveness of the intervention [ , ].
Evidence of the higher effectiveness of JITAIs compared with that of non-JITAI treatments and wait-list control groups was investigated in a recent meta-analysis , which found moderate to large effect sizes (Hedges g=1.65 when compared with wait-list control and Hedges g=0.89 when compared with non-JITAI treatments) of the primary outcomes produced by 33 empirical studies. Owing to the potential of JITAIs to increase effectiveness while reducing the burden on users and the prominence of the JITAI framework in the scientific community, the aim of this study is to determine the degree to which popular apps that address depression use JITAI mechanisms by reviewing which relevant symptoms of depression (eg, mood) are measured as well as how they are measured. We are also interested in learning whether peer-reviewed studies can be found that investigated the increased effectiveness or efficacy of these apps attributable to the use of JITAI mechanisms.
To this end, we systematically assessed popular apps that target depression, that is, apps that have the most reviews on the Apple App Store and most downloads on the Google Play Store. We argue that the investigation of JITAI mechanisms is necessary because of their potential to increase effectiveness while simultaneously decreasing the burden on users. The focus on popular apps is important because they are listed at the top of search results and thus are very likely to be downloaded and used . Moreover, a high number of downloads implies that they have been found to be useful by users [ ] and may indicate that people continue to use or recommend them. Recent evidence also indicates that the 2 most popular apps for depression and anxiety were responsible for 90% of the 13.3 million active monthly users (Headspace with 6,959,000 users equivalent to 52% of the active users and Calm with 5,000,000 users equivalent to 38% of the active users) [ ]. Although the review by Wasil and Gillespie [ ] focused on depression and anxiety apps, similar user statistics should be observed when focusing only on apps for depression.
Search Strategy and Selection Criteria
We conducted this systematic app search and literature review following the methods used in existing reviews [, - ] of popular apps that address mental health problems. We systematically identified and reviewed apps that were publicly available in the US and UK app stores because number of venture capital backed digital health deals in the first half of 2020 in these countries ranked first (United States: 475) and second (United Kingdom: 41) of all English-speaking countries [ ]. The Apple App Store and Google Play Store were used because they have a combined market share of approximately 99.4% [ ]. We searched the two stores by entering the term depression in the search fields of the stores and included all apps found in both stores of both countries. We also reviewed curated lists of health apps from prominent organizations, namely the Anxiety and Depression Association of America [ ], the National Health Service [ ], and the American Psychological Association [ ], to ensure that we did not miss any app recommended by important institutions and experts for mental health. The apps found in these lists addressed several different mental health problems. We selected only those apps that addressed depression for further assessment. The searches were carried out in August 2020. All apps found regardless of the category in which they were found (eg, lifestyle), download category, or number of reviews were included in the initial list of apps. This list of all the apps found was screened according to the inclusion and exclusion criteria outlined below.
For further assessment, we included the apps that targeted the treatment of depression or reduction of the symptoms of depression by delivering at least one active ingredient and were available in English. We defined an active ingredient, on the lines of the study by Michie et al , as a function supporting users in their management of depression that is designed to reliably and causally change processes that govern behavior [ ]. An example of an active ingredient for depression could be a goal-setting task, a breathing exercise, or a recording of daily mood. Apps targeting other mental health illnesses such as anxiety or posttraumatic stress disorder were not excluded if depression was addressed as well. We included both free and paid apps. Browser-based treatments were not included. We excluded apps that only targeted professionals (eg, Depression Psychopharmacology), only offered a diagnostic service (eg, Patient Health Questionnaire-9 Depression Test Questionnaire), only provided quotes or inspirational text (eg, Depression Quote Wallpapers), or only conveyed information without the goal of eliciting behavior change or engaging with individuals (eg, Psychology Book: 1000+ Amazing Psychology Facts).
Keeping the inclusion and exclusion criteria in mind, 2 authors (GWT and ADF) separately reviewed each app identified from the initial search for depression apps in the two stores and the curated lists. The interrater agreement was excellent (Cohen κ=0.91). In case of disagreements, a consensus was reached through discussion. After this initial assessment, we ranked all included apps from the Apple App Store separately by their number of reviews and all included apps from the Google Play Store by their download category (eg, 1,000,000+ and 500,000+ downloads). We decided to screen the apps before ranking them to ensure that the most popular apps for depression—and not the most popular apps from our initial lists—were included in the analysis.
Next, we separately identified the most popular apps available only in the Apple App Store, available only in the Google Play Store, or available in both stores. For apps only available in the Apple App Store, we selected the 5 most reviewed apps because users rarely scroll past the first 5 apps on the list . For the Google Play Store, we used the download category of the app ranked fifth on the list (eg, 500,000+ downloads). All apps in the 500,000+ download category were included. For apps available in both app stores, we used the Google Play Store’s 500,000+ download category to determine inclusion, regardless of the number of reviews on the Apple App Store. Regardless of the number of downloads or reviews, we included all apps from the curated lists that met the inclusion criteria and did not violate the exclusion criteria.
Our evaluation covered the following areas: general information about the app, potential mechanisms for delivery of the JITAI, and peer-reviewed evidence. We developed our evaluation framework before reviewing the apps and used Covidence Systematic Review software (Veritas Health Innovation Ltd; version accessed in August 2020) to review the apps. All the questions are listed in the codebook in Tables S1-S6 of[ , , - ], along with the sources from which we derived them. Each included app was evaluated separately by 2 raters (GWT and ADF) as follows:
First, we gathered general information about the apps, including the name of the provider, additional affiliated organizations (eg, other companies, universities, governments, or nongovernmental organizations), and time since last update. Second, we reviewed the app’s website and recorded all the publications provided as well as information about JITAIs. Third, we searched for peer-reviewed publications in PubMed, Cochrane Register of Controlled Trials, PsycINFO, Google Scholar, IEEE Xplore, Web of Science, ACM Portal, and Science Direct using the search term [(Name AND App) OR (Name AND Application AND Smartphone)].
Fourth, we reviewed the full text of each study found on the website and in the different databases. We excluded books, theses, systematic reviews evaluating several different apps, and clinical trial registrations. Subsequently, each study was evaluated in line with prior work , including the year of publication, journal name, journal impact factor, the number of subjects, study purpose, and study design (ie, randomized controlled trial or open trial). We also extracted the information available about the JITAI mechanism. We determined the degree to which the apps could be considered JITAIs by reviewing whether, and to what degree, the relevant features derived from the JITAI concept described in the study by Nahum-Shani et al [ ] had been implemented. In the first step, we reviewed how potential tailoring variables were measured. In the second step, we reviewed whether these measurements were used to determine a state of vulnerability (ie, a moment of heightened exposure or receptiveness for a negative health outcome) or state of receptivity (ie, the time when the input could be perceived, processed, and used). To leverage the potential of JITAIs, the state of vulnerability and state of receptivity not only need to be detected through measurements but the treatment content and timing also need to be tailored along these measurements. We assessed how the tailoring variables were measured by reviewing the symptoms of depression that were measured (derived from the International Classification of Diseases, 10th Revision and Diagnostic and Statistical Manual of Mental Disorders, Edition 5) and the self-report data or sensor and device analytics (derived from related work [ ] from the Android Developers Guide [ ] and the iPhone operating system Security Guide [ ]) that were used. We also reviewed whether tailoring to traits (ie, “tailoring-to-people” [ ]) was used by checking for questions about demographics and socioeconomic status. Furthermore, we checked whether receptivity was measured by GPS location, self-reports, or by measuring symptoms. As it has been proposed that JITAI mechanisms increase the effectiveness or efficacy of such apps [ ], we reviewed whether the publications addressed effectiveness or efficacy and whether JITAI mechanisms were investigated in these publications.
Finally, we reviewed the app itself and extracted the information available about JITAI mechanisms. The results from each rater were compared, and consensus was reached by discussion, if necessary. We reviewed each app in September 2020, and the process is illustrated in.
We found 249 apps in the Apple App Store, 217 apps in the Google Play Store, 57 apps in both stores, and 135 apps in the curated list, yielding a total of 658 apps. Of the 658 apps, we removed 17 (2.6%) duplicates, 349 (53%) apps that did not mention depression, 123 (18.7%) apps with no active component, 8 (1.2%) apps that were not accessible, 1 (0.2%) app not available in English, and 1 (0.2%) app that targeted professionals. The reasons for excluding each app are presented in Table S7 of. The reasons for excluding apps because of missing active ingredients were also summarized and are presented in Table S8 of with further details. We ranked the apps found only in the Apple App Store based on their number of reviews and included the 5 most reviewed apps. We ranked the remaining apps found in the Google Play Store and in both stores according to their number of downloads category. The fifth most downloaded app in the Google Play Store fell in the 500,000+ download category. Therefore, we included all apps found in the Google Play Store and all apps available in both Apple App Store and Google Play Store with more than 500,000+ downloads, yielding 17 apps. We included 6 apps from the curated lists that met the inclusion criteria of mentioning depression and did not violate the exclusion criteria, yielding a total of 28 apps. A flowchart of the results of the review process is presented in .
Using methods similar to those used in the study by Baumel et al , we calculated the number of users and reviews covered by our selection of the top 5 reviewed apps from the Apple App Store and apps with more than 500,000 downloads found in the Google Play Store and apps found in both the Apple App Store and Google Play Store. The calculations are presented in Tables S9 and S10 of (Table S9 for apps ranked by their download category and Table S10 for apps ranked according to their number of reviews). Using the cutoff point of the 5 most reviewed apps, we included 91% of the reviews of the 35 apps found only in the Apple App Store that met the inclusion criteria. Including the sixth most reviewed app would have resulted in an increase of 1.97% of the reviews in the sample. By including apps with more than 500,000 downloads (ie, the 500,000+ download category), we included 96.7% of the users of the 118 apps found in the Google Play Store and both Google Play Store and Apple App Store that remained after screening. Including the download category of more than 100,000 downloads would have resulted in an increase of 3% in the downloads in the sample.
We found 70 peer-reviewed publications for the 28 reviewed apps (TableS12) [ - ]. We found at least one publication for 57% (16/28) of the apps, at least one peer-reviewed publication investigating the effectiveness of 32% (9/28) of the apps, and at least one peer-reviewed publication investigating the efficacy of 18% (5/28) of the apps. Of the 70 publications, although 23 (33%) investigated the effectiveness of the apps, and 14 (20%) investigated their efficacy, no publication evaluated an increase in effectiveness or efficacy achieved by using JITAI mechanisms. The extracted information from all the reviewed publications is presented in Tables S11 and S12 of .
The 28 apps included were rated 2,808,465 times, with each app being rated on average 100,302.32 times (SD 279,723.5; range 127-1,229,623; median 8,109, IQR 28,310). The average rating for all apps was 4.44 stars (SD 0.34; range 3.40-4.95; median 4.50, IQR 0.37) out of a possible five stars. The number of reviews and average reviews were calculated using weighted means with information from the Apple App Store and Google Play Store, if available.lists the general information and JITAI mechanisms of each reviewed app and summarizes the 28 apps that were included in the review and evaluated.
|App||Operating system||Number of reviews||AVRb||NODCc||Symptoms measured and measurements used||NDSd used||NSRe used||NSDAf used|
|Calm||ANDg, iOSh||1,229,623||4.71||10,000,000+||Activity, mood, unhelpful beliefs, sleep, open questions, multiple-choice questions||4||2||—i|
|Daylio||AND, iOS||328,048||4.62||10,000,000+||Activity, mood, appetite, sleep, ratings, multiple-choice questions||4||2||—|
|Youper||AND, iOS||61,478||4.67||1,000,000+||Activity, mood, interest or pleasure, sleep, vital signs, location, open questions, ratings, multiple-choice questions||4||3||2|
|Moodpath||AND, iOS||57,011||4.64||1,000,000+||Activity, mood, interest or pleasure, appetite, unhelpful beliefs, concentration, sleep, vital signs, open questions, ratings, multiple-choice questions||7||3||1|
|Wysa||AND, iOS||56,520||4.70||1,000,000+||Activity, mood, interest or pleasure, unhelpful beliefs, concentration, ratings, multiple-choice questions||5||2||—|
|Friend Shoulder||AND||32,847||4.30||1,000,000+||Mood, unhelpful beliefs, suicidal thoughts, open questions||3||1||—|
|BetterHelp||AND, iOS||30,592||4.63||500,000+||Mood, unhelpful beliefs, suicidal thoughts, open questions||3||1||—|
|Sanvello||AND, iOS||27,536||4.67||1,000,000+||Activity, mood, interest or pleasure, unhelpful beliefs, suicidal thoughts, sleep, microphone, open questions, ratings||6||2||1|
|7 Cups||AND, iOS||20,709||4.29||1,000,000+||Mood, interest or pleasure, cognition, unhelpful beliefs, distorted perception, open questions, ratings||5||2||—|
|Control and Monitor||AND||16,149||4.50||1,000,000+||Activity, mood, appetite, unhelpful beliefs, suicidal thoughts, sleep, ratings, multiple-choice questions||6||2||—|
|Remente||AND, iOS||11,494||4.34||1,000,000+||Activity, mood, open questions, ratings, multiple-choice questions||2||3||—|
|Reflexio||AND||8118||4.30||1,000,000+||Mood, open questions, ratings||1||2||—|
|Moodnotes||iOS||8100||4.70||—||Mood, illogical thinking, camera, open questions, ratings, multiple-choice questions||2||3||1|
|Online Therapy—Mental Help||AND||5979||4.40||500,000+||Mood, unhelpful beliefs, suicidal thoughts, open questions||3||1||—|
|InnerHour||AND, iOS||5402||4.50||500,000+||Activity, mood, interest or pleasure, unhelpful beliefs, concentration, suicidal thoughts, sleep, ratings, multiple-choice questions||8||2||—|
|Happify||AND, iOS||5164||4.21||500,000+||Activity, mood, interest or pleasure, cognition, unhelpful beliefs, sleep, illogical thinking, distorted perception, vital signs, camera, open questions, ratings, multiple-choice questions||8||3||2|
|What\'s Up?—A Mental Health App||AND, iOS||3446||4.22||500,000+||Activity, mood, interest or pleasure, open questions, ratings, multiple-choice questions||3||3||—|
|MoodTools—Depression Aid||AND, iOS||3167||4.31||100,000+||Activity, mood, interest or pleasure, appetite, cognition, unhelpful beliefs, concentration, suicidal thoughts, sleep, distorted perception, ratings, multiple-choice questions, Patient Health Questionnaire||10||3||—|
|DBTj Coach||AND, iOS||3067||4.95||10,000+||Activity, mood, interest or pleasure, appetite, unhelpful beliefs, concentration, suicidal thoughts, sleep, ratings, multiple-choice questions, Patient Health Questionnaire||8||3||—|
|CBTk Thought Diary||AND, iOS||2182||4.58||100,000+||Mood, distorted perception, open questions, ratings, multiple-choice questions||2||3||—|
|T2 Mood Tracker||AND, iOS||1873||3.40||100,000+||Activity, mood, cognition, unhelpful beliefs, concentration, sleep, open questions, ratings||6||2||—|
|Joyable||AND, iOS||1522||4.48||5000+||Activity, mood, interest or pleasure, appetite, unhelpful beliefs, suicidal thoughts, sleep, open questions, ratings, multiple-choice questions||7||3||—|
|Breeze||iOS||1313||4.70||—||Activity, mood, ratings, multiple-choice questions||2||2||—|
|Moodkit||iOS||159||4.40||—||Activity, mood, appetite, sleep, open questions, ratings, multiple-choice questions||4||3||—|
|Catch It||AND, iOS||144||3.66||50,000+||Mood, open questions, ratings, multiple-choice questions||1||3||—|
|Feeling Good||AND, iOS||127||3.91||10,000+||—||—||—||—|
aJITAI: just-in-time adaptive intervention.
bAVR: average rating out of five possible stars.
cNODC: number of downloads category.
dNDS: number of depression symptoms.
eNSR: number of self-reports.
fNSDA: number of sensors and device analytics.
gAND: Android operating system.
hiOS: Apple operating system.
iApp did not measure a symptom, did not use a self-report, or did not use sensors and device analytics.
jDBT: dialectical behavior therapy.
kCBT: cognitive behavioral therapy.
|Sum||Count||Value, mean (SD; range)||Value, median (IQR)|
|Reviews||2,808,465||28||100,302.32 (279,723.50; 127-1,229,623)||8109 (28,310)|
|Ratingb||—c||28||4.44 (0.34; 3.40-4.95)||4.50 (0.37)|
|Download category||42,375,000||25||1,695,000 (3,153,748.43; 5000-10,000,000)||500,000 (900,000)|
|Depression symptoms||114||25||4.56 (2.47; 1-10)||4 (3)|
|Self-reports||59||25||2.36 (0.7; 1-3)||2 (1)|
|Sensors and device analytics||7||5||1.4 (0.55; 1-2)||1 (1)|
aJITAI: just-in-time adaptive intervention.
bOut of five possible stars.
We found that, of the 28 reviewed apps, 25 (89%) measured some kind of depression symptom when users interacted with the app (eg, initial assessment when starting the app), whereas 3 (11%) did not use any measurements, 20 (71%) used at least one self-report (eg, daily report of mood changes through a rating), and 5 (18%) used self-reports and sensors and device analytics (eg, tracking activity through location data).illustrates the number of depression symptoms measured by different self-reports or sensors and device analytics for each of the reviewed apps. Mood Tools—Depression Aid measured the most depression symptoms (10 different symptoms measured) while not using any sensors and device analytics. Happify and Youper measured fewer depression symptoms (eight and four, respectively) but used two different sensors and device analytics.
Our findings regarding the use of self-reports and sensors and device analytics are summarized in. We found that a symptom was measured by a self-report or sensors and device analytics 196 times. For example, the app Calm measured four symptoms (activity, mood, unhelpful beliefs, and sleep) by using two different self-reports for each symptom (open questions and multiple-choice questions), resulting in a symptom measurement rate of 4.1% (8/196). To measure different depressive symptoms, self-reports were used almost exclusively (189/196, 96.4%), and sensors and device analytics were used rarely (7/196, 3.6%). The self-reports used most frequently to measure different depressive symptoms were closed questions consisting of ratings (eg, rate your mood from 1 to 10), Likert scales (eg, indicate how often you felt sad last week, with the following options: never, almost never, sometimes, most of the time, and always), and multiple-choice questions (eg, select the activities that made you happy in the last week) for a measurement rate of 77% (151/196). The measurement rate for open questions (ie, questions without a fixed response or text field with open input) was 19.4% (38/196). Of the 3.6% (7/196) using sensors and device analytics, the most frequently used were vital signs (mostly heart rate), location and camera with each 0.5% (1/196) measurement rate. The measurement rate for the symptom mood (most frequently measured symptom) was 30.1% (59/196), for activity 15.8% (31/196), for unhelpful beliefs 11.7% (23/196), and for sleep 10.2% (20/196).
When possible, we tried to match the measurement of the depressive symptom to a mechanism relevant to the JITAI concept. We were able to do so for state of vulnerability, proximal outcomes, distal outcomes, and initial tailoring. Some of the measurements could have been used as tailoring variables for two or three JITAI mechanisms. Therefore, it is possible that double counting of the symptoms and measurements for each mechanism occurred.illustrates our findings, including which measurements were used to measure which symptom and for which JITAI feature. The figure shows that some sensors and device analytics were not used as passive measurements; rather, they were used to actively capture changes. For example, the camera was used as a measurement for activity, with users being asked to take pictures of locations that they had visited or a picture of something that made them sad to describe their mood.
We reviewed the 28 most popular or recommended apps for depression found in the Apple App Store, Google Play Store, and in curated lists compiled by respected mental health authorities. Regarding our main aim to investigate JITAI mechanisms, we found that none of the reviewed apps specifically mentioned the use of JITAI mechanisms in the app, on their websites, or in the identified peer-reviewed publications. Of the 28 apps, we found that 3 (11%) did not use any measurements, and 20 (71%) only used self-reports (189/196, 96.4% of all measurements were self-reported). Although such self-reports can be used as in-the-moment assessments (ie, ecological momentary assessments) that are closely related to the JITAI concept , we argue that they are insufficient for leveraging the full potential of JITAIs. Of the 28 apps, we found that 5 apps (18%) also used sensors anddevice analytics (7/196, 3.6% of all measurements involved sensoranddevice analytics). However, we found that most sensors such as the camera were used as in-the-moment assessments or as part of an app’s features; they were not used to tailor the content or timing. Some of the apps measured depressive symptoms by using self-reports when the app was first opened to determine what kind of content should be presented (eg, measuring the need to focus on sleep or mood). However, these measurements were not used to tailor the content that was presented afterward. Furthermore, static tailoring, which has been observed to be less effective than dynamic tailoring [ ], is, in our view, not sufficient for an app to be considered as a JITAI. Relying on this initial assessment may also be insufficient because symptoms of depression seem to change frequently through the day and may be highly specific to an individual [ ].
Interestingly, we found that besides mood (symptom measurement rate of 59/196, 30.1%) and decreased activity (symptom measurement rate of 31/196, 15.8%), other symptoms of depression were measured less frequently. Given the broad variety and severity of depression  and the high comorbidity with other mental health problems such as anxiety [ ], this focus on a subset of symptoms may not be sufficient to detect changes that might indicate a need for support. In addition, a focus on the improvement of the main symptoms (eg, mood and activity, derived from the Diagnostic and Statistical Manual of Mental Disorders, Edition 5) may not be sufficient to contribute to the understanding of the complex processes involved in depression. Accurate and continuous measurements of psychophysiological changes enabled by passive measurements of various physiological features (eg, changes in breathing patterns or vital signs) may, however, improve the understanding of depression in general. Such an understanding could, in turn, enable an even more successful implementation of JITAIs.
Related work has shown that user adherence to various mental health apps dropped substantially after interacting with the app for only a couple of days . Although details of user engagement with the apps reviewed in the study by Baumel et al [ ] were not available, several apps that were included in our study were also included in this review (eg, Headspace, Calm, Daylio, 7Cups, Happify, Moodpath, and T2 Mood Tracker). These reviewed apps showed a drastic decline in user numbers after a couple of days, highlighting the importance of finding ways to increase user adherence. A proposed key advantage of a JITAI is to address this challenge of declining adherence by offering support in times of need and reducing the burden on the individual. An app that solely relies on self-reports entered by the user while interacting with the app cannot achieve this goal. Individuals would need to open the app and engage with it to enter the responses before tailoring to the right time could be carried out. Continuously requiring such self-reports may even be counterproductive to adherence because of the increase in the burden. Therefore, future work should address the extent to which JITAIs that tailor content along passive measurements are capable of increasing adherence. A comparison among apps that do not use any measurements, that use static tailoring to measurements, that use tailoring to self-reports only, or that use tailoring to both self-reports and passive measurements would be necessary to determine the benefit of dynamic tailoring with regard to increasing user adherence.
Considering self-reported questions further by focusing on open questions, the question arises as to what degree the answers to these questions could be used to tailor content without any further use of analytics. Although self-reported closed questions already suffer from some biases (eg, recall bias and biases due to mental illness), this shortcoming is addressed by using different questions to infer an underlying construct and to validate questionnaires in different samples. In contrast, open questions may still be limited to measuring symptoms of depression and need to rely on methods derived from natural language processing to cope consistently and accurately with the large amount of data entered. Contrary to closed-question questionnaires, these methods have not been validated as rigorously as existing questionnaires for symptoms and symptom severity. Therefore, it is unclear to what degree open questions can be used to measure symptoms of depression. Existing work using free text has focused on using data from other sources to measure symptom changes. In a recent review by Chancellor and De Choudhury , 75 such studies were investigated. In these studies, social media data were used to infer various mental health problems. The authors share our concern regarding consistency and accuracy by concluding that construct validity and lack of reflection may limit the ability to use text to measure health status. In contrast, passive measurements that could be derived when users answer open questions, such as keystroke frequency or pressure on display, have been used to measure cognitive functions associated with mental health [ ].
As stated in the Introduction section, the JITAI concept aims to tailor content and timing based on the measured state of vulnerability and state of receptivity. Our results indicate that the measurements used in the reviewed apps may only serve for tailoring to the state of vulnerability (eg, mood and activity). We did not find any measurement that may serve to tailor to the state of receptivity. Plausible examples of such receptivity tailoring are still conceivable for both self-reports (eg, asking whether an individual is alert enough to read a text or complete a demanding exercise) and passive measurements (eg, detecting the user’s arrival at home through GPS data and time of day to trigger a journaling exercise). In general, studies investigating possible markers to detect receptivity and the effectiveness of such receptivity tailoring are sparse. This research gap becomes even more evident when one considers the various studies that aim to develop passive measurements that could be used to determine vulnerability [, - ] together with the finding that tailoring to what had a significant impact on the effectiveness of JITAIs [ ].
Other concepts related to receptivity, such as interruptibility , have been investigated in greater detail [ ]. In accordance with the study by Künzler et al [ ], we would still argue that receptivity extends these concepts by combining concepts such as the willingness to receive an intervention (interruptibility), engage with the intervention, and process the provided information. To the best of our knowledge, only the following studies have specifically focused on investigating receptivity. The study by Künzler et al [ ] developed machine learning models to predict receptivity in a physical activity intervention. In the study by Choi et al [ ], the authors describe the results of a 3-week study that they conducted to investigate the context and cognitive or physical state to understand the response to the components of sedentary behavior. The study by Sarker et al [ ] used the response to ecological momentary assessments—but not to an intervention component—to explore discriminative features for building machine learning models to detect receptivity and reported 77.9% accuracy. Although these studies used post hoc analysis to determine receptivity, the study by Mishra et al [ ] used data from the study by Künzler et al [ ] to test the two different models in a real-world study. The authors found that receptivity increased by 40% compared with the control when a static machine learning model was used and that a dynamic machine learning model led to an increase in receptivity over time, whereas receptivity in the control conditions declined. These results are promising because they show that with relatively easily derived information such as time of day or Wi-Fi connection, receptivity and engagement can be maintained, or engagement can even be increased over time, which is something that mental health interventions have been observed to struggle with [ ].
The finding that most measurements could have been used to tailor to the state of vulnerability but not to the state of receptivity may be explained by three considerations. First, as shown in the study by Wang and Miller , the tailoring to what (ie, both previous behavioral patterns of people and their current need states) showed significant improvement. However, results regarding the tailoring to when have not been reported. Second, most related work aimed at developing markers for detecting changes in symptom severity or depressive severity [ , - ]. Although some of these outcomes may also be relevant for the state of receptivity, the primary use of these measurements would be to determine the state of vulnerability (eg, mood changes detected by voice), distal outcome (eg, depression severity score calculated by combining different symptom outcomes), and proximal outcome (eg, sleep quality measured through vital signs). Finally, as mentioned above, it seems that receptivity as a concept has been investigated less often, with related work focusing on concepts such as interruptibility [ ] or detection of boredom [ ]. Future work should focus on transferring the findings on contextual factors such as interruptibility or boredom by using the day of the week [ ], time of day [ ], location [ ], or different proxies for social context (eg, number of calls and messages sent or received) [ ] to investigate tailoring to the state of receptivity.
Taken together, these findings highlight that although the JITAI concept seems to be widely known in the scientific digital health community  and different studies outline the possibility of detecting changes in depression or depressive symptoms such as mood by using different passive measurements [ , - ], these mechanisms have not been implemented in the real world, aside from baseline or progress assessments. Although tailoring to both the state of vulnerability and state of receptivity may be challenging, it is still surprising that no JITAI mechanisms have been implemented by the reviewed apps. This finding is especially unexpected given the existing work on passive sensing and the popularity of the JITAI concept.
Related to these findings, we were interested in the degree to which the effectiveness and efficacy of the included apps were investigated in peer-reviewed publications because it has been proposed that JITAIs increase the effectiveness or efficacy of such apps. None of the 70 reviewed publications investigated JITAI mechanisms. Therefore, our findings highlight that the proposed increase in effectiveness or efficacy achieved by using JITAIs is not evaluated in real-world app settings. In addition, we found great variability in the scientific evidence provided for the reviewed apps—despite an increased interest in digital health—in several publications addressing this topic, especially within the last 5 years [, , ].
The strengths of this study are the large number of apps initially screened; the analyses using a framework developed from existing work; and the rigorous methodology used to review all identified studies addressing the apps, the apps’ websites, and the apps themselves. However, this study includes several limitations. We reviewed the apps at a single point, which is a shortcoming found in related studies as well. We are aware that the app stores are dynamic with constant changes , but a long-term review of the apps would not have been feasible. We may address this issue in our future work. Besides the lists we reviewed from the Anxiety and Depression Association of America, National Health Service, and American Psychological Association, rating systems or lists of reviewed apps are offered by other organizations, too. These include, but are not limited to, the American Psychiatric Association, PsyberGuide, and iMedicalApps. We did not review these lists because we expected a high number of overlaps and because not all the apps found in the lists had been reviewed (eg, Dartmouth PATH had not been reviewed on PsyberGuide when last checked [January 27, 2021]), and not all the apps mentioned in the lists had been recommended (eg, Mood Watch Review, with low credibility, user experience, and transparency ratings on PsyberGuide when last checked [January 27, 2021]). However, we see the value in a central platform for reviews of mental health apps and suggest incorporating the findings regarding the use of JITAI mechanisms into the existing review criteria.
We did not search for apps developed by research groups as was done in a recent meta-analytic review by Wang and Miller  that evaluated studies addressing the effectiveness of JITAIs. Their search included, but was not limited to, apps. Although this work is highly relevant for the field, we argue that a review investigating the degree to which tailoring is used in popular apps was necessary. This notion was derived from findings highlighting that academia has not always kept up with the high speed of digital health developments in the real world [ ]. In addition, apps developed for research purposes may not be available to the public, limiting the impact of these apps to a small number of individuals participating in the review. Our aim was to investigate whether popular apps that reach many individuals use these promising mechanisms. Finally, the review of the apps initially included other aspects such as the use of evidence-based treatment, conversational agents, and the revenue model. Reporting these findings would have exceeded the scope of this review.
Comparison With Prior Work
We found 11 reviews investigating different aspects of the apps that address depression [, , , , , - ]. Of these 11 reviews, 6 (45%) assessed the content or features of the apps [ , , , , , ]; of these 6 reviews, 1 (17%) adjusted its analysis to the number of users [ ]. A meta-analysis focused on the efficacy of apps for depression, indicating a moderate positive effect when the apps were compared with inactive control and a small effect when comparing the apps with active control [ ]. The remaining studies investigated usability [ ], adherence to clinical guidelines [ ], claims [ ], or data sharing and privacy practice [ ]. A meta-analytic review investigated the effect sizes of JITAIs compared with control groups or other interventions, but it did not focus on apps or mental health [ ]. This meta-analytic review indicated that two aspects of tailoring—tailoring to what and the tailoring approach—were significantly associated with greater JITAI efficacy. We found no study investigating the use of JITAI mechanisms or reviewing the measurements used to capture changes when discussing the relevant features of such apps. Furthermore, we did not find any studies reviewing whether real-world apps provide evidence of an improvement in their effectiveness or efficacy achieved by using JITAI mechanisms.
In conclusion, our findings indicate that because of the limited use of measurements for depressive symptoms, except for self-reports as indicators of progress or initial tailoring, the 28 most popular or recommended apps that address depression cannot be considered JITAIs. Although self-reports and, occasionally, passive measurements were used to capture changes in depressive symptoms, these changes were not used to dynamically tailor the content to a state of vulnerability or state of receptivity. An increase in app effectiveness or efficacy achieved by using JITAI mechanisms was also not evaluated in any of the reviewed publications. Therefore, future work should focus on evaluating the effectiveness of real-world JITAIs that tailor their content and timing to the state of vulnerability and state of receptivity. On the basis of these findings, we argue that the reviewed apps do not yet leverage the full potential of digital health interventions by providing tailored support when it is most needed and in the most helpful way.
The authors would like to thank Jacqueline Louise Mair for reviewing the manuscript and providing feedback. This work was partially supported by the National Institute of Mental Health (R01 MH123482) and CSS Insurance, Switzerland. This research was also partially supported by the National Research Foundation, Prime Minister’s Office, Singapore under its Campus for Research Excellence and Technological Enterprise program and was in part conducted at the Future Health Technologies program which was established collaboratively between ETH Zurich and the National Research Foundation Singapore. This research is supported by the National Research Foundation, Prime Minister’s Office, Singapore under its Campus for Research Excellence and Technological Enterprise program. The National Institute of Mental Health, the National Research Foundation, and CSS Insurance had no role in the study design, data collection, data analysis and interpretation, writing the manuscript, or reviewing and approving the manuscript for publication. The reviewed apps and reviewed studies are publicly available. The extracted data used in this review will be shared with full open access beginning 3 months and ending 24 months after the publication of the study. For further questions or material requests, please contact GWT.
GWT designed the evaluation framework using inputs from TK and ADF. GWT and TK designed and implemented a search strategy. GWT and ADF screened and coded the apps, websites, and studies, and extracted the data. GWT analyzed the data and drafted the initial manuscript, supervised by TK. BK, NCJ, ASS, LTC, and EF provided methodological guidance and feedback on the manuscript. All authors reviewed and approved the final manuscript.
Conflicts of Interest
GWT, EF, and TK are affiliated with the Centre for Digital Health Interventions, a joint initiative of the Department of Management, Technology, and Economics at ETH Zurich and the Institute of Technology Management at the University of St. Gallen, which is funded in part by the Swiss health insurer CSS. EF and TK are also cofounders of Pathmate Technologies, a university spin-off company that creates and delivers digital clinical pathways. However, Pathmate Technologies was not involved in this study. NCJ and Dartmouth College are the owners of a depression and anxiety app named Mood Triggers. However, owning Mood Triggers is not a financial conflict of interest, given that Mood Triggers is not intended to be revenue generating; rather, it is used to deliver and evaluate no-cost scalable treatment treatments using just-in-time adaptive interventions (JITAIs).
Codebook for app review, calculations of downloads and reviews covered, number of studies and type of studies found for each app, and list of all reviewed publications.DOCX File , 168 KB
- Depression and other common mental disorders: global health estimates. World Health Organization. 2017. URL: https://apps.who.int/iris/bitstream/handle/10665/254610/WHO-MSD-MER-2017.2-eng.pdf [accessed 2021-08-07]
- James S, Abate D, Abate K, Abay S, Abbafati C, Abbasi N, et al. Global, regional, and national incidence, prevalence, and years lived with disability for 354 diseases and injuries for 195 countries and territories, 1990–2017: a systematic analysis for the Global Burden of Disease Study 2017. The Lancet 2018 Nov;392(10159):1789-1858. [CrossRef]
- Cuijpers P, Stringaris A, Wolpert M. Treatment outcomes for depression: challenges and opportunities. Lancet Psychiatry 2020 Nov;7(11):925-927. [CrossRef] [Medline]
- Delgadillo J, Duhne P. Targeted prescription of cognitive-behavioral therapy versus person-centered counseling for depression using a machine learning approach. J Consult Clin Psychol 2020 Jan;88(1):14-24. [CrossRef] [Medline]
- Delgadillo J, Rubel J, Barkham M. Towards personalized allocation of patients to therapists. J Consult Clin Psychol 2020 Sep;88(9):799-808. [CrossRef] [Medline]
- Kazdin AE, Blase SL. Rebooting psychotherapy research and practice to reduce the burden of mental illness. Perspect Psychol Sci 2011 Jan;6(1):21-37. [CrossRef] [Medline]
- Andrade LH, Alonso J, Mneimneh Z, Wells JE, Al-Hamzawi A, Borges G, et al. Barriers to mental health treatment: results from the WHO World Mental Health surveys. Psychol Med 2013 Aug 09;44(6):1303-1317. [CrossRef]
- Gulliver A, Griffiths KM, Christensen H. Perceived barriers and facilitators to mental health help-seeking in young people: a systematic review. BMC Psychiatry 2010;10:113 [FREE Full text] [CrossRef] [Medline]
- Global health workforce, finances remain low for mental health. World Health Organization. 2015. URL: https://www.who.int/news/item/14-07-2015-global-health-workforce-finances-remain-low-for-mental-health [accessed 2021-08-07]
- Wilhelm S, Weingarden H, Ladis I, Braddick V, Shin J, Jacobson NC. Cognitive-behavioral therapy in the digital age: presidential address. Behav Ther 2020 Jan;51(1):1-14 [FREE Full text] [CrossRef] [Medline]
- Fairburn CG, Patel V. The impact of digital technology on psychological treatments and their dissemination. Behav Res Ther 2017 Jan;88:19-25 [FREE Full text] [CrossRef] [Medline]
- Kazdin AE. Addressing the treatment gap: A key challenge for extending evidence-based psychosocial interventions. Behav Res Ther 2017 Jan;88:7-18. [CrossRef] [Medline]
- The Lancet Child and Adolecent Health. Growing up in a digital world: benefits and risks. Lancet Child Adol Health 2018 Feb;2(2):79. [CrossRef]
- Weisel KK, Fuhrmann LM, Berking M, Baumeister H, Cuijpers P, Ebert DD. Standalone smartphone apps for mental health-a systematic review and meta-analysis. NPJ Digit Med 2019;2:118 [FREE Full text] [CrossRef] [Medline]
- Ebert DD, Van Daele T, Nordgreen T, Karekla M, Compare A, Zarbo C, et al. Internet- and mobile-based psychological interventions: applications, efficacy, and potential for improving mental health efficacy, and potential for improving mental health. Eur Psychol 2018 May;23(2):167-187. [CrossRef]
- Chisholm D, Sweeny K, Sheehan P, Rasmussen B, Smit F, Cuijpers P, et al. Scaling-up treatment of depression and anxiety: a global return on investment analysis. Lancet Psychiatry 2016 May;3(5):415-424 [FREE Full text] [CrossRef] [Medline]
- Andrews G, Basu A, Cuijpers P, Craske M, McEvoy P, English C, et al. Computer therapy for the anxiety and depression disorders is effective, acceptable and practical health care: An updated meta-analysis. J Anxiety Disord 2018 Apr;55:70-78 [FREE Full text] [CrossRef] [Medline]
- Deady M, Choi I, Calvo RA, Glozier N, Christensen H, Harvey SB. eHealth interventions for the prevention of depression and anxiety in the general population: a systematic review and meta-analysis. BMC Psychiatry 2017 Aug 29;17(1):310 [FREE Full text] [CrossRef] [Medline]
- Ebert DD, Zarski A, Christensen H, Stikkelbroek Y, Cuijpers P, Berking M, et al. Internet and computer-based cognitive behavioral therapy for anxiety and depression in youth: a meta-analysis of randomized controlled outcome trials. PLoS One 2015;10(3):e0119895 [FREE Full text] [CrossRef] [Medline]
- Firth J, Torous J, Nicholas J, Carney R, Pratap A, Rosenbaum S, et al. The efficacy of smartphone-based mental health interventions for depressive symptoms: a meta-analysis of randomized controlled trials. World Psychiatry 2017 Oct;16(3):287-298 [FREE Full text] [CrossRef] [Medline]
- Nahum-Shani I, Hekler EB, Spruijt-Metz D. Building health behavior models to guide the development of just-in-time adaptive interventions: A pragmatic framework. Health Psychol 2015 Dec;34 Suppl:1209-1219. [CrossRef] [Medline]
- Nahum-Shani I, Smith SN, Spring BJ, Collins LM, Witkiewitz K, Tewari A, et al. Just-in-Time Adaptive Interventions (JITAIs) in mobile health: Key components and design principles for ongoing health behavior support. Ann Behav Med 2016 Sep 23:446-462. [CrossRef] [Medline]
- Cornet VP, Holden RJ. Systematic review of smartphone-based passive sensing for health and wellbeing. J Biomed Inform 2018 Jan;77:120-132. [CrossRef] [Medline]
- Wang L, Miller LC. Just-in-the-Moment Adaptive Interventions (JITAI): A meta-analytical review. Health Commun 2019 Sep 05:1-14. [CrossRef] [Medline]
- Dogruel L, Joeckel S, Bowman ND. Choosing the right app: An exploratory perspective on heuristic decision processes for smartphone app selection. Mobile Media Commun 2014 Dec 29;3(1):125-144. [CrossRef]
- Baumel A, Torous J, Edan S, Kane JM. There is a non-evidence-based app for that: A systematic review and mixed methods analysis of depression- and anxiety-related apps that incorporate unrecognized techniques. J Affect Disord 2020 Aug 01;273:410-421. [CrossRef] [Medline]
- Wasil AR, Gillespie S, Patel R, Petre A, Venturo-Conerly KE, Shingleton RM, et al. Reassessing evidence-based content in popular smartphone apps for depression and anxiety: Developing and applying user-adjusted analyses. J Consult Clin Psychol 2020 Nov;88(11):983-993. [CrossRef] [Medline]
- Shen N, Levitan M, Johnson A, Bender JL, Hamilton-Page M, Jadad AA, et al. Finding a depression app: a review and content analysis of the depression app marketplace. JMIR Mhealth Uhealth 2015;3(1):e16 [FREE Full text] [CrossRef] [Medline]
- Qu C, Sas C, Roquet C, Doherty G. Functionality of top-rated mobile apps for depression: systematic search and evaluation. JMIR Ment Health 2020 Jan 24;7(1):e15321. [CrossRef]
- Baumel A, Faber K, Mathur N, Kane JM, Muench F. Enlight: A comprehensive quality and therapeutic potential evaluation tool for mobile and web-based ehealth interventions. J Med Internet Res 2017 Mar 21;19(3):e82 [FREE Full text] [CrossRef] [Medline]
- Number of venture capital backed digital health deals in select countries in HY1 2020. Statista. 2020. URL: https://www.statista.com/statistics/1063913/vc-backed-digital-health-deals-in-select-countries/ [accessed 2021-08-07]
- Mobile operating system market share worldwide. statcounter Global Stats. 2021. URL: https://gs.statcounter.com/os-market-share/mobile/worldwide [accessed 2020-04-04]
- ADAA Reviewed mental health apps. Anxiety and Depression Association of America (ADAA). 2020. URL: https://adaa.org/finding-help/mobile-apps [accessed 2020-07-04]
- NHS Apps Library. National Health Service. 2020. URL: https://www.nhs.uk/apps-library/ [accessed 2020-07-04]
- Mobile apps and websites. American Psychological Associatin. 2020. URL: https://www.apa.org/careers/early-career/useful-apps?tab=3 [accessed 2020-07-04]
- Michie S, Ashford S, Sniehotta FF, Dombrowski SU, Bishop A, French DP. A refined taxonomy of behaviour change techniques to help people change their physical activity and healthy eating behaviours: The CALO-RE taxonomy. Psychol Health 2011 Nov;26(11):1479-1498. [CrossRef]
- Carey RN, Connell LE, Johnston M, Rothman AJ, de Bruin M, Kelly MP, et al. Behavior change techniques and their mechanisms of action: a synthesis of links described in published intervention literature. Ann Behav Med 2019 Jul 17;53(8):693-707 [FREE Full text] [CrossRef] [Medline]
- Safavi K, Mathews SC, Bates DW, Dorsey ER, Cohen AB. Top-funded digital health companies and their impact on high-burden, high-cost conditions. Health Aff (Millwood) 2019 Jan;38(1):115-123. [CrossRef] [Medline]
- Sensors overview. Developers Android. 2020. URL: https://developer.android.com/guide/topics/sensors/sensors_overview [accessed 2020-07-07]
- iOS security guide. Apple Platform Security. URL: https://support.apple.com/en-gb/guide/security/welcome/web [accessed 2020-07-07]
- Mathews SC, McShea MJ, Hanley CL, Ravitz A, Labrique AB, Cohen AB. Digital health: a path to validation. NPJ Digit Med 2019;2:38 [FREE Full text] [CrossRef] [Medline]
- Barlow DH, Farchione TJ, Sauer-Zavala S, Murray-Latin H, Bullis JR, Ellard K, et al. Unified Protocol for Transdiagnostic Treatment of Emotional Disorders: Therapist Guide. Oxford, United Kingdom: Oxford University Press; 2017:1-208.
- Kroenke K, Spitzer RL, Williams JB. The PHQ-9: validity of a brief depression severity measure. J Gen Intern Med 2001 Sep;16(9):606-613 [FREE Full text] [CrossRef] [Medline]
- Hamilton M. Hamilton rating scale for depression. In: Sartorius N, Ban TA, editors. Assessment of Depression. Berlin: Springer; 1986:143-152.
- Sheehan DV, Lecrubier Y, Sheehan KH, Amorim P, Janavs J, Weiller E, et al. The Mini-International Neuropsychiatric Interview (M.I.N.I.): the development and validation of a structured diagnostic psychiatric interview for DSM-IV and ICD-10. J Clin Psychiatry 1998;59 Suppl 20:22-34. [Medline]
- Beck AT, Steer RA, Brown G. Beck Depression Inventory–II. Psychol Assess 1996:A. [CrossRef]
- Lambert MJ, Gregersen AT, Burlingame GM. The Outcome Questionnaire-45. In: Maruish ME, editor. The use of psychological testing for treatment planning and outcomes assessment: Instruments for adults. Oxfordshire, United Kingdom: Routledge; 2004:1-528.
- Evans C, Mellor-Clark J, Margison F, Barkham M, Audin K, Connell J, et al. CORE: Clinical outcomes in routine evaluation. J Mental Health 2009 Jul 06;9(3):247-255. [CrossRef]
- Baumel A, Muench F, Edan S, Kane JM. Objective user engagement with mental health apps: systematic search and panel-based usage analysis. J Med Internet Res 2019 Sep 25;21(9):e14567 [FREE Full text] [CrossRef] [Medline]
- Huberty J, Green J, Glissmann C, Larkey L, Puzia M, Lee C. Efficacy of the mindfulness meditation mobile app "Calm" to reduce stress among college students: randomized controlled trial. JMIR Mhealth Uhealth 2019 Jun 25;7(6):e14273 [FREE Full text] [CrossRef] [Medline]
- Huberty J, Eckert R, Larkey L, Kurka J, De Jesús SA, Yoo W, et al. Smartphone-based meditation for myeloproliferative neoplasm patients: feasibility study to inform future trials. JMIR Form Res 2019 Apr 29;3(2):e12662 [FREE Full text] [CrossRef] [Medline]
- Huberty J, Eckert R, Larkey L, Joeman L, Mesa R. Experiences of using a consumer-based mobile meditation app to improve fatigue in myeloproliferative patients: qualitative study. JMIR Cancer 2019 Jul 22;5(2):e14292 [FREE Full text] [CrossRef] [Medline]
- Huberty J, Vranceanu A, Carney C, Breus M, Gordon M, Puzia ME. Characteristics and usage patterns among 12,151 paid subscribers of the calm meditation app: cross-sectional survey. JMIR Mhealth Uhealth 2019 Nov 03;7(11):e15648 [FREE Full text] [CrossRef] [Medline]
- Huberty J, Puzia M, Eckert R, Larkey L. Cancer patients' and survivors' perceptions of the calm app: cross-sectional descriptive study. JMIR Cancer 2020 Jan 25;6(1):e16926 [FREE Full text] [CrossRef] [Medline]
- Puzia ME, Huberty J, Eckert R, Larkey L, Mesa R. Associations between global mental health and response to an app-based meditation intervention in myeloproliferative neoplasm patients. Integr Cancer Ther 2020 Jun 20;19:1534735420927780 [FREE Full text] [CrossRef] [Medline]
- Huberty J, Puzia ME, Larkey L, Irwin MR, Vranceanu A. Use of the consumer-based meditation app calm for sleep disturbances: cross-sectional survey study. JMIR Form Res 2020 Nov 13;4(11):e19508 [FREE Full text] [CrossRef] [Medline]
- Clarke J, Draper S. Intermittent mindfulness practice can be beneficial, and daily practice can be harmful. An in depth, mixed methods study of the "Calm" app's (mostly positive) effects. Internet Interv 2020 Mar;19:100293 [FREE Full text] [CrossRef] [Medline]
- Callender KA, Trustey CE, Alton L, Hao Y. Single case evaluation of a mindfulness-based mobile application with a substance abuse counselor. Counsel Outcome Res Eval 2019 Nov 22;12(1):16-29. [CrossRef]
- Puzia M, Laird B, Green J, Huberty J. Parents' perceptions of their children's engagement in a consumer-based meditation mobile app: cross-sectional survey study. JMIR Pediatr Parent 2020 Nov 13;3(2):e24536 [FREE Full text] [CrossRef] [Medline]
- Howells A, Ivtzan I, Eiroa-Orosa FJ. Putting the ‘app’ in Happiness: A randomised controlled trial of a smartphone-based mindfulness intervention to enhance wellbeing. J Happiness Stud 2014 Oct 29;17(1):163-185. [CrossRef]
- Lim D, Condon P, DeSteno D. Mindfulness and compassion: an examination of mechanism and scalability. PLoS One 2015 Feb 17;10(2):e0118221 [FREE Full text] [CrossRef] [Medline]
- Taylor M, Hageman JR, Brown M. A mindfulness intervention for residents: relevance for pediatricians. Pediatr Ann 2016 Oct 01;45(10):373-376. [CrossRef] [Medline]
- Laurie J, Blandford A. Making time for mindfulness. Int J Med Inform 2016 Dec;96:38-50. [CrossRef] [Medline]
- Wen L, Sweeney TE, Welton L, Trockel M, Katznelson L. Encouraging mindfulness in medical house staff via smartphone app: a pilot study. Acad Psychiatry 2017 Oct 9;41(5):646-650. [CrossRef] [Medline]
- Wylde CM, Mahrer NE, Meyer RM, Gold JI. Mindfulness for novice pediatric nurses: smartphone application versus traditional intervention. J Pediatr Nurs 2017 Sep;36:205-212. [CrossRef] [Medline]
- Mistler LA, Ben-Zeev D, Carpenter-Song E, Brunette MF, Friedman MJ. Mobile mindfulness intervention on an acute psychiatric unit: feasibility and acceptability study. JMIR Ment Health 2017 Aug 21;4(3):e34 [FREE Full text] [CrossRef] [Medline]
- Bennike IH, Wieghorst A, Kirk U. Online-based mindfulness training reduces behavioral markers of mind wandering. J Cogn Enhanc 2017 Apr 25;1(2):172-181. [CrossRef]
- Kubo A, Altschuler A, Kurtovich E, Hendlish S, Laurent CA, Kolevska T, et al. A pilot mobile-based mindfulness intervention for cancer patients and their informal caregivers. Mindfulness (N Y) 2018 Dec;9(6):1885-1894 [FREE Full text] [CrossRef] [Medline]
- Noone C, Hogan MJ. A randomised active-controlled trial to examine the effects of an online mindfulness intervention on executive control, critical thinking and key thinking dispositions in a university student sample. BMC Psychol 2018 Apr 05;6(1):13 [FREE Full text] [CrossRef] [Medline]
- Yang E, Schamber E, Meyer RM, Gold JI. Happier healers: randomized controlled trial of mobile mindfulness for stress management. J Altern Complement Med 2018 May;24(5):505-513. [CrossRef] [Medline]
- Economides M, Martman J, Bell MJ, Sanderson B. Improvements in stress, affect, and irritability following brief use of a mindfulness-based smartphone app: a randomized controlled trial. Mindfulness (N Y) 2018;9(5):1584-1593 [FREE Full text] [CrossRef] [Medline]
- DeSteno D, Lim D, Duong F, Condon P. Meditation inhibits aggressive responses to provocations. Mindfulness 2017 Nov 2;9(4):1117-1122. [CrossRef]
- Rosen KD, Paniagua SM, Kazanis W, Jones S, Potter JS. Quality of life among women diagnosed with breast Cancer: A randomized waitlist controlled trial of commercially available mobile app-delivered mindfulness training. Psychooncology 2018 Aug 01;27(8):2023-2030. [CrossRef] [Medline]
- Champion L, Economides M, Chandler C. The efficacy of a brief app-based mindfulness intervention on psychosocial outcomes in healthy adults: A pilot randomised controlled trial. PLoS One 2018;13(12):e0209482 [FREE Full text] [CrossRef] [Medline]
- Kubo A, Kurtovich E, McGinnis M, Aghaee S, Altschuler A, Quesenberry C, et al. A randomized controlled trial of mhealth mindfulness intervention for cancer patients and informal cancer caregivers: a feasibility study within an integrated health care delivery system. Integr Cancer Ther 2019 May 16;18:1534735419850634 [FREE Full text] [CrossRef] [Medline]
- Flett JA, Conner TS, Riordan BC, Patterson T, Hayne H. App-based mindfulness meditation for psychological distress and adjustment to college in incoming university students: a pragmatic, randomised, waitlist-controlled trial. Psychol Health 2020 Sep;35(9):1049-1074. [CrossRef] [Medline]
- Collins E, Cox A, Wilcock C, Sethu-Jones G. Digital games and mindfulness apps: comparison of effects on post work recovery. JMIR Ment Health 2019;6(7):e12853. [CrossRef] [Medline]
- Bostock S, Crosswell AD, Prather AA, Steptoe A. Mindfulness on-the-go: Effects of a mindfulness meditation app on work stress and well-being. J Occup Health Psychol 2018 May 03:127-138. [CrossRef] [Medline]
- Kirk U, Wieghorst A, Nielsen CM, Staiano W. On-the-spot binaural beats and mindfulness reduces behavioral markers of mind wandering. J Cogn Enhanc 2018 Nov 26;3(2):186-192. [CrossRef]
- Quinones C, Griffiths MD. Reducing compulsive Internet use and anxiety symptoms via two brief interventions: A comparison between mindfulness and gradual muscle relaxation. J Behav Addict 2019 Sep 01;8(3):530-536 [FREE Full text] [CrossRef] [Medline]
- Björkstrand J, Schiller D, Li J, Davidson P, Rosén J, Mårtensson J, et al. The effect of mindfulness training on extinction retention. Sci Rep 2019 Dec 27;9(1):19896 [FREE Full text] [CrossRef] [Medline]
- Flett JA, Fletcher BD, Riordan BC, Patterson T, Hayne H, Conner TS. The peril of self-reported adherence in digital interventions: A brief example. Internet Interv 2019 Dec;18:100267 [FREE Full text] [CrossRef] [Medline]
- Avalos LA, Aghaee S, Kurtovich E, Quesenberry C, Nkemere L, McGinnis MK, et al. A mobile health mindfulness intervention for women with moderate to moderately severe postpartum depressive symptoms: feasibility study. JMIR Ment Health 2020 Nov 12;7(11):e17405 [FREE Full text] [CrossRef] [Medline]
- Nübold A, Van Quaquebeke N, Hülsheger UR. Be(com)ing Real: A multi-source and an intervention study on mindfulness and authentic leadership. J Bus Psychol 2019 Jun 21;35(4):469-488. [CrossRef]
- Rung AL, Oral E, Berghammer L, Peters ES. Feasibility and acceptability of a mobile mindfulness meditation intervention among women: intervention study. JMIR Mhealth Uhealth 2020 Jun 02;8(6):e15943 [FREE Full text] [CrossRef] [Medline]
- Ball E, Newton S, Rohricht F, Steed L, Birch J, Dodds J, et al. mHealth: providing a mindfulness app for women with chronic pelvic pain in gynaecology outpatient clinics: qualitative data analysis of user experience and lessons learnt. BMJ Open 2020 Mar 12;10(3):e030711 [FREE Full text] [CrossRef] [Medline]
- Hussain A, Mkpojiogu E, Hussein I, Muhi O, Yosri M. The effectiveness, efficiency and reliability-in-use of daylio mobile app. Int J Adv Sci Technol 2020;29(8):180-187 [FREE Full text]
- Cristol S. Patient's perspective on using mobile technology as an aid to psychotherapy. JMIR Ment Health 2018 Oct 24;5(4):e10015 [FREE Full text] [CrossRef] [Medline]
- Chaudhry BM. Daylio: mood-quantification for a less stressful you. Mhealth 2016;2:34 [FREE Full text] [CrossRef] [Medline]
- Scherr S, Goering M. Is a self-monitoring app for depression a good place for additional mental health information? Ecological momentary assessment of mental help information seeking among smartphone users. Health Commun 2020 Jul 26;35(8):1004-1012. [CrossRef] [Medline]
- Inkster B, Sarda S, Subramanian V. An empathy-driven, conversational artificial intelligence agent (Wysa) for digital mental well-being: real-world data evaluation mixed-methods study. JMIR Mhealth Uhealth 2018 Nov 23;6(11):e12106 [FREE Full text] [CrossRef] [Medline]
- Kretzschmar K, Tyroll H, Pavarini G, Manzini A, Singh I, NeurOx Young People’s Advisory Group. Can your phone be your therapist? Young people's ethical perspectives on the use of fully automated conversational agents (Chatbots) in mental health support. Biomed Inform Insights 2019 Mar 05;11:1178222619829083 [FREE Full text] [CrossRef] [Medline]
- Inkster B, O'Brien R, Selby E, Joshi S, Subramanian V, Kadaba M, et al. Digital health management during and beyond the COVID-19 pandemic: Opportunities, barriers, and recommendations. JMIR Ment Health 2020 Jun 01:- [FREE Full text] [CrossRef] [Medline]
- Marcelle ET, Nolting L, Hinshaw SP, Aguilera A. Effectiveness of a Multimodal Digital Psychotherapy Platform for Adult Depression: A Naturalistic Feasibility Study. JMIR Mhealth Uhealth 2019 Jan 23;7(1):e10948 [FREE Full text] [CrossRef] [Medline]
- Moberg C, Niles A, Beermann D. Guided Self-Help Works: Randomized Waitlist Controlled Trial of Pacifica, a Mobile App Integrating Cognitive Behavioral Therapy and Mindfulness for Stress, Anxiety, and Depression. J Med Internet Res 2019 Jun 08;21(6):e12556 [FREE Full text] [CrossRef] [Medline]
- Broglia E, Millings A, Barkham M. Counseling With Guided Use of a Mobile Well-Being App for Students Experiencing Anxiety or Depression: Clinical Outcomes of a Feasibility Trial Embedded in a Student Counseling Service. JMIR Mhealth Uhealth 2019 Aug 15;7(8):e14318 [FREE Full text] [CrossRef] [Medline]
- Baumel A. Online emotional support delivered by trained volunteers: users' satisfaction and their perception of the service compared to psychotherapy. J Ment Health 2015 Oct 20;24(5):313-320. [CrossRef] [Medline]
- Baumel A, Schueller SM. Adjusting an Available Online Peer Support Platform in a Program to Supplement the Treatment of Perinatal Depression and Anxiety. JMIR Ment Health 2016 Mar 21;3(1):e11 [FREE Full text] [CrossRef] [Medline]
- Baumel A, Tinkelman A, Mathur N, Kane JM. Digital Peer-Support Platform (7Cups) as an Adjunct Treatment for Women With Postpartum Depression: Feasibility, Acceptability, and Preliminary Efficacy Study. JMIR Mhealth Uhealth 2018 Feb 13;6(2):e38 [FREE Full text] [CrossRef] [Medline]
- Baumel A, Correll CU, Birnbaum M. Adaptation of a peer based online emotional support program as an adjunct to treatment for people with schizophrenia-spectrum disorders. Internet Interv 2016 May;4:35-42 [FREE Full text] [CrossRef] [Medline]
- Malik A, Kalidhar A, Hanon C. InnerHour: Solving the Treatment Gap in Mental Health. Using technology to disrupt the mental health landscape. L'information psychiatrique 2020;96(5):348-354. [CrossRef]
- Carpenter J, Crutchley P, Zilca RD, Schwartz HA, Smith LK, Cobb AM, et al. Seeing the “Big” Picture: Big Data Methods for Exploring Relationships Between Usage, Language, and Outcome in Internet Intervention Data. J Med Internet Res 2016 Aug 31;18(8):e241 [FREE Full text] [CrossRef]
- Parks AC, Williams AL, Tugade MM, Hokes KE, Honomichl RD, Zilca RD. Testing a scalable web and smartphone based intervention to improve depression, anxiety, and resilience: A randomized controlled trial. Intnl. J. Wellbeing 2018 Dec 08;8(2):22-67. [CrossRef]
- Williams AL, Parks AC, Cormier G, Stafford J, Whillans A. Improving resilience among employees high in depression, anxiety, and workplace distress. Int J Manag Res 2018;9(1-2):4-22 [FREE Full text]
- Hunter JF, Olah MS, Williams AL, Parks AC, Pressman SD. Effect of Brief Biofeedback via a Smartphone App on Stress Recovery: Randomized Experimental Study. JMIR Serious Games 2019 Nov 26;7(4):e15974 [FREE Full text] [CrossRef] [Medline]
- Martínez-Guzmán A, Lara A. Affective modulation in positive psychology’s regime of happiness. Theory Psychol 2019 May 30;29(3):336-357. [CrossRef]
- Parks AC, Williams AL, Kackloudis GM, Stafford JL, Boucher EM, Honomichl RD. The effects of a digital well-being intervention on patients with chronic conditions: observational study. J Med Internet Res 2020 Jan 10;22(1):e16211 [FREE Full text] [CrossRef] [Medline]
- Sarkar U, Gourley GI, Lyles CR, Tieu L, Clarity C, Newmark L, et al. Usability of commercially available mobile applications for diverse patients. J Gen Intern Med 2016 Dec;31(12):1417-1426. [CrossRef] [Medline]
- Rizvi SL, Dimeff LA, Skutch J, Carroll D, Linehan MM. A pilot study of the DBT coach: an interactive mobile phone application for individuals with borderline personality disorder and substance use disorder. Behav Ther 2011 Dec;42(4):589-600. [CrossRef] [Medline]
- Rizvi SL, Hughes CD, Thomas MC. The DBT Coach mobile application as an adjunct to treatment for suicidal and self-injuring individuals with borderline personality disorder: A preliminary evaluation and challenges to client utilization. Psychol Serv 2016 Nov;13(4):380-388. [CrossRef] [Medline]
- Bush NE, Ouellette G, Kinn J. Utility of the T2 mood tracker mobile application among army warrior transition unit service members. Military Medicine 2014 Dec;179(12):1453-1457. [CrossRef]
- Dewar AR, Bull TP, Sproat JM, Reyes NP, Malvey DM, Szalma JL. Testing the reliability of a measure of motivation to engage with telehealth technology. Proceedings of the Human Factors and Ergonomics Society Annual Meeting 2016 Sep 15;60(1):1114-1118. [CrossRef]
- McCreight SJ, Brinton C, Kinn J, Bush N, Hoyt T. Integration of mCare and T2 Mood Tracker: Illustrating mHealth usability testing. J Technol Behav Sci 2019 Mar 26;4(2):130-138. [CrossRef]
- Dryman MT, McTeague LM, Olino TM, Heimberg RG. Evaluation of an open-access CBT-based Internet program for social anxiety: Patterns of use, retention, and outcomes. J Consult Clin Psychol 2017 Oct;85(10):988-999. [CrossRef] [Medline]
- Bakker D, Kazantzis N, Rickwood D, Rickard N. A randomized controlled trial of three smartphone apps for enhancing public mental health. Behav Res Ther 2018 Oct;109:75-83. [CrossRef] [Medline]
- Dahne J, Lejuez CW, Diaz VA, Player MS, Kustanowitz J, Felton JW, et al. Pilot randomized trial of a self-help behavioral activation mobile app for utilization in primary care. Behav Ther 2019 Jul;50(4):817-827. [CrossRef] [Medline]
- Kinderman P, Hagan P, King S, Bowman J, Chahal J, Gan L, et al. The feasibility and effectiveness of Catch It, an innovative CBT smartphone app. BJPsych Open 2016 May;2(3):204-209 [FREE Full text] [CrossRef] [Medline]
- Jayachandran C, Shyamala K. Kohonen SOM deployment in android app-based cognitive behavioral therapy for personality disorders. In: Proeedings of the World Congress on Computing and Communication Technologies (WCCCT). 2017 Presented at: World Congress on Computing and Communication Technologies (WCCCT); Feb. 2-4, 2017; Tiruchirappalli, India. [CrossRef]
- Flett J, Hayne H, Riordan B, Thompson LJ, Conner T. Mobile mindfulness meditation: a randomised controlled trial of the effect of two popular apps on mental health. Mindfulness 2018 Oct 31;10(5):863-876. [CrossRef]
- Krebs P, Prochaska JO, Rossi JS. A meta-analysis of computer-tailored interventions for health behavior change. Prev Med 2010;51(3-4):214-221 [FREE Full text] [CrossRef] [Medline]
- Jacobson NC, Chung YJ. Passive sensing of prediction of moment-to-moment depressed mood among undergraduates with clinical levels of depression sample using smartphones. Sensors (Basel) 2020 Jun 24;20(12):3572 [FREE Full text] [CrossRef] [Medline]
- Mitchell AJ, Vaze A, Rao S. Clinical diagnosis of depression in primary care: a meta-analysis. Lancet 2009 Aug 22;374(9690):609-619. [CrossRef] [Medline]
- Jacobson NC, Newman MG. Anxiety and depression as bidirectional risk factors for one another: A meta-analysis of longitudinal studies. Psychol Bull 2017 Nov;143(11):1155-1200. [CrossRef] [Medline]
- Chancellor S, De Choudhury M. Methods in predictive techniques for mental health status on social media: a critical review. NPJ Digit Med 2020 Mar 24;3(1):43 [FREE Full text] [CrossRef] [Medline]
- Dagum P. Digital biomarkers of cognitive function. NPJ Digit Med 2018;1:10 [FREE Full text] [CrossRef] [Medline]
- Jacobson NC, Weingarden H, Wilhelm S. Using digital phenotyping to accurately detect depression severity. J Nerv Ment Dis 2019;207(10):893-896. [CrossRef]
- Jacobson NC, Weingarden H, Wilhelm S. Digital biomarkers of mood disorders and symptom change. NPJ Digit Med 2019;2:3 [FREE Full text] [CrossRef] [Medline]
- Saeb S, Lattie EG, Kording KP, Mohr DC. Mobile phone detection of semantic location and its relationship to depression and anxiety. JMIR Mhealth Uhealth 2017 Aug 10;5(8):e112 [FREE Full text] [CrossRef] [Medline]
- Saeb S, Lattie E, Schueller S, Kording K, Mohr D. The relationship between mobile phone location sensor data and depressive symptom severity. PeerJ 2016;4:e2537 [FREE Full text] [CrossRef] [Medline]
- Wahle F, Kowatsch T, Fleisch E, Rufer M, Weidt S. Mobile sensing and support for people with depression: a pilot trial in the wild. JMIR Mhealth Uhealth 2016 Sep 21;4(3):e111 [FREE Full text] [CrossRef] [Medline]
- Lu J, Shang C, Yue C, Morillo R, Ware S, Kamath J, et al. Joint modeling of heterogeneous sensing data for depression assessment via multi-task learning. Proc ACM Interact Mob Wearable Ubiquitous Technol 2018 Mar 26;2(1):1-21. [CrossRef]
- Osmani V. Smartphones in mental health: detecting depressive and manic episodes. IEEE Pervasive Comput 2015 Jul;14(3):10-13. [CrossRef]
- Mehrotra A, Musolesi M, Hendley R, Pejovic V. Designing content-driven intelligent notification mechanisms for mobile applications. In: UbiComp '15: Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing. 2015 Presented at: UbiComp '15: The 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing; September 7 - 11, 2015; Osaka Japan p. 813-824. [CrossRef]
- Mishra V, Künzler F, Kramer J, Fleisch E, Kowatsch T, Kotz D. Detecting receptivity for mHealth interventions in the natural environment. Proc ACM Interact Mob Wearable Ubiquitous Technol 2021 Jun 23;5(2):1-24. [CrossRef]
- Künzler F, Mishra V, Kramer J, Kotz D, Fleisch E, Kowatsch T. Exploring the state-of-receptivity for mHealth interventions. Proc ACM Interact Mob Wearable Ubiquitous Technol 2019 Dec 11;3(4):1-27. [CrossRef]
- Choi W, Park S, Kim D, Lim Y, Lee U. Multi-stage receptivity model for mobile just-in-time health intervention. Proc ACM Interact Mob Wearable Ubiquitous Technol 2019 Jun 21;3(2):1-26. [CrossRef]
- Sarker H, Sharmin M, Ali A, Rahman M, Bari R, Hossain S, et al. Assessing the availability of users to engage in just-in-time intervention in the natural environment. Proc ACM Int Conf Ubiquitous Comput 2014;2014:909-920 [FREE Full text] [CrossRef] [Medline]
- Pielot M, Dingler T, Pedro J, Oliver N. When attention is not scarce - detecting boredom from mobile phone usage. In: Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing. 2015 Presented at: UbiComp '15: The 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing; September 7 - 11, 2015; Osaka Japan p. 825-836. [CrossRef]
- Sucala M, Cuijpers P, Muench F, Cardoș R, Soflau R, Dobrean A, et al. Anxiety: There is an app for that. A systematic review of anxiety apps. Depress Anxiety 2017 Jun 15;34(6):518-525. [CrossRef] [Medline]
- Van Ameringen M, Turna J, Khalesi Z, Pullia K, Patterson B. There is an app for that! The current state of mobile applications (apps) for DSM-5 obsessive-compulsive disorder, posttraumatic stress disorder, anxiety and mood disorders. Depress Anxiety 2017 Jun 01;34(6):526-539. [CrossRef] [Medline]
- Fleming T, Bavin L, Lucassen M, Stasiak K, Hopkins S, Merry S. Beyond the trial: systematic review of real-world uptake and engagement with digital self-help interventions for depression, low mood, or anxiety. J Med Internet Res 2018 Jun 06;20(6):e199 [FREE Full text] [CrossRef] [Medline]
- Huckvale K, Torous J, Larsen ME. Assessment of the data sharing and privacy practices of smartphone apps for depression and smoking cessation. JAMA Netw Open 2019 Apr 05;2(4):e192542 [FREE Full text] [CrossRef] [Medline]
- Huguet A, Rao S, McGrath PJ, Wozney L, Wheaton M, Conrod J, et al. A systematic review of cognitive behavioral therapy and behavioral activation apps for depression. PLoS One 2016;11(5):e0154248 [FREE Full text] [CrossRef] [Medline]
- Martinengo L, Van Galen L, Lum E, Kowalski M, Subramaniam M, Car J. Suicide prevention and depression apps' suicide risk assessment and management: a systematic assessment of adherence to clinical guidelines. BMC Med 2019 Dec 19;17(1):231 [FREE Full text] [CrossRef] [Medline]
- Wasil AR, Venturo-Conerly KE, Shingleton RM, Weisz JR. A review of popular smartphone apps for depression and anxiety: assessing the inclusion of evidence-based content. Behav Res Ther 2019 Dec;123:103498. [CrossRef] [Medline]
|JITAI: just-in-time adaptive intervention|
Edited by G Eysenbach; submitted 06.04.21; peer-reviewed by Y Zhang, A Teles; comments to author 23.04.21; revised version received 06.07.21; accepted 26.07.21; published 28.09.21Copyright
©Gisbert W Teepe, Ashish Da Fonseca, Birgit Kleim, Nicholas C Jacobson, Alicia Salamanca Sanabria, Lorainne Tudor Car, Elgar Fleisch, Tobias Kowatsch. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 28.09.2021.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.