Background: Suboptimal understanding of depression and mental health disorders by the general population is an important contributor to the wide treatment gap in depression. Mental health literacy encompasses knowledge and beliefs about mental disorders and supports their recognition, management, and prevention. Besides knowledge improvement, psychoeducational interventions reduce symptoms of depression, enhance help-seeking behavior, and decrease stigma. Mental health apps often offer educational content, but the trustworthiness of the included information is unclear.
Objective: The aim of this study is to systematically evaluate adherence to clinical guidelines on depression of the information offered by mental health apps available in major commercial app stores.
Methods: A systematic assessment of the educational content regarding depression in the apps available in the Apple App Store and Google Play was conducted in July 2020. A systematic search for apps published or updated since January 2019 was performed using 42matters. Apps meeting the inclusion criteria were downloaded and assessed using two smartphones: an iPhone 7 (iOS version 14.0.1) and a Sony XPERIA XZs (Android version 8.0.0). The 156-question assessment checklist comprised general characteristics of apps, appraisal of 38 educational topics and their adherence to evidence-based clinical guidelines, as well as technical aspects and quality assurance. The results were tabulated and reported as a narrative review, using descriptive statistics.
Results: The app search retrieved 2218 apps, of which 58 were included in the analysis (Android apps: n=29, 50%; iOS apps: n=29, 50%). Of the 58 included apps, 37 (64%) apps offered educational content within a more comprehensive depression or mental health management app. Moreover, 21% (12/58) of apps provided non–evidence-based information. Furthermore, 88% (51/58) of apps included up to 20 of the educational topics, the common ones being listing the symptoms of depression (52/58, 90%) and available treatments (48/58, 83%), particularly psychotherapy. Depression-associated stigma was mentioned by 38% (22/58) of the apps, whereas suicide risk was mentioned by 71% (41/58), generally as an item in a list of symptoms. Of the 58 included apps, 44 (76%) highlighted the importance of help seeking, 29 (50%) emphasized the importance of involving the user’s support network. In addition, 52% (30/58) of apps referenced their content, and 17% (10/58) included advertisements.
Conclusions: Information in mental health and depression apps is often brief and incomplete, with 1 in 5 apps providing non–evidence-based information. Given the unmet needs and stigma associated with the disease, it is imperative that apps seize the opportunity to offer quality, evidence-based education or point the users to relevant resources. A multistakeholder consensus on a more stringent development and publication process for mental health apps is essential.
Depression affects >264 million people worldwide and was the third major contributor of years lost to disability in 2017 . Yet, approximately 50% of the people living in high-income countries and at least 75% of the people living in low- and middle-income countries [ , ] have neither been diagnosed nor receive treatment. Untreated depression is associated with increased morbidity and mortality, including death by suicide, poverty and unemployment at an individual level, and significant increase in health expenditure at health system level because of increased health care use and decreased workforce productivity of the affected individuals [ , ]. Although the reasons for this wide treatment gap are multifarious, suboptimal understanding of depression and mental health disorders in the general population is considered an important contributor to low use of mental health services. Studies assessing the general population’s mental health literacy through the use of standardized clinical vignettes have shown poor recognition of common mental health disorders [ - ].
Mental Health Literacy
Mental health literacy is defined as “the knowledge and beliefs about mental disorders, which aid their recognition, management or prevention” . It encompasses the acquisition of factual knowledge of mental health disorders as well as development of competencies and beliefs enabling prevention, early recognition of symptoms, help seeking, self-help, and provision of first aid to others [ ]. Evidence-based clinical guidelines for management of depression indicate that psychoeducation is pivotal in terms of disease management [ - ] and necessary to expand the shared-decision model of care in mental health services [ ].
Educational topics focused on depression may include information related to its natural history, symptoms and signs, treatment options such as pharmacotherapy and a range of nonpharmacological treatments and possible side effects, as well as information related to prognosis and effective self-management interventions . Psychoeducational interventions have been shown to increase mental health literacy and, most importantly, to yield small but significant reductions in symptoms of depression and mental distress [ , ], hence offering simple, inexpensive, and readily available tools for symptom management [ ].
There are other notable benefits of psychoeducation, particularly stigma reduction [- ] and increased help-seeking behavior [ , - ]. Mental health stigma remains a significant barrier to the use of mental health services because it affects the access and quality of health care provision for people living with mental health disorders and depression [ , ]. Social stigma or public stigma are linked to discrimination, avoidance, and inadequate treatment [ ], whereas self-stigma or internalized stigma may erode self-esteem and self-efficacy or lead to anger or indifference [ ], which may in turn hamper help seeking [ ]. Although multilevel coordinated and sustained efforts are needed to mitigate the prevailing mental health stigma, simple interventions such as the use of personal narratives have shown promise in fostering depression awareness [ , ] and more positive attitudes among the public [ , , ].
Mental Health Apps
Since the early 2000s, the search for health information has shifted increasingly to the web , and this trend has further intensified with the advent of smartphones [ ] and mobile apps. A recent systematic review on the provision of medical education using smartphones reported that approximately two-thirds of the reviewed interventions were effective in improving patients’ knowledge and clinical outcomes [ ]. The Apple App Store and Google Play, which are the major sources of apps worldwide [ ], currently include >10,000 mental health apps [ ] that offer a wide range of functionalities comprising education, screening, and self-management programs for a wide range of disorders [ , ]. However, most of the published apps have not been assessed in clinical trials and are not evidence based [ , ].
Given the increasing popularity of health apps, a number of studies evaluating direct-to-consumer apps have been published in recent years, including the general characteristics of the apps , key features and functionalities of highly downloaded [ ] or highly rated apps [ ], techniques commonly used in face-to-face psychotherapy [ ], or adherence of self-guided app interventions to evidence-based clinical guidelines [ ]. These studies include a variety of assessment methodologies, from researcher-developed checklists [ , , ] used in specific projects to the use of standardized assessment tools such as the Mobile App Rating Scale [ , ] or the Organization for the Review of Care and Health Applications–24 Question Assessment [ ], focusing on the usability and technical aspects of the apps. In contrast, the health-related content of apps is seldom included in assessment checklists.
Existing mental health apps often include educational components, either as its sole functionality or within a depression or mental health management app. The trustworthiness, adherence to evidence, and depth of information offered by these apps is unclear because, to date, no reviews offering an in-depth assessment of the content of education on depression modules provided by mental health apps has been published.
Therefore, this study aims to systematically evaluate the adherence to evidence-based clinical guidelines of the information on depression provided by mental health and depression apps available in the Apple App Store and Google Play.
We followed a rigorous assessment process, developed at our center and used in previous app assessment projects [- ], by adapting systematic review methodology for the app search, selection, assessment, and data analysis.
Development of the Assessment Criteria
The research team designed the assessment criteria to evaluate the clinical and technical features of apps. The criteria included the following three domains:
- General attributes, as described in the app store description, including developer, category, ratings, education-delivery format, target group, country of origin, and cost.
- Appraisal of depression education modules, comprising the scope of information provided by the app and its adherence to evidence-based clinical guidelines from the Royal Australian and New Zealand College of Psychiatrists [ ], Singapore’s Ministry of Health [ ], the United Kingdom’s National Institute for Health and Care Excellence [ ], and the American Psychiatric Association [ ], as well as concordance with expert opinions on content of mental health awareness public campaigns [ , ]. The criteria included 46 questions encompassing 38 educational topics on symptoms and diagnostic criteria of depression, natural history, importance of help seeking, treatment (including pharmacological and nonpharmacological treatments, side effects of medication, and importance of treatment adherence), stigma, recovery, and suicide prevention information and resources. [ , , , , ] presents the educational content appraisal questionnaire.
- Technical aspects and quality assurance of the app, including usability, app credibility, in-app advertisements, privacy and security safeguards, and gamification.
Selection of Apps
The Apple App Store and Google Play were systematically searched using 42matters, a proprietary app database , on July 8, 2020, using the terms depression, depressive, depressed, mood disorder, sadness, and melancholia. The search was limited to four app store categories: education, health and fitness, lifestyle, and medical. Eligible apps were required to conform to the criteria presented in .
The app selection process is presented as a flowchart () [ ]. Before screening for eligibility, we excluded all Android apps with <1000 downloads as reported in the 42matters search output. As iOS does not report the number of app downloads, the iOS versions of the excluded Android apps were also excluded on the assumption that the app would have a similar number of downloads in both app stores. The remaining apps underwent a 2-step selection process that consisted of (1) screening the app name and app store description from the 42 matters search output and (2) downloading and screening for eligibility all apps included in step 1. Working in parallel, 2 pairs of investigators (LM and Goh Jun Wei; LM and Matthew Teo Siu Yan) independently completed the app selection process. Disagreements were resolved through discussion.
Inclusion and exclusion criteria for smartphone app selection.
- Provides information about depression, including clinical presentation, diagnosis, and management (both pharmacological and nonpharmacological)
- Targets depression or includes depression information within a general mental health app
- Uploaded or updated from January 1, 2019, onward
- Downloaded at least 1000 times
- Available for free or requires payment to download, use, or expand functionalities (in-app purchases) and is available for download in the Apple App Store or Google Play
- Available in English
- Presents an overview of common physical and mental disorders, using a glossary format
- Targets health care providers (eg, physicians, psychologists, and counselors) or the support network of a person with depression or consists exclusively of peer-support forums
- Offers teleconsultation services with physicians, psychologists, counselors, or other health care providers
- Consists of a stand-alone depression screening questionnaire, without education modules
- Offers complementary medicine, meditation, or lifestyle improvement
- Does not provide any depression-related information or includes non–health-related content (ie, music playlists, wallpapers, and so on)
- Requires an access code to log in, or could not be used after 2 log in attempts because of technical problems, or was withdrawn from the app store at the time of access
App Assessment and Data Analysis
Apps were assessed by 2 assessors working in parallel using an iPhone 7 (iOS version 14.0.1) and a Sony XPERIA XZs (Android version 8.0.0) smartphone. If apps were available on both platforms, as per the 42matters search output, we assessed both versions and counted each version as an individual app. We used descriptive statistics to analyze the data. We compiled and tabulated the results and reported them as a narrative synthesis.
Subgroup Analysis of Android Apps
We performed a subgroup analysis of Android apps to assess whether the number of app downloads was associated with the educational content or the quality or number of features offered by the apps. Apps were categorized into two groups: apps downloaded 1000-10,000 times and apps downloaded >10,000 times. The assessment included selected items from all 3 sections of the assessment. The data were tabulated and compared using a significance test for categorical variables: the chi-square test was used if each category contained >10 variables and a 2-tailed Fisher exact test was used if any of the categories in the contingency table contained <10 variables. Statistical significance was set at P<.05. Statistical analyses were performed in RStudio (R version 4.0.3). iOS apps were not included in this analysis because the Apple App Store does not include this information.
The app search retrieved 2218 results after removing duplicates and excluding apps with <1000 downloads, of which 245 were downloaded and 58 were included in the analysis.describes the app selection process.
General Characteristics of Apps
presents a summary of the characteristics of the included apps. Of the 58 apps included in this analysis, 29 (50%) were Android apps and 29 (50%) were iOS apps; 21% (12/58) of the apps were available on both platforms, and 60% (35/58) of the apps belonged to the health and fitness app store category. Of the 29 Android apps, 3 (10%) had been downloaded >1 million times [ - ]. Of the 58 included apps, 15 (26%) offered only education and information modules, whereas 37 (64%) offered education modules along with other mental health or depression management features. A simple interface that did not allow for user feedback or customization was offered by 41% (12/29) of the Android apps and 31% (9/29) of the iOS apps. More than 10% of the apps offered education modules targeted to specific user groups.
|Feature||Android (n=29), n (%)||iOS (n=29), n (%)||Total (N=58), n (%)|
|App store category|
|Education||4 (14)||0 (0)||4 (7)|
|Health and fitness||17 (59)||18 (62)||35 (60)|
|Lifestyle||1 (3)||1 (3)||2 (3)|
|Medical||7 (24)||10 (34)||17 (29)|
|App store rating, stars|
|3.6 to 5||22 (76)||18 (62)||40 (69)|
|1 to 3.5||3 (10)||4 (14)||7 (12)|
|No ratings||4 (14)||7 (24)||11 (19)|
|Free||18 (62)||15 (52)||33 (57)|
|Free + in-app purchases||11 (38)||14 (48)||25 (43)|
|Paid||0 (0)||0 (0)||0 (0)|
|English||26 (90)||23 (79)||49 (84)|
|English and other languages||3 (10)||6 (21)||9 (16)|
|Target user of the app|
|No target user||25 (86)||26 (90)||51 (88)|
|Police officers||1 (3)||0 (0)||1 (2)|
|Veterans||1 (3)||1 (3)||2 (3)|
|Youth aged 12-18 yearsa||2 (7)||2 (7)||4 (7)|
|Scope of the app|
|General mental health||21 (72)||18 (62)||39 (67)|
|Depression||8 (28)||11 (38)||19 (33)|
|Type of app|
|Information and education||10 (34)||5 (17)||15 (26)|
|Disease management with education section||16 (55)||21 (72)||37 (64)|
|Education with disease management section||1 (3)||2 (7)||3 (5)|
|Multimedia education||2 (7)||1 (3)||3 (5)|
|Number of education topics|
|<10||7 (24)||6 (21)||13 (22)|
|10-20||18 (62)||20 (69)||38 (66)|
|>20||4 (14)||3 (10)||7 (12)|
|Emergency contact information for users at risk of suicide||16 (55)||19 (66)||35 (60)|
|Peer-support communities||2 (7)||2 (7)||4 (7)|
|Non–evidence-based information||10 (34)||2 (7)||12 (21)|
aYouth was defined by the cutoff age provided by the apps.
Depression Education Modules
presents a detailed description of the educational content in included apps. The apps offered a variety of information or educational topics, as summarized in . Most of the apps (51/58, 88%) included up to 20 educational topics. One in five apps provided non–evidence-based information, mostly in the form of personal opinions of the developers or columnists. Most of the apps informed users about the symptoms of depression and listed available treatments. Personal narratives on depression [ ] were included only in approximately 20% of the apps.
|Education topics included in the app||Android (n=29), n (%)||iOS (n=29), n (%)||Total (N=58), n (%)|
|General information on depression|
|Personal narratives of depression||6 (21)||6 (21)||12 (21)|
|Depression is different from sadness||20 (69)||19 (66)||39 (67)|
|Demographic and epidemiological facts||15 (52)||18 (62)||33 (57)|
|Natural history of the disease||6 (21)||12 (41)||18 (31)|
|Lists symptoms of depression||26 (90)||26 (90)||52 (90)|
|Explains what recurrence and relapse are||9 (31)||5 (17)||14 (24)|
|Addresses stigma linked to depression||11 (38)||11 (38)||22 (38)|
|Mentions suicide risk linked to depression||23 (79)||20 (69)||43 (74)|
|Screening for depression|
|Describes diagnostic criteria of depression||7 (24)||6 (21)||13 (22)|
|Provides reference to DSM-5a or ICD-10b||3 (10)||3 (10)||6 (10)|
|Administers a screening questionnaire||11 (38)||16 (55)||27 (47)|
|PHQ-9c||5 (45)||8 (50)||13 (48)|
|PHQ-9 + GAD-7d||1 (9)||0 (0)||1 (4)|
|PHQ-9 + other validated questionnaires||1 (9)||2 (13)||3 (11)|
|Other validated questionnaires||1 (9)||2 (13)||3 (11)|
|Nonvalidated questionnaires||3 (27)||4 (25)||7 (26)|
|Explains the need for a confirmatory diagnosis after screening||7 (24)||3 (10)||10 (17)|
|Treatment of depression|
|Importance of seeking help||23 (79)||21 (72)||44 (76)|
|Addresses phases and types of treatment (stepped or integrated treatment)||1 (3)||1 (3)||2 (3)|
|Advises to seek specialist treatment||22 (76)||21 (72)||43 (74)|
|Importance of involving support network||16 (55)||13(45)||29 (50)|
|Importance of complying with treatment||8 (28)||8 (28)||16 (28)|
|Lists available treatments||27 (93)||26 (90)||53 (91)|
|Treatments mentioned by the app|
|Psychotherapy||3 (10)||4 (14)||7 (12)|
|Psychotherapy + pharmacotherapy||13 (45)||10 (34)||23 (40)|
|Psychotherapy + pharmacotherapy + others||7 (24)||7 (24)||14 (24)|
|Psychotherapy + others||0 (0)||2 (7)||2 (3)|
|Complementary medicine||3 (10)||1 (3)||4 (7)|
|Others||1 (3)||2 (7)||3 (5)|
|Importance of lifestyle changes||18 (62)||21 (72)||39 (67)|
|Effects of physical exercise||19 (66)||20 (69)||39 (67)|
|Offers emergency helpline phone numbers||16 (55)||19 (66)||35 (60)|
|Prognosis in patients receiving treatment||7 (24)||13 (45)||20 (34)|
|Addresses recovery and provides a hopeful outlook||3 (10)||3 (10)||6 (10)|
aDSM-5: Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition.
bICD-10: International Classification of Diseases, Tenth Revision.
cPHQ-9: Patient Health Questionnaire, 9-item version.
dGAD-7: Generalized Anxiety Disorder, 7-item scale.
Symptoms and Natural History of Depression
Of the 58 included apps, 52 (90%) listed the symptoms of depression, whereas 39 (67%) stated the difference between depression and occasional low moods or sadness. Brief information on the epidemiology of depression was included in half of the assessed apps. Stigma associated with depression was mentioned by 38% (22/58) of the apps, whereas only 10% (6/58) included information on recovery.
Suicide risk associated with depression was mentioned by 74% (43/58) of the apps, generally as an item in a list of symptoms. Suicide prevention resources were offered by 60% (35/58) of the apps, including 55% (16/29) of the Android apps and 66% (19/29) of the iOS apps. The most common resource offered by the apps was information on crisis helplines (telephone numbers or website addresses).
Screening for Depression
Of the 58 included apps, 13 (22%) offered information on the diagnostic criteria for depression, 6 (10%) referenced psychiatric diagnostic manuals, and 10 (17%) explained the need to confirm diagnosis of depression after a positive screening test. Of the 58 apps, 27 (47%) apps administered a screening questionnaire for depression, commonly the Patient Health Questionnaire-9 . Only 48% (13/27) of the apps responded to positive screening test scores by either offering self-management exercises for the user to engage with or suggesting consulting with a health care provider, commonly a physician, whereas the rest of the apps did not provide any feedback.
Treatment of Depression
Of the 58 included apps, 44 (76%) highlighted the importance of seeking help when affected by depression and consulting specialized providers if the symptoms were severe or persistent. Half of the Android apps and one-third of the iOS apps also emphasized the importance of involving members of the user’s support network. Few apps included providers’ contact information, and when such information was present, it was limited to contact information of the app development team; none of the apps included geolocated lists of health care providers by; for example, facilitating a search through navigation providers such as Google Maps.
More than 90% of the apps listed existing treatments for depression, without providing a description of what each category entailed. Psychotherapy was the most commonly mentioned treatment category (44/58, 76%), followed by medications (37/58, 64%), whereas 12% (7/58) of the apps included brain stimulation treatments as well. Nevertheless, few apps offered more comprehensive information on the different treatment modalities, including explanations of what each treatment involves and relevant examples (eg, types of psychotherapy and antidepressant groups). Information on psychotherapy modalities was offered by 34% (10/29) of the Android apps and 52% (15/29) of the iOS apps, whereas few Android or iOS apps offered information on medication types (8/58, 14%), side effects (13/58, 22%), nonaddictive nature of frequently used antidepressants (5/58, 9%), or gradual initiation and discontinuation of therapy (10/58, 17%).
Benefits of lifestyle modifications, including regular exercise, was mentioned by 67% (39/58) of the assessed apps, whereas complementary medicine, including St John’s wort, light therapy, and acupuncture, was mentioned by 7% (4/58) of the apps.
Technical Aspects and Quality Assurance of the App
All assessed apps worked as intended and were generally easy to use. Data entry from users was not requested by 31% (9/29) of the iOS apps and 41% (12/29) of the Android apps ().
Of the 58 apps, 30 (52%) included references or the author’s signature or shared links to reputable websites for the information provided. Half of the Android apps and a quarter of the iOS apps did not declare developers’ affiliations, and 41% (24/58) of the apps included a disclaimer that the information provided did not replace a health care provider’s advice.
Of the 58 apps, 10 (17%) contained advertisements. These apps were free to download and use, except for an Android app that offered in-app purchases. The advertisements included in 75% (6/8) of the Android apps filled the screen and disrupted the use of the app, whereas 25% (2/8) of the apps included banner advertisements that allowed users to continue using the app. All advertisements were generally unrelated to the app content, promoting other Google Play apps, video games, beauty products, education support centers, and health care services such as dental services. Of these 8 Android apps, 1 (13%) promoted psychological services. In contrast, 100% (2/2) of the iOS apps offered banner advertisements that allowed for continued use of the app. The advertisements were not related to the app content.
|Characteristics||Android (n=29), n (%)||iOS (n=29), n (%)||Total (N=58), n (%)|
|App content referenced or signed by the author||17 (59)||13 (45)||30 (52)|
|Includes disclaimer: information does not replace health care provider’s advice||11 (38)||13 (45)||24 (41)|
|App development team included the following|
|Government agency or academic institution or NGOa||4 (14)||8 (28)||12 (21)|
|Health care professional||11 (38)||14 (48)||25 (43)|
|Not declared||14 (48)||7 (24)||21 (36)|
|Authentication required to access app||13 (45)||21 (72)||34 (59)|
|Presented before account creation||14 (61)||10 (45)||24 (53)|
|Explains how data are collected||23 (100)||22 (100)||45 (100)|
|Shares information with third party providers||20 (87)||20 (91)||40 (89)|
|Contact details of data protection officer provided||4 (17)||9 (41)||13 (29)|
|App allows users to share data||6 (21)||7 (24)||13 (22)|
|In-app advertisements (for details, please see text in this section)||8 (28)||2 (7)||10 (17)|
aNGO: nongovernmental organization.
Subgroup Analysis of Android Apps
Our subgroup analysis assessing the association between the Android apps’ popularity (as indexed by the number of app downloads) and the breadth and depth of the educational topics covered showed no statistically significant difference between apps downloaded <10,000 times and apps downloaded >10,000 times. Nonetheless, apps downloaded 1000-10,000 times provided non–evidence-based information more often than apps downloaded >10,000 times (7/15, 47% vs 3/14, 21%, respectively); however, this difference was not statistically significant. Of the 7 apps providing non–evidence-based information and downloaded <10,000 times, 2 (29%) belonged to the medical category in the app store. Of these 2 apps, 1 (50%) claimed that adhering to the Law of Attraction improve depression outcomes.summarizes the analysis.
|Feature||>10,000 downloads |
(n=14), n (%)
|1000-10,000 downloads |
(n=15), n (%)
|App store category||.75|
|Education||1 (7)||3 (20)|
|Health and fitness||9 (64)||8 (53)|
|Lifestyle||0 (0)||1 (7)|
|Medical||4 (29)||3 (20)|
|App store rating (stars)||.09|
|3.6 to 5||13 (93)||9 (60)|
|1 to 3.5||1 (7)||2 (13)|
|No ratings||0 (0)||4 (27)|
|Free||6 (43)||12 (80)|
|Free + in-app purchase||8 (57)||3 (20)|
|Paid||0 (0)||0 (0)|
|Type of app||.07|
|Information and education||2 (14)||8 (53)|
|Disease management with education section||10 (71)||6 (40)|
|Education disease management section||1 (7)||0 (0)|
|Multimedia education||1 (7)||1 (7)|
|Number of education topics||.66|
|<10||3 (21)||4 (27)|
|10-20||8 (57)||10 (67)|
|>20||3 (21)||1 (7)|
|Peer-support communities||2 (14)||0 (0)||.22|
|Non–evidence-based information||3 (21)||7 (47)||.25|
|General information on depression|
|Personal narratives of depression||4 (29)||2 (13)||.39|
|Depression is different from sadness||11 (79)||9 (60)||.43|
|Demographic and epidemiological facts||9 (64)||6 (40)||.35|
|Natural history of the disease||6 (43)||0 (0)||.006|
|Lists symptoms of depression||14 (100)||12 (80)||.22|
|Explains what recurrence and relapse are||4 (29)||5 (33)||.99|
|Addresses stigma linked to depression||7 (50)||4 (27)||.36|
|Mentions suicide risk linked to depression||12 (86)||11 (73)||.65|
|Screening of depression|
|Describes diagnostic criteria of depression||4 (29)||3 (20)||.69|
|Administers a screening questionnaire||7 (50)||4 (27)||.36|
|Explains need for a confirmatory diagnosis after screening||4 (29)||3 (20)||.68|
|Treatment of depression|
|Importance of seeking help||12 (86)||11 (73)||.65|
|Advises to seek specialist treatment||9 (64)||13 (87)||.21|
|Importance of involving support network||9 (64)||7 (47)||.56|
|Importance of complying with treatment||4 (29)||4 (27)||.99|
|Lists available treatments||11 (79)||12 (80)||.99|
|Importance of lifestyle changes||10 (71)||8 (53)||.53|
|Lists complementary medicine options||0 (0)||3 (20)||.22|
|Offers emergency helpline phone numbers||10 (71)||6 (40)||.18|
|Prognosis in patients receiving treatment||5 (36)||0 (0)||.02|
|Addresses recovery and provides a hopeful outlook||2 (14)||2 (13)||.99|
|App content referenced or signed by author||12 (86)||5 (33)||.012|
|Include disclaimer: information does not replace health care provider’s advice||7 (50)||4 (27)||.36|
|App development team included the following||.11|
|Government agency or academic institution or NGOb||2 (14)||2 (13)|
|Health care professional||8 (57)||3 (20)|
|Not declared||4 (29)||10 (67)|
|Authentication required to access app||9 (64)||4 (27)||.10|
|Presented before account creation||9 (64)||5 (33)||.20|
|Explains how data are collected||12 (86)||11 (73)||.65|
|Shares information with third party providers||10 (71)||10 (67)||.99|
|Contact details of data protection officer provided||4 (29)||0 (0)||.04|
|In-app advertisements||2 (14)||6 (40)||.21|
aStatistically significant values in italics.
bNGO: nongovernmental organization.
To our knowledge, this is the first systematic assessment of the information on depression in existing mental health and depression apps. Our findings suggest that the information included in these apps is often limited and not aligned with evidence. Most of the assessed apps offered brief, factual information on symptoms and treatment options, with only a few apps addressing all aspects of disease presentation and management or including information on recovery or providing personal narratives of people living with depression. The inclusion of personal accounts of people living with, and recovering from, a mental health disorder has been associated with increased understanding of the mental health disorder and recovery process, validation of the personal experience, and reduction of stigma [, ].
Patient education is one of the pillars of effective depression management resulting in better treatment compliance and improved outcomes [, ], particularly if they are complemented with personal narratives and positive messages emphasizing recovery [ ]. Adequate health literacy is associated with tolerance and acceptance of people living with a mental disorder [ ], increased help-seeking behavior [ ], and adherence to treatment [ , ], and it is a prerequisite to engage in shared-decision models of care [ ]. Yet, 25% of the potentially eligible apps, including some of the most popular mental health apps with >1 million downloads each, were excluded in the second round of screening because they did not provide any information on depression.
Access to mental health care is often inadequate, particularly in low- and middle-income countries  and among people of lower socioeconomic status as well as those living in remote areas in high-income countries [ , ].
Mental health apps may play an important role in improving access to mental health care, supported by an increasing expansion of mobile network coverage to remote areas worldwide . However, our assessment shows that apps currently available in public app marketplaces do not offer a holistic, evidence-based self-management program to meet these health needs. For example, less than half of the assessed apps include a screening test for depression and only a subset of these apps offer follow-up recommendations to users potentially living with depression. Of the 58 assessed apps, 12 (21%) offered non–evidence-based information, including developers’ or columnists’ personal views on depression, and suggested scientifically unproved treatments. Of these 12 apps, 2 (17%) were classified as medical in the app store. Lack of evidence-based information in health apps is a recurrent theme in most systematic assessments performed by our group [ , , ] and others [ , , ], revealing that current app development and publication processes overseen by app stores might not be suitable for health apps. We previously highlighted the lack of governance and quality assurance of the health app industry [ ], which we believe constrains the further development of health apps as genuine tools to support access to mental health care. Therefore, it is imperative to develop a multifaceted approach involving the research community, commercial app developers, app store managers, and official regulatory bodies to define development and publication regulatory frameworks for health apps.
Our subgroup assessment of Android apps suggests an association between the number of downloads and the overall quality of the apps. In the analysis, apps downloaded >10,000 times were more trustworthy, included more evidence-based content, presented better compliance with data privacy and security, presented references to endorse claims, and incorporated health care providers into the app development team more often than apps downloaded <10,000 times. Excluding app credibility, the differences were not statistically significant, probably because of the small size of the sample.
In this study, we used an established systematic assessment approach, which is based on the rigorous systematic review and which we have applied to various health domains [- ] over the years. This methodology included the use of a commercial database to search for eligible apps, which provides a wider geographical scope for the search, and the development of exhaustive assessment criteria using renowned evidence-based clinical guidelines and our center’s criteria for technical and quality assurance of apps.
There are limitations to our research as well. The search strategy included depression-related terms and omitted education-based terminology; thus, we may have omitted some relevant apps. However, the inclusion of such terms would have greatly increased the overall number of retrieved apps and would have also led to retrieval of many irrelevant apps because the database used for app retrieval does not allow the use of search strings; instead, it retrieves individual results for each search term. We restricted our search to apps for mental health, excluding apps offering medical or general health advice that may have presented summaries on depression presentation or management. Our assessment was limited to apps downloaded at least 1000 times, which may have omitted newly launched apps that have yet to reach the minimum required number of downloads. We also restricted our search to apps in English, potentially excluding relevant apps in languages other than English.
Information in mental health and depression apps is often brief and incomplete, with 1 in 5 apps providing non–evidence-based information. Given the unmet needs and stigma associated with depression, it is imperative that apps seize the opportunity to offer quality, evidence-based education and point users to relevant resources. A multistakeholder consensus on a more stringent development and publication process for mental health apps is essential.
The authors wish to thank Mr Matthew Teo Siu Yan and Mr Goh Jun Wei for their assistance in screening and assessing the apps. LM gratefully acknowledges Nanyang Technological University Research Scholarship (LKCMedicine) support for her PhD studentship which enabled this work. JL is supported by the Ministry of Health National Medical Research Council Clinician Scientist Award (MOH-000013). JC’s post at Imperial College London is supported by the National Institute for Health Research Northwest London Applied Research Collaboration. JC is supported by the National Research Foundation, Prime Minister’s Office, Singapore, under its Campus for Research Excellence and Technological Enterprise program.
LM had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis. LM, KG, and JC provided the concept and design of the study. LM acquired, analyzed, and interpreted the data; performed statistical analysis; and provided administrative, technical and material support. The manuscript was drafted by LM and ACS. All authors contributed for critical revision of the manuscript for important intellectual content. JC obtained the funding. KG and JC were responsible for supervision.
Conflicts of Interest
JL has received honoraria from Otsuka and Janssen. There are no other conflicts of interests.
Depression education topics assessment criteria.DOCX File , 37 KB
Characteristics of included apps.DOCX File , 39 KB
- GBD 2017 Disease and Injury Incidence and Prevalence Collaborators. Global, regional, and national incidence, prevalence, and years lived with disability for 354 diseases and injuries for 195 countries and territories, 1990-2017: a systematic analysis for the Global Burden of Disease Study 2017. Lancet 2018 Nov 10;392(10159):1789-1858 [FREE Full text] [CrossRef] [Medline]
- Kohn R, Saxena S, Levav I, Saraceno B. The treatment gap in mental health care. Bull World Health Organ 2004 Nov;82(11):858-866 [FREE Full text] [Medline]
- Luitel NP, Jordans MJ, Kohrt BA, Rathod SD, Komproe IH. Treatment gap and barriers for mental health care: a cross-sectional community survey in Nepal. PLoS One 2017;12(8):e0183223 [FREE Full text] [CrossRef] [Medline]
- König H, König HH, Konnopka A. The excess costs of depression: a systematic review and meta-analysis. Epidemiol Psychiatr Sci 2019 Apr 05;29:e30 [FREE Full text] [CrossRef] [Medline]
- Jorm AF. Mental health literacy: empowering the community to take action for better mental health. Am Psychol 2012 Apr;67(3):231-243. [CrossRef] [Medline]
- Picco L, Abdin E, Pang S, Vaingankar JA, Jeyagurunathan A, Chong SA, et al. Association between recognition and help-seeking preferences and stigma towards people with mental illness. Epidemiol Psychiatr Sci 2018 Feb;27(1):84-93 [FREE Full text] [CrossRef] [Medline]
- Jorm AF, Christensen H, Griffiths KM. The public's ability to recognize mental disorders and their beliefs about treatment: changes in Australia over 8 years. Aust N Z J Psychiatry 2006 Jan;40(1):36-41. [CrossRef] [Medline]
- Jorm AF, Korten AE, Jacomb PA, Christensen H, Rodgers B, Pollitt P. "Mental health literacy": a survey of the public's ability to recognise mental disorders and their beliefs about the effectiveness of treatment. Med J Aust 1997 Feb 17;166(4):182-186. [CrossRef] [Medline]
- Practice guideline for the treatment of patients with major depressive disorder (revision). American Psychiatric Association. Am J Psychiatry 2000 Apr;157(4 Suppl):1-45. [Medline]
- Depression The Treatment and Management of Depression in Adults (Updated Edition). Leicester (UK): British Psychological Society; 2010.
- Malhi GS, Bassett D, Boyce P, Bryant R, Fitzgerald PB, Fritz K, et al. Royal Australian and New Zealand College of Psychiatrists clinical practice guidelines for mood disorders. Aust N Z J Psychiatry 2015 Dec;49(12):1087-1206. [CrossRef] [Medline]
- Menear M, Dugas M, Careau E, Chouinard M, Dogba MJ, Gagnon M, et al. Strategies for engaging patients and families in collaborative care programs for depression and anxiety disorders: a systematic review. J Affect Disord 2020 Feb 15;263:528-539 [FREE Full text] [CrossRef] [Medline]
- Slade M. Implementing shared decision making in routine mental health care. World Psychiatry 2017 Jun;16(2):146-153 [FREE Full text] [CrossRef] [Medline]
- Donker T, Griffiths KM, Cuijpers P, Christensen H. Psychoeducation for depression, anxiety and psychological distress: a meta-analysis. BMC Med 2009 Dec 16;7:79 [FREE Full text] [CrossRef] [Medline]
- Griffiths KM, Christensen H. Internet-based mental health programs: a powerful tool in the rural medical kit. Aust J Rural Health 2007 Apr;15(2):81-87. [CrossRef] [Medline]
- Rüsch N, Evans-Lacko SE, Henderson C, Flach C, Thornicroft G. Knowledge and attitudes as predictors of intentions to seek help for and disclose a mental illness. Psychiatr Serv 2011 Jun;62(6):675-678. [CrossRef] [Medline]
- Wang J, Lai D. The relationship between mental health literacy, personal contacts and personal stigma against depression. J Affect Disord 2008 Sep;110(1-2):191-196. [CrossRef] [Medline]
- Tonsing KN. A review of mental health literacy in Singapore. Soc Work Health Care 2018 Jan;57(1):27-47. [CrossRef] [Medline]
- Bonabi H, Müller M, Ajdacic-Gross V, Eisele J, Rodgers S, Seifritz E, et al. Mental health literacy, attitudes to help seeking, and perceived need as predictors of mental health service use: a longitudinal study. J Nerv Ment Dis 2016 Apr;204(4):321-324. [CrossRef] [Medline]
- Zhuang XY, Wong DF, Cheng C, Pan S. Mental health literacy, stigma and perception of causation of mental illness among Chinese people in Taiwan. Int J Soc Psychiatry 2017 Sep;63(6):498-507. [CrossRef] [Medline]
- Cheng H, Wang C, McDermott RC, Kridel M, Rislin JL. Self-stigma, mental health literacy, and attitudes toward seeking psychological help. J Counseling Develop 2018 Jan 04;96(1):64-74. [CrossRef]
- Henderson C, Noblett J, Parke H, Clement S, Caffrey A, Gale-Grant O, et al. Mental health-related stigma in health care and mental health-care settings. Lancet Psychiatry 2014 Nov;1(6):467-482. [CrossRef] [Medline]
- Knaak S, Mantler E, Szeto A. Mental illness-related stigma in healthcare: barriers to access and care and evidence-based solutions. Healthc Manage Forum 2017 Mar;30(2):111-116 [FREE Full text] [CrossRef] [Medline]
- Corrigan PW, Watson AC. Understanding the impact of stigma on people with mental illness. World Psychiatry 2002 Feb;1(1):16-20 [FREE Full text] [Medline]
- Clement S, Schauman O, Graham T, Maggioni F, Evans-Lacko S, Bezborodovs N, et al. What is the impact of mental health-related stigma on help-seeking? A systematic review of quantitative and qualitative studies. Psychol Med 2015 Jan;45(1):11-27. [CrossRef] [Medline]
- Jorm AF, Christensen H, Griffiths KM. The impact of beyondblue: the national depression initiative on the Australian public's recognition of depression and beliefs about treatments. Aust N Z J Psychiatry 2005 Apr;39(4):248-254. [CrossRef] [Medline]
- Depression: let's talk campaign essentials. World Health Organization. URL: https://www.who.int/campaigns/world-health-day/2017/toolkit.pdf?ua=1 [accessed 2021-01-21]
- Henderson C, Evans-Lacko S, Thornicroft G. Mental illness stigma, help seeking, and public health programs. Am J Public Health 2013 May;103(5):777-780. [CrossRef] [Medline]
- Evans-Lacko S, Corker E, Williams P, Henderson C, Thornicroft G. Effect of the Time to Change anti-stigma campaign on trends in mental-illness-related public stigma among the English population in 2003-13: an analysis of survey data. Lancet Psychiatry 2014 Jul;1(2):121-128 [FREE Full text] [CrossRef] [Medline]
- Montagni I, Donisi V, Tedeschi F, Parizot I, Motrico E, Horgan A. Internet use for mental health information and support among European university students: the e-MentH project. Digit Health 2016;2:2055207616653845 [FREE Full text] [CrossRef] [Medline]
- Measuring digital development: facts and figures. International Telecommunications Union. URL: https://www.itu.int/en/ITU-D/Statistics/Documents/facts/FactsFigures2020.pdf [accessed 2021-02-09]
- Timmers T, Janssen L, Kool RB, Kremer JA. Educating patients by providing timely information using smartphone and tablet apps: systematic review. J Med Internet Res 2020 Apr 13;22(4):e17342 [FREE Full text] [CrossRef] [Medline]
- Number of apps available in leading app stores as of 1st quarter 2021. Statista. URL: https://www.statista.com/statistics/276623/number-of-apps-available-in-leading-app-stores/ [accessed 2022-02-23]
- Torous J, Roberts LW. Needed innovation in digital health and smartphone applications for mental health: transparency and trust. JAMA Psychiatry 2017 May 01;74(5):437-438. [CrossRef] [Medline]
- Van Ameringen M, Turna J, Khalesi Z, Pullia K, Patterson B. There is an app for that! The current state of mobile applications (apps) for DSM-5 obsessive-compulsive disorder, posttraumatic stress disorder, anxiety and mood disorders. Depress Anxiety 2017 Jun;34(6):526-539. [CrossRef] [Medline]
- Shang J, Wei S, Jin J, Zhang P. Mental health apps in China: analysis and quality assessment. JMIR Mhealth Uhealth 2019 Nov 07;7(11):e13236 [FREE Full text] [CrossRef] [Medline]
- Leigh S, Flatt S. App-based psychological interventions: friend or foe? Evid Based Ment Health 2015 Nov;18(4):97-99. [CrossRef] [Medline]
- Shen N, Levitan MJ, Johnson A, Bender JL, Hamilton-Page M, Jadad AA, et al. Finding a depression app: a review and content analysis of the depression app marketplace. JMIR Mhealth Uhealth 2015 Feb 16;3(1):e16 [FREE Full text] [CrossRef] [Medline]
- Baumel A, Torous J, Edan S, Kane JM. There is a non-evidence-based app for that: a systematic review and mixed methods analysis of depression- and anxiety-related apps that incorporate unrecognized techniques. J Affect Disord 2020 Aug 01;273:410-421. [CrossRef] [Medline]
- Qu C, Sas C, Daudén Roquet C, Doherty G. Functionality of top-rated mobile apps for depression: systematic search and evaluation. JMIR Ment Health 2020 Jan 24;7(1):e15321 [FREE Full text] [CrossRef] [Medline]
- Wasil AR, Venturo-Conerly KE, Shingleton RM, Weisz JR. A review of popular smartphone apps for depression and anxiety: assessing the inclusion of evidence-based content. Behav Res Ther 2019 Dec;123:103498. [CrossRef] [Medline]
- Bubolz S, Mayer G, Gronewold N, Hilbel T, Schultz JH. Adherence to established treatment guidelines among unguided digital interventions for depression: quality evaluation of 28 web-based programs and mobile apps. J Med Internet Res 2020 Jul 13;22(7):e16136 [FREE Full text] [CrossRef] [Medline]
- Tinschert P, Jakob R, Barata F, Kramer J, Kowatsch T. The potential of mobile apps for improving asthma self-management: a review of publicly available and well-adopted asthma apps. JMIR Mhealth Uhealth 2017 Aug 02;5(8):e113 [FREE Full text] [CrossRef] [Medline]
- Leigh S, Ouyang J, Mimnagh C. Effective? Engaging? Secure? Applying the ORCHA-24 framework to evaluate apps for chronic insomnia disorder. Evid Based Ment Health 2017 Nov;20(4):e20. [CrossRef] [Medline]
- Huckvale K, Adomaviciute S, Prieto JT, Leow MK, Car J. Smartphone apps for calculating insulin dose: a systematic assessment. BMC Med 2015 May 06;13:106 [FREE Full text] [CrossRef] [Medline]
- Lum E, Jimenez G, Huang Z, Thai L, Semwal M, Boehm BO, et al. Decision support and alerts of apps for self-management of blood glucose for type 2 diabetes. JAMA 2019 Apr 16;321(15):1530-1532 [FREE Full text] [CrossRef] [Medline]
- van Galen LS, Xu X, Koh MJ, Thng S, Car J. Eczema apps conformance with clinical guidelines: a systematic assessment of functions, tools and content. Br J Dermatol 2020 Feb;182(2):444-453. [CrossRef] [Medline]
- Martinengo L, Van Galen L, Lum E, Kowalski M, Subramaniam M, Car J. Suicide prevention and depression apps' suicide risk assessment and management: a systematic assessment of adherence to clinical guidelines. BMC Med 2019 Dec 19;17(1):231 [FREE Full text] [CrossRef] [Medline]
- Depression. Ministry of Health Singapore. URL: https://www.moh.gov.sg/hpp/doctors/guidelines/GuidelineDetails/cpgmed_depression [accessed 2022-02-23]
- Depression in adults: recognition and management. National Institute for Health and Care Excellence. 2009. URL: http://www.nice.org.uk/guidance/cg90 [accessed 2022-02-23]
- Clement S, Jarrett M, Henderson C, Thornicroft G. Messages to use in population-level campaigns to reduce mental health-related stigma: consensus development study. Epidemiol Psichiatr Soc 2010;19(1):72-79. [CrossRef] [Medline]
- Gronholm PC, Henderson C, Deb T, Thornicroft G. Interventions to reduce discrimination and stigma: the state of the art. Soc Psychiatry Psychiatr Epidemiol 2017 Mar;52(3):249-258 [FREE Full text] [CrossRef] [Medline]
- Thornicroft G, Mehta N, Clement S, Evans-Lacko S, Doherty M, Rose D, et al. Evidence for effective interventions to reduce mental-health-related stigma and discrimination. Lancet 2016;387(10023):1123-1132. [CrossRef] [Medline]
- All the app intelligence data you need. 42matters. URL: https://42matters.com/ [accessed 2022-02-23]
- Moher D, Liberati A, Tetzlaff J, Altman DG, PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med 2009 Jul 21;6(7):e1000097 [FREE Full text] [CrossRef] [Medline]
- Google Play. URL: https://play.google.com/store/apps/details?id=com.sevencupsoftea.app&hl=en_SG [accessed 2022-02-23]
- MindDoc companion. Google Play. URL: https://play.google.com/store/apps/details?id=de.moodpath.android&hl=en_SG [accessed 2022-02-23]
- Youper therapy. Google Play. URL: https://play.google.com/store/apps/details?id=br.com.youper&hl=en_SG [accessed 2022-02-23]
- Kroenke K, Spitzer RL, Williams JB. The PHQ-9: validity of a brief depression severity measure. J Gen Intern Med 2001 Sep;16(9):606-613 [FREE Full text] [CrossRef] [Medline]
- Rennick-Egglestone S, Morgan K, Llewellyn-Beardsley J, Ramsay A, McGranahan R, Gillard S, et al. Mental health recovery narratives and their impact on recipients: systematic review and narrative synthesis. Can J Psychiatry 2019 Oct;64(10):669-679 [FREE Full text] [CrossRef] [Medline]
- Miller TA. Health literacy and adherence to medical treatment in chronic and acute illness: a meta-analysis. Patient Educ Couns 2016 Jul;99(7):1079-1086 [FREE Full text] [CrossRef] [Medline]
- Bauer AM, Schillinger D, Parker MM, Katon W, Adler N, Adams AS, et al. Health literacy and antidepressant medication adherence among adults with diabetes: the diabetes study of Northern California (DISTANCE). J Gen Intern Med 2013 Sep;28(9):1181-1187 [FREE Full text] [CrossRef] [Medline]
- Muscat DM, Shepherd HL, Nutbeam D, Trevena L, McCaffery KJ. Health literacy and shared decision-making: exploring the relationship to enable meaningful patient engagement in healthcare. J Gen Intern Med 2021 Feb;36(2):521-524 [FREE Full text] [CrossRef] [Medline]
- Fundamental Facts about mental health 2016. Mental Health Foundation. 2016. URL: https://www.mentalhealth.org.uk/publications/fundamental-facts-about-mental-health-2016 [accessed 2020-06-17]
- Kirby JB, Zuvekas SH, Borsky AE, Ngo-Metzger Q. Rural residents with mental health needs have fewer care visits than urban counterparts. Health Aff (Millwood) 2019 Dec;38(12):2057-2060. [CrossRef] [Medline]
- Nicholas J, Larsen ME, Proudfoot J, Christensen H. Mobile apps for bipolar disorder: a systematic review of features and content quality. J Med Internet Res 2015 Aug 17;17(8):e198 [FREE Full text] [CrossRef] [Medline]
- Wisniewski H, Liu G, Henson P, Vaidyam A, Hajratalli NK, Onnela J, et al. Understanding the quality, effectiveness and attributes of top-rated smartphone health apps. Evid Based Ment Health 2019 Feb;22(1):4-9 [FREE Full text] [CrossRef] [Medline]
Edited by R Kukafka; submitted 22.03.21; peer-reviewed by K Schroeder, KL Mauco; comments to author 26.04.21; revised version received 08.05.21; accepted 16.12.21; published 09.03.22Copyright
©Laura Martinengo, Anne-Claire Stona, Lorainne Tudor Car, Jimmy Lee, Konstadina Griva, Josip Car. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 09.03.2022.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.