Current research on apps for screening and monitoring mental health has shown feasibility across diverse ranges of mental health conditions, including depression [3], bipolar disorder [7,8], anxiety disorders [9,10], and substance abuse disorders [2,5,6-11]. Furthermore, recent studies suggest that mobile phone ownership is very common among mental health patients and they have a strong interest in using mobile phones to monitor their mental health [12,13]. However, there is still very limited evidence regarding the efficacy of mobile phone-delivered mental health interventions or screening tools [14]. The limited quantity of studies conducted in this domain might be due to the lack of feasibility and low confidence that these interventions and/or screening tools will reach the targeted populations [3], and may be related to the fact that mobile phone technology is still nascent compared with other delivery channels for health interventions.

In a recent cross-sectional study, 8241 users from 66 countries from Apple’s App Store downloaded a depression-screening app [3]. A high percentage (73.9%) of app downloaders also submitted responses to the screening questionnaire [3], with 25.7% reporting that they had previously been diagnosed with depression [3]. Using two cutoff thresholds of the Patient Health Questionnaire (PHQ-9) depression screening tool, it was found that a large number of participants had high depressive symptoms yet were undiagnosed [3-15]. The studied app also reached various groups of people who were not previously diagnosed with depression yet had high depressive symptoms, and the app was able to reach a group of participants at risk of suicide [3]. In another study in South Korea, 27,159 participants were screened for bipolar spectrum disorders within a few months [8].

Active data, in the form of questionnaires for mental health screening or monitoring, can bring clinical assessments from outside of the health care setting into the real-life environment, lived and experienced by patients [16]. In addition, there are dozens of mobile phone apps that provide mental health and depression screening, but few advise users to discuss results with their health care professionals and most do not provide any guidance about how to use such results. Due to the wide popularity of screening tests on mobile phone apps and on the Internet, people are likely to access and use apps, especially if aware of their depressive symptoms. However, once they are aware of the potential seriousness of symptoms, it is critical they take appropriate action and seek professional follow-up by discussing the results with a health care professional. However, it is not yet known how effective such screening methods are in motivating users to seek appropriate professional help. As the objective of screening for disease is to discover the undiagnosed problem so that the users can be placed under treatment [17], establishing an association between mobile phone self-screening for mental health conditions and the actions taken by the users based on their screening results is a critical step in assessing the feasibility, efficacy, and cost-effectiveness of such screening and monitoring methods.

The aim of this study was to address the following question: Does a mobile phone depression-screening app motivate users with high depressive symptoms to seek health care professional advice?

This study was a single-cohort, prospective, observational study of a free mobile phone depression app that was developed in English utilizing the “Health Monitor” app template [18] and was released on Apple’s App Store. The users of the Apple’s App Store from any country can download the app after consenting to provided participant information; however, we limited the app availability to 5 countries: Australia, Canada, New Zealand (NZ), the United Kingdom (UK), and the United States (US). We selected these countries based on their high download and response rates in a previous feasibility study [3]. Users from other countries were excluded because, as provided by a previous study, the app stores function of limiting app users to specific countries in not fully accurate [2]. The research ethics committee at King Saud University approved this study.

Apple App Store users aged 18 or above, from the 5 nominated countries were recruited directly via the app’s download page in the Apple App Store. The consent and participants’ information are summarized in the app download page (Figure 1) and are included in the “about” section of the app. The studied app was published during the recruitment period (January 25 to March 25, 2015) to the Apple App Store, which was the main portal for advertising this study’s app. In addition, to boost the recruitment process, we advertised the app with demographic targeting (by country) using an in-app advertisement, that results in the app ad being displayed to Apple iPhone users while they are using other apps.

When a participant downloaded the app and ran it for the first time, the app assigned a unique device identifier to the user’s device and registered it in our secure online research database. The identifier will not change even if the user deletes the app or resets the device. This allowed anonymous data collection and prevented duplicate enrollments. After submitting demographic and baseline data on the first screen of the app, which included educational level, employment and income status, and other health-related characteristics such as chronic conditions, (Figure 2) the app users were able to complete the PHQ-9 on the second screen, including previous depression diagnosis and treatment as well as other depression risk factors, and were able to see their depression screening score. The PHQ-9 was selected over other depression screening tools because (1) it has been validated for use among various age groups [19-21] and (2) there appears to be strong correlation between scores reported from the app and scores reported on paper, with app-collected scores 3.02 (SD 2.25) points higher on average [22].

If the users get a score of 11 or more, the app recommends they discuss the test score with a health care professional. A threshold score of 11 or above was selected based on the literature, which, in pooled estimates of 10 studies had the best trade-off between sensitivity, 0.89 (95% CI 0.75 to 0.96), and specificity, 0.89 (95% CI 0.79 to 0.94) [23]. It is important to note that our app did not provide a diagnosis to the user as our main intention was to replicate the available mental health screening tests on the mobile phone app stores or the Internet. The classification of participants based on the PHQ-9 cutoff 11 was used internally and was not communicated to the participants.

In addition, all of the participants with PHQ-9 scores of 11 or above who did not indicate a previous diagnosis of depression were listed for automated follow-up after 1 month using push notifications. Push notification permits the database server to send messages to a specific user (similar to mobile short message service) at specified times or after a specific task, free of cost [24]. The 1-month follow-up period was selected arbitrarily because there are no specific guidelines regarding a definitive follow-up period but it struck a balance between clinical responsibility and effective push notification practicality. However, at the follow-up, we asked participants if they had discussed the results the app provided with a health care professional, and if so, that professional had diagnosed them with depression or not. We sent 5 push notification reminders over a 10-day period to help increase the follow-up response rate.

The mean and SD of the quantitative variables were presented if they had a normal distribution, or the median and range were presented, as appropriate, and compared using t tests. We presented the categorical variables as percentages and CIs, and compared using Pearson’s chi-square test. As this study used automated electronic data collection, there were no missing values in the baseline data; the app also includes a data integrity check to prevent users from entering invalid data (such as a maximum age of 99). We used logistic regression to explore factors associated with response rate to the follow-up and with seeking professional help.

In 2 months, 3984 users downloaded the app and 2689 out of 3984 (67.49%) completed the PHQ-9 screening. Of those who completed the screening, 151 users were not eligible for follow-up because they were not from the 5 countries targeted in this study and we were unable to identify from which country were. Thus, 2538 participants were included in this study and were eligible for follow-up. Table 1 shows the participants’ demographics.

There were 741 participants out of 2538 (29.1%) with high depressive symptoms (PHQ ≥ 11) as shown in Table 2, of which 419 (56.5%) had been previously diagnosed with depression.

A total of 322 participants, therefore, fulfilled the study criteria for follow-up reminder messages. Of those who followed up, 239 out of 322 (74.2%) completed the follow-up questions. Addressing the principal aim of this study, 91 out of 239 (38%) self-reported consulting their health care professionals about the depression score provided via the app.

Broken down by country of origin, 26 out of 53 (49%) Australian participants, who followed up, reported consulting their health care professionals regarding their PHQ-9 scores, compared with 23 out of 50 (46%) from Canada, 28 out of 85 (33%) from the United Kingdom, and only 14 out of 51 (27%) from the United States, with no differences between the countries (χ²₃= 7.4, P=.059). Eventually, 27 out of 91 (29%) participants self-reported being diagnosed with depression as a result of the consultation with health care professionals.

Logistic regression analysis incorporating demographics (Table 1) and health characteristics variables (Table 3) identified only positive attitude toward depression as a real disease as being associated with increased follow-up response rate OR 3.2, CI 1.38–8.29.

This study demonstrated that mobile phone users from a variety of countries were willing to use the depression-screening app and some acted on the results. More than one-third of the follow-up respondents acted on the recommendation provided via the app and 29% (27 out of 91) self-reported that they had received a diagnosis of depression from their health care professional at follow-up. This shows that mobile phone depression screening can influence users to discuss results obtained from a mobile phone screening test. However, we still do not know how the majority who did not sought help benefited from or used screening results.

The findings of this study on the ability of depression screening to motivate some of those screened to discuss the results with health care professionals are well matched with previous studies [25]. Although in this study we relied on the user to initiate the processes of seeking the health care professional help to interpret the results, such mobile phone mental health screening might be more effective if linked to electronic health records so that clinicians can view it to enhance the communication process. This should also follow the US Preventive Services Task Force’s recent recommendations about screening for depression in the general adult population, which also recommends adequate systems in place to ensure accurate diagnosis, effective treatment, and appropriate follow-up [26].

Different studies have shown a strong relationship between socioeconomic status and the prevalence of depression. In this study, the majority of the participants who used the depression-screening app were in a lower-income demographic and earned less than US $20,000/year [27]. Although the prevalence of depression in the United States is supposedly high and has been reported to affect approximately 1 in 5 people [27,28], there were fewer participants from the United States, 497 (19.6%), than from the United Kingdom, 1108 (43.66%). There was a similar finding, 109 (4.3%) participants in the New Zealand subgroup that had previously been reported to have a high prevalence of depression had a lower response rate in this study. Thus, variations in mobile phone use and app ranking in each country might be the major causes of such variation [29].

Australia and Canada had the highest proportions of users 49% (26 out of 53) and 46% (23 out of 50) respectively, who scored high on the app and reported going on to discuss the results with health care professionals. One of the possible reasons why more participants from Australia and Canada sought professional help could have been access to low-cost subsidized government-run health care systems. As previously mentioned, the majority of the participants had incomes of less than $20K/year, 1276 out of 2538 (50.28%). In the United States, the average direct annual per patient costs were $10,402 for bipolar patients and $7494 for depressed patients [30]. Therefore, providing free or affordable health care costs might be a major factor that motivates people with depression to seek professional help after general population screening. In addition, employees in the United States who are treated for depression incur annual per capita health and disability costs of $5415, which is significantly more than other diseases like hypertension, and treatment associated with a mean of 9.86 annual sick days, which is significantly more than other conditions [31].

There are various mobile phone apps in app stores that provide mental health or other health screening assessment tools (such as cancer screeners). However, the impact of such apps on consumer decisions is unknown. Moreover, there is no universal app store policy to responsibly direct consumers’ behavior regarding specific therapy [32]. It is recommended that app stores implement a responsible policy to force medical apps to declare that screening results obtained should be discussed with appropriate health care professionals. This criterion should also be included in the quality evaluation of any mobile phone health app. However, while positive screening results via mobile phone apps may stress a user, false negatives may also harm users, demotivating them from seeking health care advice.

Mental health screening apps need to implement functions to better motivate users to act on the provided screening results and encourage them to discuss the results with health care professionals. This study demonstrated that 148 out 239 (62%) respondents who demonstrated significant depressive symptoms did not discuss their results with a health care professional. The screening function alone might therefore not be sufficiently effective in motivating users to seek professional assistance. Improving this utility is critical when designing and implanting mobile phone technology for assistance with health care provision.

In a previous mobile phone depression-screening study, (2642/6089) 43.38% of participants completed the PHQ-9 questionnaire an average 5.3 times in the 4-month study period, with a depression score above 11 at the first test being associated with multiple PHQ-9 completions [3]. In another case study, 13 patients with major depressive disorder used a simple mobile phone app to answer 3 randomly sampled questions from the PHQ-9 survey 3 times per day for the duration of 1 month [33]. Given the feasibility of using mobile phone apps to monitor depressive symptoms, comparing a patient’s current responses to previous responses might help mental health practitioners make informed decisions [34]. Finally, although 2184 out of 2538 (86.05%) participants recorded a positive attitude toward depression as a real disease, only 1107 out of 2538 (43.61 %) had a positive attitude toward antidepressant medications’ ability to restore normal level of functioning. Future mental health mobile phone interventions may need to consider adding a component to improve attitude toward antidepressant medications.

This study focused on simulating the real-world use of such screening tools. It deliberately recruited participants the typical way mobile phone users will seek apps and provided new evidence that relevant users in various countries seek and use mobile phone-based mental health interventions. This process limited the ability to validate the self-reported data, which is less rigorous than clinically validated data. Another limitation of this study is that the passive recruitment strategy might have drawn in participants more motivated to complete the screening and the follow-up, perhaps those aware of their mental health problem. To test the feasibility of using push notifications to follow up with participants, we used a short set of follow-up questions. Therefore, this study did not provide details about the type of depression diagnosis or who provided the depression diagnosis (eg, a general practitioner or specialist). It is also unlikely that the population included in this study is representative of the depression prevalence in the population, as users had to have a mobile phone and be active app users. Moreover, the cohort design lacked a comparison or control group, which can provide results that are more rigorous. Finally, the demographics of users in this study do not reflect the national averages. For example, in the United States, depression rates were higher in 40-59 year olds and women. [35] However, in this study most of the participants were younger and more participants were male than females.

Previous studies have confirmed the feasibility of depression screening using mobile phone apps in various countries; however, it was unknown if such screening could motivate users to discuss the obtained results with health care professionals, and lead to clinical diagnosis. This study showed that a mobile phone depression-screening app could motivate some users to discuss the obtained results of such tests with health care professionals for further diagnosis and management. However, further study should investigate how other app users use the screening results provided by depression-screening apps.