Maintenance Notice

Due to necessary scheduled maintenance, the JMIR Publications website will be unavailable from Monday, March 11, 2019 at 4:00 PM to 4:30 PM EST. We apologize in advance for any inconvenience this may cause you.

Who will be affected?

Advertisement

Citing this Article

Right click to copy or hit: ctrl+c (cmd+c on mac)

Published on 07.11.19 in Vol 21, No 11 (2019): November

Preprints (earlier versions) of this paper are available at http://preprints.jmir.org/preprint/14269, first published Apr 05, 2019.

This paper is in the following e-collection/theme issue:

    Original Paper

    A New Mental Health Mobile App for Well-Being and Stress Reduction in Working Women: Randomized Controlled Trial

    1Hospital Israelita Albert Einstein, Sao Paulo, Brazil

    2Natura Cosméticos SA, Cajamar, Brazil

    *these authors contributed equally

    Corresponding Author:

    Elisa Harumi Kozasa, PhD

    Hospital Israelita Albert Einstein

    Av. Albert Einstein, 627/701 bloco A 2ºSS

    Sao Paulo, 05601-901

    Brazil

    Phone: 55 11 99179 9721

    Email: ehkozasa@gmail.com


    ABSTRACT

    Background: Although the availability and use of mobile mental health apps has grown exponentially in recent years, little data are available regarding their efficacy.

    Objective: This study aimed to evaluate the effectiveness of an app developed to promote stress management and well-being among working women compared with a control app.

    Methods: Female employees at a private hospital were invited to participate in the study via mailing lists and intranet ads. A total of 653 individuals self-enrolled through the website. Eligible participants were randomized between control (n=240) and intervention (n=250) groups. The well-being mobile app provides an 8-week program with 4 classes per week (including a brief theoretical portion and a 15-min guided practice). The active control app also provided 4 assessments per week that encouraged participants to self-observe how they were feeling for 20 min. We also used the app to conduct Web-based questionnaires (10-item Perceived Stress Scale and 5-item World Health Organization Well-Being Index) and ask specific questions to assess subjective levels of stress and well-being at baseline (t1), midintervention (t4=4 weeks after t1) and postintervention (t8=8 weeks after t1). Both apps were fully automated without any human involvement. Outcomes from the control and intervention conditions at the 3 time points were analyzed using a repeated measures analysis of variance.

    Results: Among the randomized participants (n=490), 185 participants were excluded at the 4-week follow-up and another 79 at the 8-week follow-up because of noncompliance with the experimental protocol. Participants who did not complete t4 and t8 assessments were equally distributed between groups (t4: control group=34.6% [83/240] and intervention group=40.8% [102/250]; P=.16; t8: control group=29.9% [47/157] and intervention group=21.6% [32/148]; P=.10). Both groups showed a significant increase in general well-being as a function of time (F2,426=5.27; P=.006), but only the intervention group presented a significant increase in work-related well-being (F2,426=8.92; P<.001), as well as a significant reduction in work-related and overall stress (F2,426=5.50; P=.004 and F2,426=8.59; P<.001, respectively).

    Conclusions: The well-being mobile app was effective in reducing employee stress and improving well-being.

    Trial Registration: Clinicaltrials.gov NCT02637414; https://clinicaltrials.gov/ct2/show/NCT02637414.

    J Med Internet Res 2019;21(11):e14269

    doi:10.2196/14269

    KEYWORDS



    Introduction

    Women's Mental Health

    Over the last few decades, studies have been investigating the impact of occupational stress complaints on workers’ mental health [1,2]. Currently, there is a pressing need for a greater understanding of gender as a social determinant of health in the work context [3]. According to the American Psychological Association’s report Stress in America: The State of Our Nation [4], women consistently report higher stress levels than men, are more likely to say they experience symptoms of stress, and have difficulty dealing with it. Several studies indicate that work-related factors may affect women and men differently and suggest that women may be disproportionately affected because of work and family roles [5,6]. Research from the World Economic Forum’s Gender Gap study showed that women work nearly an hour longer than men every day, when both paid and unpaid tasks are considered [7]. Therefore, these findings indicate that working women may experience more stress than men and that the sources of stress are related to the expected and actual roles of women in society. Considering the increasing numbers of women in the workforce, it is critical that more resources be directed toward reducing the negative effects of work stress on women’s health and well-being.

    Meditation and Positive Psychology

    Some very successful programs in stress management are based on the principle of Mindfulness—a metacognitive ability, defined by Jon Kabat-Zinn as “the awareness that emerges through paying attention on purpose, in the present moment, and nonjudgmentally to the unfolding experience moment to moment” [8]. The mechanism of action for mindfulness may be explained through a set of 4 interacting components [9]: (1) attention regulation (sustained attention, with returned attention on the main object of focus upon distraction), (2) body awareness (focused attention on subtle bodily sensations to enhance attunement with one’s body), (3) emotional regulation (practice of nonjudgmental awareness of one’s emotional responses in the moment), and (4) change in self-perspective (detachment from the view of an unchanging self). Although research exploring the effectiveness of online mindfulness-based interventions (MBIs) is still in its infancy, a recent meta-analysis concluded that there is emerging evidence that online MBIs have the potential to improve mental health outcomes, most notably stress [10].

    Although the mental health care system has traditionally focused more on treatment of mental disorders than on prevention, it is recognized that mental health is more than just the absence of mental illness. Positive psychology is the study of well-being, engagement, and optimal functioning, fitting well with the World Health Organization’s (WHO) definition of mental health: “a state of well-being in which the individual realizes his or her own abilities, can cope with the normal stresses of life, can work productively and fruitfully, and is able to make a contribution to his or her community” [11]. The PERMA model proposed by Seligman [12] stipulates that happiness and psychological well-being is made up of 5 core components: positive emotions, engagement (similar to the concept of mindfulness), positive relationships, meaning, and accomplishments. Well-being has been shown to reduce the risk of developing mental health alterations or disorders [13,14]. Results from a meta-analysis of 51 positive psychology interventions with more than 4000 individuals revealed that such interventions do indeed significantly enhance well-being [15]. A number of studies conducted with adult [16] and younger [17] populations have shown improvements in well-being through the use of online positive psychology interventions. Technology-based self-care programs allow individuals to circumvent some of the limitations associated with traditional support methods, such as time, resources, flexibility, accessibility, and availability [18]. Indeed, this is in line with WHO directives recommending “the promotion of self-care, for instance, through the use of electronic and mobile health technologies” [19].

    Mobile Health

    Mobile health (mHealth), which promotes the use of wireless technologies in health care, is one of the fastest growing fields within electronic health (eHealth) [20]. According to the Digital 2019 Q2 Global Digital Statshot report, 5.11 billion people own a mobile phone, which represents 66% of the world’s population [21]. About internet and mobile use in Brazil, the recently launched Digital in 2019 report [22] showed that more than 149 million out of the country’s nearly 212 million inhabitants are active internet users and that there are 215.2 million mobile connections in Brazil, which represents a penetration of 102%. Moreover, the report stated that 66% of all Brazilians are mobile internet users and that there is an average use of 34 mobile apps per month per smartphone in the country. According to the IQVIA Institute, there are currently more than 318,000 health apps worldwide, nearly double the number of apps available in 2015, with more than 200 new health apps being added to app stores every day [23]. Even with this substantial increase over the previous report [24], consumer Digital Health apps targeting wellness management still account for the majority of health apps (60%), and mental health remains the largest focus for disease-specific mobile apps (28%) [23].

    The use of a mobile device makes it possible to intervene and interact with the participants within the context (ie, their work environment) and during moments of their daily life, which is a form of intervention called ecological momentary intervention (EMI). The most reliable method for investigating real-world emotion is experience sampling, which involves contacting people as they engage in their everyday activities and asking them to report their thoughts, feelings, and actions at that moment [25]. Therefore, app-based EMIs offer a versatile, multifaceted, and interactive way of promoting training, mindfulness, self-awareness, motivation, and environmental awareness within the context of everyday life. The near ubiquitous use of smartphone apps provides a vehicle for making EMI a widespread and effective way of promoting positive change in a large nonclinical, nontherapeutic population.

    Although smartphones and mobile apps seem to be ideal tools in many aspects for providing instantly accessible interventions to promote health, the content of these apps must be based on systematic research, as without such content, any observed effects could be placebo or even harmful. Even though the availability and use of mental health mobile apps have grown exponentially in recent years, few have been thoroughly tested to provide robust scientific evidence regarding their efficacy in modulating behavior or promoting health [26-28].

    Objectives

    From these considerations, we developed 2 apps and hypothesized that a well-being mobile app, composed of an 8-week program with 4 lessons per week—based on relaxation training, breathing techniques, guided meditation, and positive psychology principles—could be effective in reducing stress and increasing well-being among working women when compared with an active control app designed to encourage self-observation and evaluation of subjective levels of stress and well-being, for the same period and weekly frequency. Here, we report the results of a randomized controlled trial in which we compared a well-being mobile app and an active control app on improving well-being and reducing stress in a group of women working in a private hospital in São Paulo, Brazil. In short, the aim of this study was to evaluate whether it is possible to improve psychological health by promoting stress management and well-being using a mobile app.


    Methods

    Trial Design

    The experimental design of this study and the format of the manuscript followed the Consolidated Standards of Reporting Trials (CONSORT) statement for reporting randomized controlled trials [29], its extension for nonpharmacological trials [30], and the CONSORT-EHEALTH checklist [31]. See Multimedia Appendix 1 for CONSORT-EHEALTH checklist (V 1.6.1). The trial was registered at ClinicalTrials.gov, number NCT02637414 (Flourishing App: Evaluation of the Effectiveness of a Well-being App for Mobile Devices), on December 11, 2015. This study was a 2-arm randomized controlled trial conducted in Brazil. Participants were randomized using a 1:1 allocation ratio to 1 of the 2 parallel groups: (1) well-being mobile app based on relaxation, breathing, meditation, and positive psychology principles and (2) control app, containing only instructions to self-observation for 20 min and recording of subjective levels of stress and well-being. Both apps were developed in Brazilian Portuguese language and were fully automated without any human involvement. Longitudinal assessments were conducted at baseline, midintervention (4 weeks after baseline), and postintervention (8 weeks after baseline).

    Participants

    We conducted this study at a large private tertiary care hospital in São Paulo, Brazil, from June 2016 to May 2017. We used mailing lists and intranet ads to invite hospital staff to participate in the study. Individuals were included if they met the following inclusion criteria: women aged between 20 and 60 years who had completed high school, owned a mobile device with either an iOS or Android operating system, and were available to participate in the 8-week training program. Potential participants first provided demographic data through an initial registration form, and those who were selected were then randomly assigned to 1 of the 2 groups (control or intervention). After that, they were given access to 1 of the 2 apps and a tutorial with instructions for its use. Excluded participants were informed via phone or email.

    Interventions

    Intervention Condition

    The well-being mobile app consists of an 8-week program divided into 2 4-week modules. Participants had to attend 4 classes per week; if they did not meet this deadline, they were given 4 more days to complete the classes, after which the app was blocked. Each class contained a brief theoretical portion and a 15-min guided practice. Twice a week, participants were asked to write down 1 good thing they had experienced and 1 good deed they had performed in a gratitude journal. They were told to do the activities whenever and wherever it felt most appropriate. Some screenshots of this app are provided in Multimedia Appendix 2.

    The well-being mobile app was designed to handle psychological stress based on relaxation training, breathing techniques, meditation (such as mindfulness, loving, kindness, and empathetic joy), and positive psychology principles. The major aim of relaxation training and the use of breathing techniques is to reduce chronic stress and enhance well-being by eliciting a relaxation response, defined as a physical state characterized by decreased arousal of the sympathetic nervous system and the opposite of the body’s stress response to perceived threats [32]. An emerging approach to increase awareness and respond skillfully to mental processes that contribute to emotional distress and maladaptive behavior is based on mindfulness meditation, a process of maintaining a moment-by-moment awareness of our thoughts, feelings, bodily sensations, and surrounding environment, without judging them [8]. Finally, positive psychology activities that aim to cultivate positive feelings, behaviors, or cognitions such as practicing optimistic thinking, expressing gratitude, practicing kindness, and replaying positive experiences are another promising approach to increase psychological well-being [33,34].

    During the first 2 weeks, the theoretical content dealt with physical and psychological consequences of chronic stress and the use of breathing techniques as a way to reduce it [32,35]. Guided practice was a technique known as body scan, which involves a gradual sweeping of attention through the entire body, focusing noncritically on any sensation or feeling in body regions and using periodic suggestions of breath awareness and relaxation [36]. The third and fourth weeks focused on mindful breathing to cultivate present moment awareness and meditation, respectively [8]. Guided practice during these 2 weeks involved meditation based on breath counting, using the breath as an anchor when the mind starts to wander [37]. The fifth week presented information on cultivating positive emotions and its impact on one’s health, life, and interpersonal relationships [12]. It included a guided loving kindness meditation, which is a method for developing the heartfelt yearning that all may find happiness [38]. During the sixth week, the theoretical content focused on the influence of positive and negative thoughts on different emotional states [12], and the guided exercise trained focusing attention and increasing awareness of one’s own repertoire of emotions, thoughts, and sensations [37]. During the seventh (second to last) week, the concept of empathy was explored [12], and the practice was a guided empathetic joy meditation aimed to train the manifestation of contentment through one’s own and others’ happiness [38]. The focus of the final week was gratitude and the different ways to cultivate it in daily routine [12], and the guided practice was mindfulness of breathing to focus on what participants were grateful for in the present moment [36]. Finally, all the practices taught during the stress management and wellness promotion program were reviewed. Toward the end of the intervention, participants were encouraged to keep practicing and to apply what they learned to everyday activities.

    Control Condition

    The control app had the same features as the well-being mobile app, including a menu, a tutorial, a profile page, evaluations, pop-up messages, and push notifications. Some screenshots of this app are provided in Multimedia Appendix 3. This active control condition followed the same period as the intervention. As in the intervention condition, participants had to answer 4 assessments per week. Each assessment was composed of a pre- and postevaluation separated by a 20-min interval. During this period, the control group received the exactly following instruction “Within 20 minutes, you will be invited to respond to these questions again. During this period, try to observe yourself and see how you are feeling,” whereas the intervention group carried out the proposed activities in the well-being mobile app. Due to this, we decided to call the active control condition Monitoring of Perceptions.

    This approach was chosen as an active control for involving a mindfulness training (being aware of their perceptions) and is significantly better than the waiting list condition, which does not consider nonspecific effects of training [39], such as received attention and demand characteristics, or the possibility of a digital placebo effect (ie, placebo-like effects seen from mobile health interventions, such as smartphone apps) [40]. Recent technological advances have led to the design of well-being interventions that include adequate experimental controls [41].

    Adherence Rates

    The adherence rates were collected for all participants by accessing their accounts. The steps of users’ navigation flow in both apps were controlled by the following 3 steps: (1) completion of the preevaluation, (2) class (intervention condition) or monitoring of perceptions (control condition), and (3) completion of the postevaluation. We had a new record in the database every time the user completed any evaluation, and we considered an activity as concluded if the user filled the respective postevaluation form. Adherence was based on the number of classes or assessments that the participant concluded during the first 4 weeks and at the end of the 8-week period of the study. Moreover, 1 push notification per day and 2 emails per week were sent to participants to engage them.

    Outcomes: Data Collection

    Primary outcome measures were taken at baseline (t1), midintervention (t4=4 weeks after t1), and postintervention (t8=8 weeks after t1). We assessed stress perception using the Brazilian Portuguese version of the 10-item perceived stress scale (PSS-10) [42,43] and subjective well-being with the Brazilian Portuguese version of the 5-item World Health Organization Well-Being Index (WHO-5) [44,45]. These Web-based questionnaires were applied using Google Forms through a link sent to participants via email. We also used an in-app questionnaire to assess subjective symptoms of stress and well-being at work and overall during the previous 30-day period. Moreover, within the app, secondary measures were gathered 4 times a week during 8 weeks, before and after the period of each class (intervention condition) or monitoring of perceptions (control condition) by assessing current subjective symptoms of stress and well-being.

    Primary Outcome Measures

    10-Item Perceived Stress Scale

    This scale was developed by Cohen et al [46] to assess the degree to which individuals perceive their life situations as stressful. The perceived stress assessed by PSS corresponds to the issue of the cognitive appraisal process, when a “situation is appraised both as threatening or otherwise demanding and as taxing or exceeding the coping resources of the person.” In summary, the PSS items check how much respondents consider their lives unpredictable, uncontrollable, and overloaded; it also includes the number of items inquiring about current levels of experienced stress. PSS is a general scale as the content of its questions is not specific to any subpopulation or age group; however, it targets participants who have completed junior high school. The 10 questions address the frequency of participants’ feelings and thoughts about events and situations that occurred during the previous 30 days. A total of 6 questions are negative (1, 2, 3, 6, 9, and 10) and 4 are positive (4, 5, 7, and 8). Each question is rated on a 5-point Likert scale from 0 (never) to 4 (very often). To calculate the total score, the 4 positive items are reverse scored and summed across all scale items. Total scores may range from 0 to 40, and higher scores indicate higher levels of perceived stress. As PSS is not a diagnostic measure and there is no official cut-off available, recent studies [47,48] used 1 standard deviation (SD 6.2) above the mean PSS-10 of 15.3 in a large working population [49] as a cut-off value to choose participants with elevated stress levels (PSS-10≥22). Heber et al [47] also calculated the clinically significant change, according to the method developed by Jacobson and Truax [50], and determined that participants changed reliably if their PSS-10 score differed by more than ±5.16 points between assessments. Finally, Ebert et al [48] considered the symptom-free status, defined at PSS-10 <17.70, as the effect outcome for the cost-effectiveness analysis of their intervention. In our sample, the Cronbach alpha for the internal reliability of the PSS-10 was .82, which is similar to that observed in the original study (alpha=.78) [42] and in the Brazilian Portuguese version of the PSS-10 (alpha=.87) [43].

    5-Item World Health Organization Well-Being Index

    WHO-5 is a short questionnaire consisting of 5 simple, noninvasive, and positively phrased items for measuring subjective well-being. Participants rate each of the 5 statements on a 6-point Likert scale, from 5 (all of the time) to 0 (at no time) to indicate how they felt during the last 2 weeks. The raw score theoretically ranges from 0 (absence of well-being) to 25 (maximal well-being), but it is recommended that one multiply the result by 4 to translate it to a percentage scale from 0 to 100, with higher scores meaning better subjective well-being. A score below 50% suggests poor emotional well-being [51], and the threshold for a clinically relevant change is considered to be 10% on standardized percentage scores [52]. A recent systematic review [52] showed that the WHO-5 is a helpful tool for clinical practice and research to evaluate well-being over time or to compare well-being between groups. The authors recommend using the general population’s mean score as a reference when using the WHO-5 as an outcome measure in clinical trials. In our sample, the Cronbach alpha was .88, which indicates a good internal validity. Similar internal consistency was found in the Brazilian Portuguese version of the WHO-5 (alpha=.83) [45].

    Subjective Symptoms of Stress and Well-Being During the Last Month

    We assessed subjective symptoms of stress and well-being at work and in general during the previous 30 days using a sliding percentage scale from 0 (absent) to 100 (maximal). Participants answered the following 4 simple questions: (1) What was your level of stress at work during the last month? (2) In general, what was your level of stress during the last month? (3) What was your level of well-being at work during the last month? and (4) In general, what was your level of well-being during the last month?

    Secondary Outcome Measures

    Subjective Symptoms of Stress and Well-Being at the Moment

    We evaluated subjective symptoms of stress and well-being before and after the period of each class (intervention condition) or monitoring of perceptions (control condition) using a sliding percentage scale from 0 (absent) to 100 (maximal). At this time, 2 simple questions were presented within the app: (1) What is your level of stress at this moment? and (2) What is your level of well-being at this moment?

    Sample Size

    The sample size was calculated according to a confidence interval of 0.95, a sampling error of 0.05, and a power effect of 0.8. From these data, a sample size calculation was conducted, and a minimum requirement of 100 participants in each group was determined.

    Recruitment, Randomization, and Blinding

    Potential participants were invited to participate by email and hospital intranet ads. Candidates were asked to access the study’s website [53], read the information package, and fill out an online registration form. Participants who met the inclusion criteria were randomly assigned to 1 of the 2 trial arms. A research assistant with no clinical involvement in the trial used the Microsoft Excel RANDBETWEEN function to randomly assign either 0 or 1 to each participant, corresponding to each experimental arm. The same research assistant generated and distributed unique access codes. The participants were aware of their allocated study arm, whereas outcome assessors and data analysts remained blind to group allocations until the completion of the study.

    Statistical Analysis

    Postintervention analyses involved a modified intention-to-treat design excluding participants who did not adequately adhere to the protocol. We developed the app to be a stress management and wellness promotion training program. As courses or trainings usually require a minimum frequency of 75% so that the participant does not fail because of absences, we decided to use this cut-off, even if it was somewhat stringent for mobile apps. Thus, only volunteers who completed 12 of the 16 activities during the first 4 weeks, completing 75% of module I, were included in the sociodemographic analysis, as well as the analysis comparing baseline with mid-intervention scores. Likewise, only volunteers who completed 23 of the 31 activities of the following 4 weeks, completing 75% of module II, were included in the analyses.

    Statistical analyses were conducted using IBM SPSS Statistics for Windows, version 24.0 (IBM Corp). Descriptive analyses were used for sociodemographic data to report absolute and relative frequencies as well as means, standard deviations, and medians. The Kolmogorov-Smirnov test was used to evaluate data normality. The Pearson chi-square test was used to compare categorical variables between groups at baseline, whereas Student t tests (or the nonparametric Wilcoxon-Mann-Whitney test, WMW) were used to compare the continuous variables. A repeated measures analysis of variance (ANOVA) was used to evaluate primary outcomes, considering 2 conditions (control and intervention) and 3 distinct periods (baseline, midintervention, and postintervention). The time × group interaction effects were assessed to investigate differences regarding the magnitude of change on dependent variables between the experimental and control groups. For the secondary outcome measures (ie, assessments made before and after each activity), we used the WMW test to compare stress and well-being level variations (∆SL and ∆WBL, respectively) between groups. The significance threshold was set at .05.

    Ethics and Consent

    Concerning data confidentiality, special efforts were made to secure online data collection and storage. Data collected using Google Forms were only accessed by the study’s first author. When these data were downloaded, other authors had access to the relevant data files. Data collected via the project website and within the app were stored on Amazon Relational Database Service. The platform used was an AWS Elastic Beanstalk, and its security criteria are available on the Web page [54]. The server stores data in an unencrypted database, but access is granted through authentication by inserting a log-in, password, and security token. All data were saved in both .xlsx and .sav formats. These files were stored in a password-protected folder on a secure server and were only available to the research team. In accordance with the Hospital Israelita Albert Einstein data retention policy, these data will be retained for 5 years after the end of the study and anonymized by replacing participant names with subject ID numbers.

    The study protocol and consent procedure were approved by the Ethics Committee of the Instituto Israelita de Ensino e Pesquisa Albert Einstein—Brazil (nº 1.469.020). Study participants were given adequate information before agreeing to participate and freely providing informed consent. Documentation of written consent was not mandatory for this study because an online consent form was used (available on the study’s website) and participants could only access the registration form if they accepted and signed the consent form [53]. It was clearly communicated that their participation was voluntary and without monetary compensation. Participants were reminded that they were free to withdraw at any time, without penalty or need for explanation, and that their data would be stored securely and anonymously. Control participants were offered access to the well-being mobile app upon study completion. Although there were no reported risks associated with the stress management and wellness promotion program or similar online interventions, the questionnaires and activities could cause discomfort for some people. Given that there were no adverse effects associated with using the app, we had no reason to discontinue the intervention for individuals who chose to continue.


    Results

    Participant Flow

    In September 2016, 653 individuals were enrolled in the study through the website and were then screened for eligibility. Among these potential participants, 163 were excluded (because they did not complete baseline measures or were unable to access the app) and the remaining 490 were randomized (see Figure 1 for a flow diagram). We excluded an additional 185 participants at the 4-week follow-up and another 79 at the 8-week follow-up because of noncompliance with the experimental protocol (ie, completed less than 75% of each module). Participants who did not complete t4 and t8 assessments were equally distributed between groups (t4: control group=34.6% [83/240] and intervention group=40.8% [102/250]; X21=2.0; P=.16; effect size for chi-square test ϕ=0.064; t8: control group=29.9% [47/157] and intervention group=21.6% [32/148]; X21=2.7; P=.10; ϕ=0.095).

    Figure 1. Consolidated Standards of Reporting Trials flow diagram. PSS-10: 10-item Perceived Stress Scale; WHO-5: 5-item World Health Organization Well-Being Index.
    View this figure

    Baseline Data

    Table 1 and Figure 2 show the baseline sociodemographic and outcome score data for both groups, respectively. Overall, the mean age of the participants was 34.6 (SD 7.62) years; 53% (n=161) were married or lived with a domestic partner; 45% (n=137) had a graduate degree; and 65% (n=197) were professionals other than nurses, physicians, or professionals in leadership positions. On average, PSS-10 scores were 22.73 (SD 6.69), and WHO-5 scores were 10.75 (SD 5.14). As seen in Table 1 and Figure 2, baseline characteristics were equivalent. See Multimedia Appendix 4 for mean (SD) and median (p25-p75) values.

    Table 1. Baseline sociodemographic data by experimental group.
    View this table
    Figure 2. Baseline outcome score data by experimental group. Scatter plots showing bivariate analyses of 10-item Perceived Stress Scale (t288=0.414; P=.68; d=0.045), 5-item World Health Organization Well-Being Index (U=10461; P=.96; r=−0.002), work-related stress (U=11078; P=.47; r=−0.041), general stress (U=11274; P=.64; r=−0.026), work-related well-being (U=11487; P=.86; r=−0.010), and general well-being (U=11060; P=.45; r=−0.043) for the control and intervention groups. Open circles show data for each participant. Red lines show group medians, except for PSS-10, which shows group means. PSS-10: 10-item Perceived Stress Scale; WHO-5: 5-item World Health Organization Well-Being Index.
    View this figure

    Outcomes

    Preintervention × Midintervention Analysis

    Efficacy results from the first 4 weeks of intervention, using a per-protocol approach, are presented in Figure 3. See Multimedia Appendix 5 for descriptive data related to repeated measures analysis of variance. Across groups, PSS-10 scores significantly decreased from preintervention (sample mean, ̄=22.35; SE=0.48) to midintervention (=19.65; SE=0.47; F1,208=41.9; P<.001; ηp2=0.168). A time × group interaction (F1,208=9.48; P=.002; ηp2=0.044) indicated that the intervention group experienced significant decreases in perceived stress compared with the control condition.

    WHO-5 scores increased for all participants as a function of time (F1,208=35.4; P<.001; ηp2=0.146). A time × group interaction was also observed (F1,208=8.54; P=.004; ηp2=0.168) with participants in the intervention condition showing a greater increase on the well-being index from preintervention to midintervention than participants in the control group.

    Work-related stress did not change as a function of time alone (F1, 280=2.54; P=.11; ηp2=0.009) but changed as a function of time × group (F1,280=6.34; P=.01; ηp2=0.022), such that people in the intervention condition reported greater improvements in work-related stress from preintervention to midintervention compared with participants in the control group.

    General stress showed a significant time effect (F1,280=10.4; P=.001; ηp2=0.036), with lower general stress ratings from preintervention (̄=57.71; SE=1.26) to midintervention (̄=52.37; SE=1.38). A significant time × group interaction emerged (F1,280=6.46; P=.01; ηp2=0.023), such that participants in the intervention condition experienced greater reductions in general stress than those in the control group.

    Work-related well-being significantly increased across participants from preintervention (̄=52.57; SE=1.22) to midintervention (̄=65.43; SE=1.34; F1,280=67.2; P<.001; ηp2=0.194), and there was a significant time × group interaction (F1,280=12.1; P=.001; ηp2=0.041), showing a greater benefit for individuals in the intervention group.

    General well-being increased as a function of time from preintervention (̄=53.53; SE=1.12) to midintervention (̄=60.72; SE=1.52; F1,280=18.5; P<.001; ηp2=0.062), but no time × group interaction was observed.

    Figure 3. Plot means with standard error bars showing all outcome data for the control (dashed black line) and intervention (solid black line) groups at pre intervention and mid intervention. Time × group interaction at 10-item Perceived Stress Scale (F1,208=9.48; P=.002; ηp2=.044), 5-item World Health Organization Well-Being Index (F1,208=8.54; P=.004; ηp2=.168), work-related stress (F1,208=6.34; P=.01; ηp2=.022), work-related well-being (F1,208=12.1; P=.001; ηp2=.041), general stress (F1,208=6.46; P=.01; ηp2=.023), and general well-being (F1,208=1.48; P=.22; ηp2=.005). PSS-10: 10-item Perceived Stress Scale; WHO-5: 5-item World Health Organization Well-Being Index.
    View this figure
    Preintervention × Midintervention × Postintervention Analysis

    Intervention efficacy results using a per-protocol approach are shown in Figure 4. See Multimedia Appendix 6 for descriptive data related to repeated measures analysis of variance. Analysis of PSS-10 scores revealed a time effect (F2,326=45.0; P<.001; ηp2=0.216), and pairwise comparisons indicated significant declines in PSS-10 scores from preintervention (̄=22.27; SE=0.56) to midintervention (̄=19.60; SE=0.53), and again from midintervention to postintervention (̄=17.88; SE=0.54). A time × group interaction (F2,326=7.19; P=.001; ηp2=0.042) indicated that the intervention group produced significant decreases in perceived stress when compared with the control group.

    WHO-5 scores increased across all participants as a function of time (F2,326=33.4; P<.001; ηp2=0.170), and pairwise comparisons indicated significant increases in WHO-5 scores from preintervention (̄=10.69; SE=0.41) to midintervention (̄=12.72; SE=0.37), and again from midintervention to postintervention (̄=13.53; SE=0.37). A time × group interaction was also observed (F2,326=7.97; P<.001; ηp2=0.047), with participants in the intervention condition showing a greater increase in well-being than participants in the control group.

    Work-related stress did not change as a function of time alone (F2,426=1.09; P=.34; ηp2=0.005) but changed as a function of time × group (F2,426=5.50; P=.004; ηp2=0.025), such that people in the intervention condition reported greater improvements in work-related stress compared with those in the control group.

    General stress showed a significant time effect (F2,426=15.3; P<.001; ηp2=0.067), and pairwise comparisons indicated significant decreases in general stress ratings from preintervention (̄=57.63; SE=1.49) to midintervention (=52.72; SE=1.61), and again from midintervention to postintervention (̄=46.60; SE=1.73). A significant time × group interaction was observed (F2,426=8.59; P<.001; ηp2=0.039), such that participants in the intervention condition experienced greater reductions in general stress than those in the control condition.

    Work-related well-being significantly increased across participants from preintervention (̄=54.55; SE=1.36) to midintervention (̄=67.03; SE=1.51) and again from midintervention to postintervention (=71.43; SE=1.47; F2,426=58.5; P<.001; ηp2=0.215), and there was a significant time × group interaction (F2,426=8.92; P<.001; ηp2=0.040), showing a greater benefit for individuals in the intervention group.

    General well-being increased as a function of time (F2,426=5.27; P=.006; ηp2=0.024), and pairwise comparisons indicated significant increases in general well-being ratings from preintervention (̄=55.08; SE=1.27) to midintervention (=61.45; SE=1.75) and to postintervention (̄=60.82; SE=1.98). Unlike the other variables analyzed thus far, no differences were observed between mid- and postintervention measurements for general well-being. There was also no time × group interaction.

    Figure 4. Plot means with standard error bars showing all outcome data for the control (dashed black line) and intervention (solid black line) groups at preintervention, midintervention, and postintervention. A time × group interaction was observed at 10-item Perceived Stress Scale (F2,326=7.19; P=.001; ηp2=.042), 5-item World Health Organization Well-Being Index (F2,326=7.97; P<.001; ηp2=.047), work-related stress (F2,426=5.50; P=.004; ηp2=.025), work-related well-being (F2,426=8.92; P<.001; ηp2=.040), general stress (F2,426=8.59; P<.001; ηp2=.039), and general well-being (F2,426=0.74; P=.47; ηp2=.003). PSS-10: 10-item Perceived Stress Scale; WHO-5: 5-item World Health Organization Well-Being Index.
    View this figure
    Changes in Subjective Symptoms of Stress and Well-Being After the Proposed Daily Activities

    For secondary outcome evaluations, we compared stress and well-being level variations (∆SL and ∆WBL, respectively) between groups after completion of 75% of the classes in module I. As expected, participants using the well-being mobile app reported a significantly greater reduction in stress levels (Table 2) and significantly greater increases in well-being levels (Table 3) than those using the active control app.

    Table 2. Between-group comparisons of stress level variations after each class.
    View this table
    Table 3. Between-group comparisons of well-being level variations after each class.
    View this table

    Discussion

    Principal Findings

    The primary aim of this study was to evaluate the effectiveness of a mobile app promoting stress management and well-being for working women. To this end, we conducted a 2-arm randomized controlled pragmatic trial with female employees at a private hospital using a long period program (8 week), a self-selected sample, and standardized questionnaires. The pragmatic trial is designed to test the effectiveness of an intervention in everyday life to maximize applicability and generalizability [55]. Groups were homogeneous regarding baseline characteristics, and although both groups showed a significant increase in general well-being, only the intervention group presented a significant increase on the well-being index (WHO-5 scores) and work-related well-being, as well as significant decreases in perceived stress (PSS-10 scores), work-related stress, and general stress. In addition, participants who used the well-being mobile app reported a significantly greater reduction in stress levels and a significant increase in well-being levels after each daily activity than participants who used the active control app. These results indicate that the well-being app was more effective at reducing employee stress and improving well-being levels.

    One of our primary outcome measures was the PSS-10, the most widely used psychological instrument for measuring the perception of stress. Our results showed that participants from both groups were severely distressed at baseline scores (control [̄=22.9; SD 6.24] and intervention ̄=22.6; SD 7.16]), and only the intervention group presented a clinically significant change in the PSS-10 score (−6.15 points, on average) at the end of the 8-week study period, reaching symptom-free status (PSS-10=15.6), as defined by Ebert et al [48].

    The other primary outcome measure used in this study was the WHO-5, one of the most widely used questionnaires to assess subjective well-being. The WHO-5 baseline score for both groups was approximately 43%, which indicates reduced well-being. After 8 weeks, the control group had a WHO-5 increase of approximately 6%, whereas improvement for the intervention group was almost 17%. Therefore, the difference in WHO-5 scores between groups was approximately 11%, which is clinically significant. However, at the end of the 8-week study period, the participants in the intervention group still had mean WHO-5 values (̄=14.93; 59.72%) below the general population reference level of 73.37% [45].

    In the field of stress research, many studies use the WHO-5 to assess a wide variety of aspects including links between working conditions and well-being [56], the association between psychosocial working conditions and psychological well-being [57], as well as the association between workplace stress and well-being [58]. Our results corroborate the findings of Goa et al [58], who found that approximately 35% (n=977) of a total of 2796 employees presented low well-being scores (WHO-5 < 50) and that women, younger workers (<40 years), workers with high educational levels, and those with higher levels of job stress reported higher rates of poor mental health.

    Several factors may contribute to the results obtained in this study. First, we applied an app-based EMI to intervene and interact with the participants in their work environment and during moments of their daily life. In addition, our system has been designed to ensure that participants respond to questions in the moment, that is, soon after being alerted to do so. This is a great advantage in relation to (1) surveys that require people to make retrospective and often generalized judgments, which tend to be affected by memory limitations and recall biases [59], and (2) laboratory experiments that do not occur within the context of a person's daily life, as we know that context can influence a person’s states and responses [60]. Self-monitoring has long been known to raise self-awareness and promote positive behavioral development under certain conditions [61,62]. Specifically, it has been theorized that being asked questions about one’s momentary states, experiences, behaviors, and/or thoughts close to the time and context of their occurrence may help one become more mindful of their occurrence, thereby providing opportunity for change [63]. Although it was only recorded as self-monitoring, participants in the control group reported that measuring stress and well-being levels before and after 20 min made them aware of changes in their physiological state (ie, they became more aware of their thought processes and more reflective). This could explain the increase in general well-being also observed in the active control group.

    Second, our 8-week training program was built on the basis of 2 central approaches aimed at reducing stress and improving well-being. As described previously, meditation was one of those foundations. Studies involving smartphone delivery of mindfulness interventions focusing on workplace stress [64,65], well-being [41,66-68], and depression [69] showed comparable results to previous traditional interventions focused on the same outcome variables. The advantage of the smartphone interventions is that they are more rigorous because of instruction standardization across participants in the experimental group, the inclusion of active control materials participants expected to benefit from, and objective measures of adherence (provided within the app) rather than self-report. The other foundation was positive psychology principles. A recent meta-analysis revealed that 67% of positive psychology interventions are delivered in a self-help format, sometimes in conjunction with face-to-face instruction and support [34]. It is already known that a shotgun approach involving multiple and different positive exercises may be more effective than engaging in only 1 activity [70]. In our well-being mobile app, we used a combination of exercises and information to develop skills across different positive psychology domains, besides providing information about the benefits of increased well-being. From a public health perspective, self-help interventions can serve as cost-effective mental health promotion tools to reach large target groups that may not otherwise be reached [71].

    Third, the theoretical basis of the intervention was confined to evidence-based components, consistent with the recommendations presented by Bakker et al [27] to create better and more rigorous mental health apps (MHapps). The chosen strategies were as follows: (1) focusing on nonclinical mental health, psychological well-being, and coping abilities, aiming to increase accessibility, enable preventive use, and reduce stigma, therefore avoiding the harmful effects of using mental illness labels [72]; (2) using self-monitoring and self-reflection—core features of many evidence-based psychological therapeutic techniques—to promote psychological growth and enable progress evaluations [73], with the advantage that MHapps make it possible for users to record self-monitoring data during their usual daily routines, while undergoing challenges or directly experiencing stressors [18]; (3) applying behavioral activation (ie, encourages individuals to engage in physiologically activating and psychologically rewarding activities) to boost self-efficacy, psychological well-being, and a repertoire of coping skills, since an app may promote self-discovery by encouraging an activity and then prompting reflection on the experience immediately after [74]; (4) presenting brief and passive psychoeducation to develop mental health literacy, in other words, to teach the participants about psychological processes underlying their distress and inform them about resources available to manage it [75]; (5) using real-time engagement to allow users to seek help for psychological challenges at the time they experience distress or soon after, thus opening new learning opportunities and applying coping strategies in ecologically valid contexts [76]; (6) promoting activities explicitly linked to specific mood problems to enhance understanding of cause-and-effect relationships between actions and emotions [77]; (7) using gamification—the use of “game-based mechanics, aesthetics and game thinking to engage people, motivate action, promote learning and solve problems” [78]—and intrinsic motivation to encourage app use via rewards and internal triggers, positive reinforcement, and behavioral conditioning, emergent approaches that may help counteract motivation problems and yield additional well-being outcomes; (8) providing reminders (email and push notifications) as external triggers for engagement, aiming to increase adherence and reduce dropout from self-help interventions [79]; and (9) conducting an experimental trial to establish the app’s efficacy before recommending it as an effective intervention.

    Donker et al [26] systematically investigated the effectiveness of MHapps. Only 8 papers (describing 5 apps) were identified as providing scientific support for MHapps. Only 1 of these was a self-contained app, whereas the other 4 required input from a mental health professional. Unfortunately, none of these apps is currently available in app stores. These findings revealed the lack of experimental evidence for MHapps, of which hundreds are available. Despite the small number of studies included in their review, Donker et al [26] concluded that although apps have the potential to benefit those with poor mental health, further studies are needed to fully understand their usefulness and effectiveness. In summary, our results show that the intervention improved subjective symptoms of well-being and reduced stress through a wide array of evidence-based techniques that were delivered via a simple, enjoyable, intuitive, and interactive design. The simplicity of a program’s interface and ease of navigation significantly influence users’ perception of the quality of Web-based mental health interventions [80]. Our results corroborated the key outcomes observed by Coulon et al [81] that evidence-based, transparent, functional, and user-friendly apps may engage patients in effective stress reduction strategies.

    Despite the proven efficacy of internet-based mobile-supported mental health interventions, high attrition rates are observed [66,82]. Nonadherence is a common issue in online psychological interventions and may reduce the effectiveness of an intervention [83]. A systematic review of attrition rates from internet-based intervention programs in which contact with a therapist was minimal reported an average dropout rate of 31% (range: 2%-83%) [84]. These data suggest that this trial had an acceptable attrition rate (54%) relative to other eHealth programs. Unfortunately, it is difficult to compare adherence levels of this trial with those of other internet-based stress management interventions because few studies thus far have reported this information. The percentage of participants in this study who completed both conditions (46%) was similar to available intervention completion rates (eg, 37%, [85]; 44%, [86]; and 88% [87]). The adherence rates of a meta-analysis of online MBIs varied between 35% and 92% [10].

    Participants who adhered perceived the intervention as manageable and felt that they had learned useful strategies. On the other hand, many nonadherent participants declared that it was not easy to make time to do the exercises, although the self-help intervention helped them recognize the importance of self-regulating attention on their daily activities and finding personal space to relax. For this reason, it is critical to analyze the potential outcomes of traditional protocols delivered by new technologies. This critical investigation allows a better understanding of whether self-help mobile apps can be beneficial to users who are seeking well-being training to obtain better mental health.

    Improving adherence is a high priority for internet interventions, as higher usage rates are associated with significant improvements in well-being [17]. Our mobile app used a one size fits all approach, which may not have been appropriate for a large group of people. Previous research indicates that providing support has a positive influence on adherence and enhances the effectiveness of online interventions [10,88]. Therefore, offering complementary instructor guidance to participants may potentially improve adherence and outcomes; however, instructor involvement is costly and may restrict the intervention’s scalability. These barriers may be partially overcome by using automated support. Automated support has been shown to improve intervention adherence and effectiveness [89].

    The authors would like to highlight that this is a well-being and stress reduction educational app and that people suffering from mental disorders or whose symptoms are difficult to manage should seek support from a mental health professional. We would also like to emphasize that this app offers an initial experience in the field of self-care with introductory classes in meditation, relaxation, attention, and positive psychology topics. Although it may be an ideal tool for those who do not have time to attend regular classes, it is not intended to substitute the guidance of experienced/certified instructors for people interested in deepening their practice.

    Limitations

    To interpret the results reported in this paper, some limitations should be taken into account. First, the main limitation of the study was that participants were not blinded, which may have increased bias. However, the designed study provided evidence of a pragmatic self-care intervention. Second, for feasibility reasons, the answers were self-reported. Third, we included only women and the volunteers self-selected into the trial, which limits the generalizability of the results. These limitations, however, should have minimal impact on the validity of the data, given the number of strengths of this study (eg, large sample size, use of an active control group, objective measures of intervention adherence, app developed for both Google’s Android and Apple’s iOS mobile operating systems).

    Future Research

    Future research should aim to replicate the results of this trial and investigate variables that may affect outcome and adherence. Personalization and tailoring intervention to individual needs, from baseline stress and well-being levels, and interactive support could contribute to increased adherence and thereby to boost the effectiveness of the intervention. As all responses here were self-reported, integrating psychophysiological sensors with the apps and collecting biological measures, such as cortisol levels, would also strengthen the evidence of the beneficial effects of the well-being mobile app. Future studies are also necessary to investigate larger and more heterogeneous samples (in healthy subjects as well as clinical populations), to assess long-term efficacy measures (eg, including follow-ups of 1 or more years), and to better establish specific effects of this particular well-being mobile-based program.

    Moreover, it would be interesting to test this protocol against the gold standard in the field (ie, face-to-face interventions) and assess which training format works best for which type of participant and under which circumstances. Although both formats may be equally effective, they may work differently on participants with different personal characteristics, and mobile-based interventions may be more advantageous in terms of efficiency and costs.

    It is important to note that no single organization regulates parameters of app efficacy or transparency. The abundance of apps that have promising descriptions but are ultimately not well developed or maintained contributes to app overload and renders people vulnerable to misdirection and discouragement as they attempt to select and adopt effective self-management strategies. Well-planned studies will improve our ability to reach the potential of mobile mental health on both individual and population levels.

    Conclusions

    Our results indicate that the well-being and stress reduction app was better than the active control app at reducing employee stress and improving well-being levels. This trial contributes to the limited evidence available regarding the feasibility and efficacy of mobile-supported stress management and well-being interventions. Moreover, it is the first study to include an active control app group. The well-being mobile app presented here was highly effective in reducing perceived stress and improving mental well-being indices over a period of 4 and 8 weeks among women working at a private hospital. Thus, self-care mobile-based interventions may be used as preventive, easily accessible, and nonstigmatizing tools in a public health environment.

    Acknowledgments

    This research was supported with a grant received from Natura Cosméticos SA, Brazil, and Instituto Israelita de Ensino e Pesquisa Albert Einstein. The funders had no role in data collection or analyses, the decision to publish, or the preparation of the manuscript. CC received a doctoral fellowship from CAPES (Brazilian Federal Agency for the Improvement of Higher Education). EHK receives a research fellowship from CNPq (National Council of Scientific and Technological Development). The authors would like to acknowledge the volunteers who were willing to participate in this research and would like to thank their colleagues, friends, and family for their valuable support, guidance, and feedback throughout the project.

    Authors' Contributions

    All authors contributed to the study design, app design, and features. AL contributed to software development. CC and AL collected the data and organized the database. CC and SL conducted the statistical analyses. CC drafted the manuscript, and all authors approved the final version of the manuscript. EHK supervised the study.

    Conflicts of Interest

    None declared.

    Multimedia Appendix 1

    CONSORT-EHEALTH checklist (V 1.6.1).

    PDF File (Adobe PDF File), 2390 KB

    Multimedia Appendix 2

    Screenshots of the well-being mobile app, showing the app on Apple’s App Store, user log-in page, pop-up messages, evaluations, home page, brief theoretical page, guided practice timer, and profile page. Image was produced by the first author.

    PNG File , 790 KB

    Multimedia Appendix 3

    Screenshots of the active control app, showing the app on Apple’s App Store, user log-in page, pop-up messages, evaluations, home page, and pre- and post evaluation separated by a 20-min interval. Image was produced by first author.

    PNG File , 695 KB

    Multimedia Appendix 4

    Baseline outcome score data by experimental group.

    PDF File (Adobe PDF File), 197 KB

    Multimedia Appendix 5

    Preintervention × midintervention between-group comparisons.

    PDF File (Adobe PDF File), 287 KB

    Multimedia Appendix 6

    Preintervention × midintervention × postintervention between-group comparisons.

    PDF File (Adobe PDF File), 292 KB

    References

    1. Colligan TW, Higgins EM. Workplace stress: etiology and consequences. J Workplace Behav Health 2006 Jul 25;21(2):89-97. [CrossRef]
    2. Wellens BT, Smith AP. Combined workplace stressors and their relationship with mood, physiology, and performance. Work Stress 2006 Jul;20(3):245-258. [CrossRef]
    3. Milner A, King T, LaMontagne AD, Bentley R, Kavanagh A. Men's work, women's work, and mental health: a longitudinal investigation of the relationship between the gender composition of occupations and mental health. Soc Sci Med 2018 May;204:16-22. [CrossRef] [Medline]
    4. American Psychological Association. 2017. Stress in America: The State of Our Nation   URL: https://www.apa.org/images/state-nation_tcm7-225609.pdf [accessed 2019-02-21]
    5. Burke RJ. Work stress and women's health: occupational status effects. J Bus Ethics 2002;37(1):91-102. [CrossRef]
    6. Olsen IB, Øverland S, Reme SE, Løvvik C. Exploring work-related causal attributions of common mental disorders. J Occup Rehabil 2015 Sep;25(3):493-505 [FREE Full text] [CrossRef] [Medline]
    7. Jesse R. Global Gender Gap Report. Switzerland: World Economic Forum; 2016.
    8. Kabat-Zinn J. Mindfulness-based interventions in context: past, present, and future. Clin Psychol Sci Pr 2003;10(2):144-156. [CrossRef]
    9. Hölzel BK, Lazar SW, Gard T, Schuman-Olivier Z, Vago DR, Ott U. How does mindfulness meditation work? Proposing mechanisms of action from a conceptual and neural perspective. Perspect Psychol Sci 2011 Nov;6(6):537-559. [CrossRef] [Medline]
    10. Spijkerman MP, Pots WT, Bohlmeijer ET. Effectiveness of online mindfulness-based interventions in improving mental health: a review and meta-analysis of randomised controlled trials. Clin Psychol Rev 2016 Apr;45:102-114 [FREE Full text] [CrossRef] [Medline]
    11. Mind Your Mind Project. 2001. Mental Health: Strengthening Mental Health Promotion   URL: https:/​/mindyourmindproject.​org/​wp-content/​uploads/​2014/​11/​WHO-Statement-on-Mental-Health-Promotion.​pdf [accessed 2019-09-14]
    12. Seligman ME. Flourish: A Visionary New Understanding of Happiness and Well-Being. New York: Free Press; 2011.
    13. Keyes CL, Dhingra SS, Simoes EJ. Change in level of positive mental health as a predictor of future risk of mental illness. Am J Public Health 2010 Dec;100(12):2366-2371. [CrossRef] [Medline]
    14. Wood AM, Joseph S. The absence of positive psychological (eudemonic) well-being as a risk factor for depression: a ten year cohort study. J Affect Disord 2010 May;122(3):213-217. [CrossRef] [Medline]
    15. Sin NL, Lyubomirsky S. Enhancing well-being and alleviating depressive symptoms with positive psychology interventions: a practice-friendly meta-analysis. J Clin Psychol 2009 May;65(5):467-487. [CrossRef] [Medline]
    16. Bolier L, Haverman M, Kramer J, Westerhof GJ, Riper H, Walburg JA, et al. An internet-based intervention to promote mental fitness for mildly depressed adults: randomized controlled trial. J Med Internet Res 2013 Sep 16;15(9):e200 [FREE Full text] [CrossRef] [Medline]
    17. Manicavasagar V, Horswood D, Burckhardt R, Lum A, Hadzi-Pavlovic D, Parker G. Feasibility and effectiveness of a web-based positive psychology program for youth mental health: randomized controlled trial. J Med Internet Res 2014 Jun 4;16(6):e140 [FREE Full text] [CrossRef] [Medline]
    18. Carissoli C, Villani D, Riva G. Does a meditation protocol supported by a mobile application help people reduce stress? Suggestions from a controlled pragmatic trial. Cyberpsychol Behav Soc Netw 2015 Jan;18(1):46-53. [CrossRef] [Medline]
    19. World Health Organization. Mental Health Action Plan 2013 - 2020. Geneva, Switzerland: World Health Organization; 2013.
    20. Free C, Phillips G, Galli L, Watson L, Felix L, Edwards P, et al. The effectiveness of mobile-health technology-based health behaviour change or disease management interventions for health care consumers: a systematic review. PLoS Med 2013;10(1):e1001362 [FREE Full text] [CrossRef] [Medline]
    21. Kemp S. DataReportal – Global Digital Insights. 2019. Digital 2019: Q2 Global Digital Statshot   URL: https://datareportal.com/reports/digital-2019-q2-global-digital-statshot [accessed 2019-05-31]
    22. Kemp S. DataReportal – Global Digital Insights. 2019. Digital 2019: Brazil   URL: https://datareportal.com/reports/digital-2019-brazil [accessed 2019-05-31]
    23. IQVIA Institute for Human Data Science. 2017. The Growing Value of Digital Health: Evidence and Impact on Human Health and the Healthcare System   URL: https://www.iqvia.com/institute/reports/the-growing-value-of-digital-health [accessed 2019-02-11]
    24. IQVIA. 2015. Patient Adoption of mHealth: Use, Evidence and Remaining Barriers to Mainstream Acceptance   URL: https://www.iqvia.com/-/media/iqvia/pdfs/institute-reports/patient-adoption-of-mhealth.pdf [accessed 2019-02-11]
    25. Killingsworth MA, Gilbert DT. A wandering mind is an unhappy mind. Science 2010 Nov 12;330(6006):932 [FREE Full text] [CrossRef] [Medline]
    26. Donker T, Petrie K, Proudfoot J, Clarke J, Birch M, Christensen H. Smartphones for smarter delivery of mental health programs: a systematic review. J Med Internet Res 2013 Nov 15;15(11):e247 [FREE Full text] [CrossRef] [Medline]
    27. Bakker D, Kazantzis N, Rickwood D, Rickard N. Mental health smartphone apps: review and evidence-based recommendations for future developments. JMIR Ment Health 2016 Mar 1;3(1):e7 [FREE Full text] [CrossRef] [Medline]
    28. McKay FH, Cheng C, Wright A, Shill J, Stephens H, Uccellini M. Evaluating mobile phone applications for health behaviour change: a systematic review. J Telemed Telecare 2018 Jan;24(1):22-30. [CrossRef] [Medline]
    29. Schulz KF, Altman DG, Moher D, CONSORT Group. CONSORT 2010 statement: updated guidelines for reporting parallel group randomised trials. Br Med J 2010 Mar 23;340:c332 [FREE Full text] [CrossRef] [Medline]
    30. Boutron I, Altman DG, Moher D, Schulz KF, Ravaud P, CONSORT NPT Group. CONSORT statement for randomized trials of nonpharmacologic treatments: a 2017 update and a CONSORT extension for nonpharmacologic trial abstracts. Ann Intern Med 2017 Jul 4;167(1):40-47. [CrossRef] [Medline]
    31. Eysenbach G, CONSORT-EHEALTH Group. CONSORT-EHEALTH: improving and standardizing evaluation reports of web-based and mobile health interventions. J Med Internet Res 2011 Dec 31;13(4):e126. [CrossRef] [Medline]
    32. Dusek JA, Benson H. Mind-body medicine: a model of the comparative clinical impact of the acute stress and relaxation responses. Minn Med 2009 May;92(5):47-50 [FREE Full text] [Medline]
    33. Seligman ME, Csikszentmihalyi M. Positive psychology. An introduction. Am Psychol 2000 Jan;55(1):5-14. [CrossRef] [Medline]
    34. Bolier L, Haverman M, Westerhof GJ, Riper H, Smit F, Bohlmeijer E. Positive psychology interventions: a meta-analysis of randomized controlled studies. BMC Public Health 2013 Feb 8;13:119 [FREE Full text] [CrossRef] [Medline]
    35. Salvagioni DA, Melanda FN, Mesas AE, González AD, Gabani FL, Andrade SM. Physical, psychological and occupational consequences of job burnout: a systematic review of prospective studies. PLoS One 2017;12(10):e0185781 [FREE Full text] [CrossRef] [Medline]
    36. Kabat-Zinn J. Full Catastrophe Living: Using the Wisdom of Your Body and Mind to Face Stress, Pain, and Illness. New York: Dell Publishing; 1990.
    37. Kabat-Zinn J. Wherever You Go, There You Are: Mindfulness Meditation for Everyday Life. New York: Hyperion; 1994.
    38. Wallace BA. Meditations of a Buddhist Skeptic: A Manifesto for the Mind Sciences and Contemplative Practice. New York, United States: Columbia University Press; 2012.
    39. Chiesa A, Calati R, Serretti A. Does mindfulness training improve cognitive abilities? A systematic review of neuropsychological findings. Clin Psychol Rev 2011 Apr;31(3):449-464. [CrossRef] [Medline]
    40. Torous J, Firth J. The digital placebo effect: mobile mental health meets clinical psychiatry. Lancet Psychiatry 2016 Feb;3(2):100-102. [CrossRef] [Medline]
    41. Howells A, Ivtzan I, Eiroa-Orosa FJ. Putting the ‘app’ in happiness: a randomised controlled trial of a smartphone-based mindfulness intervention to enhance wellbeing. J Happiness Stud 2014 Oct 29;17(1):163-185. [CrossRef]
    42. Cohen S, Williamson G. Perceived stress in a probability sample of the United States. In: Spacapan S, Oskamp S, editors. The Social Psychology of Health: The Claremont Symposium on Applied Social Psychology. Thousand Oaks, CA: Sage Publications; 1988:31-67.
    43. Reis RS, Hino AA, Añez CR. Perceived stress scale: reliability and validity study in Brazil. J Health Psychol 2010 Jan;15(1):107-114. [CrossRef] [Medline]
    44. Staehr JK. WHO/Europe. 1998. Wellbeing Measures in Primary Health Care/ The Depcare Project: Report on a WHO Meeting   URL: http://www.euro.who.int/__data/assets/pdf_file/0016/130750/E60246.pdf [accessed 2019-09-14]
    45. de Souza CM, Hidalgo MP. World Health Organization 5-item well-being index: validation of the Brazilian Portuguese version. Eur Arch Psychiatry Clin Neurosci 2012 Apr;262(3):239-244. [CrossRef] [Medline]
    46. Cohen S, Kamarck T, Mermelstein R. A global measure of perceived stress. J Health Soc Behav 1983 Dec;24(4):385-396. [CrossRef] [Medline]
    47. Heber E, Lehr D, Ebert DD, Berking M, Riper H. Web-based and mobile stress management intervention for employees: a randomized controlled trial. J Med Internet Res 2016 Jan 27;18(1):e21 [FREE Full text] [CrossRef] [Medline]
    48. Ebert DD, Kählke F, Buntrock C, Berking M, Smit F, Heber E, et al. A health economic outcome evaluation of an internet-based mobile-supported stress management intervention for employees. Scand J Work Environ Health 2018 Mar 1;44(2):171-182. [CrossRef] [Medline]
    49. Lesage F, Berjot S, Deschamps F. Psychometric properties of the French versions of the perceived stress scale. Int J Occup Med Environ Health 2012 Jun;25(2):178-184 [FREE Full text] [CrossRef] [Medline]
    50. Jacobson NS, Truax P. Clinical significance: a statistical approach to defining meaningful change in psychotherapy research. J Consult Clin Psychol 1991 Feb;59(1):12-19. [CrossRef] [Medline]
    51. Löwe B, Spitzer RL, Gräfe K, Kroenke K, Quenter A, Zipfel S, et al. Comparative validity of three screening questionnaires for DSM-IV depressive disorders and physicians' diagnoses. J Affect Disord 2004 Feb;78(2):131-140. [CrossRef] [Medline]
    52. Topp CW, Østergaard SD, Søndergaard S, Bech P. The WHO-5 Well-Being Index: a systematic review of the literature. Psychother Psychosom 2015;84(3):167-176 [FREE Full text] [CrossRef] [Medline]
    53. Florescer - Bem Vindo ao Programa.   URL: http://florescer.inovaeinstein.com.br/Site/BemVindo.php [accessed 2019-02-25]
    54. Amazon Web Services. Perguntas Frequentes Sobre O Aws Elastic Beanstalk   URL: https://aws.amazon.com/pt/elasticbeanstalk/faqs/ [accessed 2019-02-25]
    55. Patsopoulos NA. A pragmatic view on pragmatic trials. Dialogues Clin Neurosci 2011;13(2):217-224 [FREE Full text] [Medline]
    56. Nielsen K, Randall R. The importance of employee participation and perceptions of changes in procedures in a teamworking intervention. Work Stress 2012 Apr;26(2):91-111 [FREE Full text] [CrossRef] [Medline]
    57. Schütte S, Chastang JF, Malard L, Parent-Thirion A, Vermeylen G, Niedhammer I. Psychosocial working conditions and psychological well-being among employees in 34 European countries. Int Arch Occup Environ Health 2014 Nov;87(8):897-907. [CrossRef] [Medline]
    58. Gao J, Weaver SR, Dai J, Jia Y, Liu X, Jin K, et al. Workplace social capital and mental health among Chinese employees: a multi-level, cross-sectional study. PLoS One 2014;9(1):e85005 [FREE Full text] [CrossRef] [Medline]
    59. Schwarz N. Retrospective and concurrent self-reports: the rationale for real-time data capture. In: Stone A, Shiffman S, Atienza A, Nebeling L, editors. The Science of Real-Time Data Capture: Self-Reports in Health Research. Oxford, United Kingdom: Oxford University Press; 2007:471.
    60. Wilhelm P, Perrez M, Pawlik K. Conducting research in daily life: a historical review. In: Mehl MR, Conner TS, Csikszentmihalyi M, editors. Handbook of Research Methods for Studying Daily Life. New York: Guilford Press; 2011:108-123.
    61. Korotitsch WJ, Nelson-Gray RO. An overview of self-monitoring research in assessment and treatment. Psychol Assess 1999;11(4):415-425. [CrossRef]
    62. Cohen JS, Edmunds JM, Brodman DM, Benjamin CL, Kendall PC. Using self-monitoring: implementation of collaborative empiricism in cognitive-behavioral therapy. Cogn Behav Pract 2013 Nov;20(4):419-428. [CrossRef]
    63. Goodwin MS, Velicer WF, Intille SS. Telemetric monitoring in the behavior sciences. Behav Res Methods 2008 Feb;40(1):328-341. [CrossRef] [Medline]
    64. Bostock SK, Steptoe A. Can Finding Headspace Reduce Work Stress? A Randomised Controlled Workplace Trial of a Mindfulness Meditation App. In: Proceedings of the 71st Annual Scientific Meeting of the American-Psychosomatic-Society. 2013 Presented at: APS'13; March 13-16, 2013; Miami, Florida p. A36-A37.
    65. Bostock S, Crosswell AD, Prather AA, Steptoe A. Mindfulness on-the-go: effects of a mindfulness meditation app on work stress and well-being. J Occup Health Psychol 2019 Feb;24(1):127-138. [CrossRef] [Medline]
    66. Mak WW, Tong AC, Yip SY, Lui WW, Chio FH, Chan AT, et al. Efficacy and moderation of mobile app-based programs for mindfulness-based training, self-compassion training, and cognitive behavioral psychoeducation on mental health: randomized controlled noninferiority trial. JMIR Ment Health 2018 Oct 11;5(4):e60 [FREE Full text] [CrossRef] [Medline]
    67. van Emmerik AA, Berings F, Lancee J. Efficacy of a mindfulness-based mobile application: a randomized waiting-list controlled trial. Mindfulness (N Y) 2018;9(1):187-198 [FREE Full text] [CrossRef] [Medline]
    68. Walsh KM, Saab BJ, Farb NA. Effects of a mindfulness meditation app on subjective well-being: active randomized controlled trial and experience sampling study. JMIR Ment Health 2019 Jan 8;6(1):e10844 [FREE Full text] [CrossRef] [Medline]
    69. Ly KH, Trüschel A, Jarl L, Magnusson S, Windahl T, Johansson R, et al. Behavioural activation versus mindfulness-based guided self-help treatment administered through a smartphone application: a randomised controlled trial. BMJ Open 2014 Jan 9;4(1):e003440 [FREE Full text] [CrossRef] [Medline]
    70. Seligman ME, Steen TA, Park N, Peterson C. Positive psychology progress: empirical validation of interventions. Am Psychol 2005;60(5):410-421. [CrossRef] [Medline]
    71. den Boer PC, Wiersma D, van den Bosch RJ. Why is self-help neglected in the treatment of emotional disorders? A meta-analysis. Psychol Med 2004 Aug;34(6):959-971. [CrossRef] [Medline]
    72. Corrigan PW, Watson AC, Barr L. The self–stigma of mental illness: implications for self–esteem and self–efficacy. J Clin Psychol 2006 Oct;25(8):875-884. [CrossRef]
    73. Humphreys KL, Marx BP, Lexington JM. Self-monitoring as a treatment vehicle. In: O'Donohue WT, Fisher JE, editors. Cognitive Behavior Therapy: Core Principles for Practice. Ssecond Edition. Hoboken, NJ: John Wiley & Sons Inc; 2009:576-583.
    74. Martell CR, Dimidjian S, Lewinsohn PM. Behavioral activation therapy. In: Kazantzis N, Reinecke MA, Freeman A, editors. Cognitive and Behavioral Theories in Clinical Practice. New York: Guilford Press; 2010:198-223.
    75. Macrodimitris SD, Hamilton KE, Backs-Dermott BJ, Mothersill KJ. CBT basics: a group approach to teaching fundamental cognitive-behavioral skills. J Cogn Psychother 2010 May 1;24(2):132-146. [CrossRef]
    76. Heron KE, Smyth JM. Ecological momentary interventions: incorporating mobile technology into psychosocial and health behaviour treatments. Br J Health Psychol 2010 Feb;15(Pt 1):1-39 [FREE Full text] [CrossRef] [Medline]
    77. Wendel S. Designing for Behavior Change: Applying Psychology and Behavioral Economics. Sebastopol, CA: O'Reilly Media Inc; 2013.
    78. Kapp KM. The Gamification of Learning and Instruction: Game–Based Methods and Strategies for Training and Education. First Edition. San Francisco, CA: Pfeiffer; 2012.
    79. Cavanagh K. Turn on, tune in and (don't) drop out: engagement, adherence, attrition, and alliance with internet-based interventions. In: Bennett-Levy J, Richards D, Farrand P, Christensen H, Griffiths K, Kavanagh D, et al, editors. Oxford Guide to Low Intensity CBT Interventions. First Edition. Oxford, UK: Oxford University Press; 2010:227-233.
    80. Kerr C, Murray E, Stevenson F, Gore C, Nazareth I. Internet interventions for long-term conditions: patient and caregiver quality criteria. J Med Internet Res 2006 Jul 28;8(3):e13 [FREE Full text] [CrossRef] [Medline]
    81. Coulon SM, Monroe CM, West DS. A systematic, multi-domain review of mobile smartphone apps for evidence-based stress management. Am J Prev Med 2016 Jul;51(1):95-105. [CrossRef] [Medline]
    82. Eysenbach G. The law of attrition. J Med Internet Res 2005 Mar 31;7(1):e11 [FREE Full text] [CrossRef] [Medline]
    83. Donkin L, Christensen H, Naismith SL, Neal B, Hickie IB, Glozier N. A systematic review of the impact of adherence on the effectiveness of e-therapies. J Med Internet Res 2011 Aug 5;13(3):e52 [FREE Full text] [CrossRef] [Medline]
    84. Melville KM, Casey LM, Kavanagh DJ. Dropout from internet-based treatment for psychological disorders. Br J Clin Psychol 2010 Nov;49(Pt 4):455-471. [CrossRef] [Medline]
    85. Drozd F, Raeder S, Kraft P, Bjørkli CA. Multilevel growth curve analyses of treatment effects of a web-based intervention for stress reduction: randomized controlled trial. J Med Internet Res 2013 Apr 22;15(4):e84 [FREE Full text] [CrossRef] [Medline]
    86. Ly KH, Asplund K, Andersson G. Stress management for middle managers via an acceptance and commitment-based smartphone application: a randomized controlled trial. Internet Interv 2014 Jul;1(3):95-101. [CrossRef]
    87. Rose RD, Buckey JC, Zbozinek TD, Motivala SJ, Glenn DE, Cartreine JA, et al. A randomized controlled trial of a self-guided, multimedia, stress management and resilience training program. Behav Res Ther 2013 Feb;51(2):106-112. [CrossRef] [Medline]
    88. Brouwer W, Kroeze W, Crutzen R, de Nooijer J, de Vries NK, Brug J, et al. Which intervention characteristics are related to more exposure to internet-delivered healthy lifestyle promotion interventions? A systematic review. J Med Internet Res 2011 Jan 6;13(1):e2 [FREE Full text] [CrossRef] [Medline]
    89. Furmark T, Carlbring P, Hedman E, Sonnenstein A, Clevberger P, Bohman B, et al. Guided and unguided self-help for social anxiety disorder: randomised controlled trial. Br J Psychiatry 2009 Nov;195(5):440-447. [CrossRef] [Medline]


    Abbreviations

    ∆SL: stress level variation
    ∆WBL: well-being level variation
    ANOVA: analysis of variance
    AWS: Amazon Web Services
    CI: confidence interval
    CONSORT: Consolidated Standards of Reporting Trials
    eHealth: electronic health
    EMI: ecological momentary intervention
    MBIs: mindfulness-based interventions
    MHapps: mental health apps
    mHealth: mobile health
    PSS: perceived stress scale
    PSS-10: 10-item perceived stress scale
    WHO: World Health Organization
    WHO-5: 5-item World Health Organization Well-Being Index
    WMW: Wilcoxon-Mann-Whitney test


    Edited by C Eichenberg; submitted 05.04.19; peer-reviewed by S Badawy, D Vilani, S Baptista, M Hart; comments to author 25.05.19; revised version received 25.06.19; accepted 01.08.19; published 07.11.19

    ©Cássia Canha Coelhoso, Patricia Renovato Tobo, Shirley Silva Lacerda, Alex Heitor Lima, Carla Regina Camara Barrichello, Edson Amaro Jr, Elisa Harumi Kozasa. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 07.11.2019.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.