This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.
Internet-based intervention platforms may improve access to mental health care for women with perinatal depression (PND). Though the majority of platforms in the market lack an evidence base, a small number of them are supported by research evidence.
This study aims to assess the current status of internet-based PND intervention platforms supported by published evidence, understand the reasons behind the disappearance of any of these previously accessible platforms, examine adjustments made by those active platforms between research trials and market implementation, and evaluate their current quality.
A cohort of internet-based PND intervention platforms was first identified by systematic searches in multiple academic databases from database inception until March 26, 2021. We searched on the World Wide Web and the iOS and Android app stores to assess which of these were available in the marketplace between April and May 2021. The basic characteristics of all platforms were collected. For inaccessible platforms, inquiries were made via email to the authors of publications to determine the reasons for their unavailability. We compared the intervention-related information of accessible platforms in the marketplace with that reported in original publications and conducted quality assessments using the App Evaluation Model of the American Psychiatric Association. Fisher exact tests were used to compare the functional characteristics in publications of available and unavailable platforms and to investigate potential associations between functional adjustments or quality indices and platform survival time.
Out of 35 platforms supported by research evidence, only 19 (54%) were still accessible in the marketplace. The main reason for platforms disappearing was the termination of research projects. No statistically significant differences were found in functional characteristics between available and unavailable platforms. A total of 18 (95%) platforms adapted their core functions from what was reported in related publications. The adjustments included changes to intervention methods (11/19, 58%), target population (10/19, 53%), human resources for intervention support (9/19, 47%), mood assessment and monitoring (8/19, 42%), communication modality (4/19, 21%), and platform type (2/19, 11%). Quality issues across platforms included low frequency of update, lack of crisis management mechanism, poor user interactivity, and weak evidence base or absence of citation of supporting evidence. Platforms that survived longer than 10 years had a higher tendency to use external resources from third parties compared to those that survived less than 10 years (
Internet-based platforms supported by evidence were not effectively translated into real-world practice. It is unclear if adjustments to accessible platforms made during actual operation may undermine the proven validity of the original research. Future research to explore the reasons behind the success of the implementation of evidence-based platforms in the marketplace is warranted.
Perinatal depression (PND) is one of the most common complications of childbirth [
One problem in this burgeoning area is that the vast majority of platforms available in the marketplace are not supported by any scientific evidence of their effectiveness. Larsen and colleagues [
In this study, we aimed to (1) identify a cohort of internet-based platforms (websites or apps) for which at least some research evidence was found in the literature; (2) track down their current statuses in the marketplace; and (3) understand the reasons behind the disappearance of previously accessible platforms. We also aimed to examine any adjustments made by these platforms to the functionality reported in the research evidence and assess the quality of currently accessible platforms against the App Evaluation Model of the American Psychiatric Association (APA’s model) [
The study cohort consisted of internet-based PND intervention platforms supported by research evidence. In this study, we defined “platform” as an internet-based technology that offers a variety of mental health interventions or services to users, accessible through mobile apps or websites on smartphones, tablets, or computers. We excluded text messaging interventions that use SMS text messaging or technology-enabled services accessed through smartwatches or virtual reality headsets from our definition. We searched PubMed, Web of Science, Cochrane Library, Embase, CNKI, Wanfang, and VIP databases using “perinatal,” “depression,” “internet,” “intervention,” and their extended terms as keywords, from database inception until March 26, 2021 (search terms are shown in the
We followed up with each eligible platform to collect relevant information regarding their real-world operations. Using information from the platform database as keywords, multiple searches were performed in the Google search engine, Apple App Store (iOS), and Google Play store (Android) between April and May 2021 to identify the corresponding platforms in the marketplace. The current status (accessible or not) of the platform was recorded. For platforms currently accessible, 2 researchers then independently collected market-related data, including platform type (website or app), estimated operating time (period between the launch of the platform and data extraction in this study), and language. For platforms that were no longer accessible at the time of follow-up, emails were addressed to the authors of the related papers, inquiring about the reason for platform inaccessibility. Our approach to contacting authors is the same as previously outlined.
The data collected in the literature and marketplace were analyzed using descriptive statistics, calculating the percentages, averages, and ranges to describe the characteristics of the platforms. We used Fisher exact tests to assess the functional differences between available and unavailable platforms, as well as potential associations between functional adjustments or quality indices and platform survival time. The statistical significance was set at α=.05. All statistical tests were conducted using IBM SPSS Statistics (version 25.0).
For platforms currently accessible, we assessed the functional adjustments by comparing intervention-related information in real-world operations with the original publications, including platform type, intervention methods, target population, human resources to support interventions, communication modality, and mood assessment and monitoring functions. The quality of accessible platforms was evaluated based on the APA’s model, which was developed by the American Psychiatric Association to assist clinicians and individuals in reviewing the risks and benefits of app use. APA’s model comprises the following 5 levels of evaluation: access and background, privacy and security, clinical foundation, usability, and data integration toward the therapeutic goal. Each level contains a series of objective indicators that address the critical standards for app evaluation as identified by mobile health leaders [
After excluding duplicates (n=4594), a total of 12,097 publications were retrieved from academic databases. A cohort of 35 platforms derived from 53 original research papers was established following the literature screening. A detailed platform database can be seen in
Flowchart of platforms included for assessment. PND: perinatal depression.
Of the 35 platforms supported by published literature, only 19 were currently accessible, with an average estimated operating time of 12 years. The authors of 7 of the 16 inaccessible platforms provided the precise out-of-service time, from which we calculated an average estimated operating time of 2.1 years. The basic characteristics of 35 platforms can be found in
Basic characteristics of perinatal depression internet-based intervention platforms.
|
Platforms included (N=35) | Platforms available (n=19) | Platforms unavailable (n=16) | |
|
||||
|
Website only | 19 | 11 | 8 |
|
App only | 13 | 5 | 8 |
|
Both website and app | 3 | 3 | 0 |
|
||||
|
Mean | 9.2a | 12.0 | 2.1a |
|
||||
|
0-5 | 12 | 5 | 7a |
|
5-10 | 3 | 3 | 0 |
|
10-15 | 5 | 5 | 0 |
|
≥15 | 6 | 6 | 0 |
|
||||
|
English | 20 | 15 | 5 |
|
Others | 12 | 4 | 8 |
|
Unknown | 3 | 0 | 3 |
|
||||
|
Single | 23 | 12 | 11 |
|
Multiple | 9 | 7 | 2 |
|
Unknown | 3 | 0 | 3 |
aData for unavailable platforms were calculated based on the platforms to which the author replied to the email (n=7).
The authors of 10 of the 16 inaccessible platforms replied to our emails and confirmed the reasons for unavailability were the termination of the research project (n=9) and a technology incompatibility issue (n=1).
Functional characteristics of included platforms in literature.
Functional characteristics | Platforms available (n=19), n (%) | Platforms unavailable (n=16), n (%) | ||
|
>.99 | |||
|
Psychotherapy (CBTa, BAb, PSTc, etc) | 9 (47.4) | 7 (43.7) | |
|
Nonpsychotherapy | 10 (52.6) | 9 (56.3) | |
|
.25 | |||
|
Antenatal | 5 (26.3) | 9 (56.3) | |
|
Postnatal | 9 (47.4) | 5 (31.2) | |
|
Perinatal | 5 (26.3) | 2 (12.5) | |
|
.32 | |||
|
Yes | 11 (57.9) | 6 (37.5) | |
|
No | 8 (42.1) | 10 (62.5) | |
|
.50 | |||
|
Yes | 6 (31.6) | 7 (43.7) | |
|
No | 13 (68.4) | 9 (56.3) | |
|
.24 | |||
|
Yes | 6 (31.6) | 2 (12.5) | |
|
No | 13 (68.4) | 14 (87.5) | |
|
.11 | |||
|
Website only | 13 (68.4) | 8 (50) | |
|
App only | 4 (21.1) | 8 (50) | |
|
Both website and app | 2 (10.5) | 0 (0) |
aCBT: cognitive behavioral therapy.
bBA: behavioral activation therapy.
cPST: problem solving therapy.
A total of 18 of the 19 still-accessible platforms adjusted their actual functions in the market in comparison to those reported in the publications (
Functional adjustments of available platforms in the market compared to those reported in the original publications. 7CS: 7 Cups; BAM: Be a mom; BBCT: Babycenter; GES: Ginger Emotional Support; HSPA: Headspace; IPA: iParent; IPK: Internetpsykiatri; MDF: Mindful-ouderschap; MMB: Mindful Mood Balance; MMM: Mamma Mia; MMOM: Mindmom; MOMB: MomMoodBooste; MTUM: MUMentum; MUMB: MumMoodBooster; NMUM: Netmums; SFMM: Strongest FamiliesManaging Our Mood (MOM); SNS: Sunnyside; UDY: Udaya; VDM: Veedamom.
Detailed results of the quality assessment of all accessible platforms following the APA’s model can be seen in
All platforms had specific developers, including private companies (n=5), academic organizations (n=5), and medical health providers (n=3). In addition, there were also 6 platforms with multiple types of developers, 5 of which were led by academic organizations in collaboration with government, private companies, and medical health providers respectively, and the remaining 1 was led by a private company in cooperation with individuals that had living experience. Eleven of the 19 platforms were exclusively accessible through websites; 5 were only delivered with apps; and 3 offered both. Two of the 8 platforms offering apps were exclusive to either Android or iOS devices, while 6 were available on both devices. Only 6 platforms have been updated within 6 months. Twelve of the 19 platforms were entirely free, while the remaining 7 charged, mostly on a weekly, monthly, or quarterly basis, according to the services, with an average weekly cost of US $14.17. Two platforms would charge extra for personalized services on top of the base rate.
The majority of platforms disclosed user-related specifications; 14 platforms included a privacy policy, and 13 provided terms of use. A total of 16 platforms claimed that they collect, use, or transmit users’ data. In terms of resource use, 13 platforms used third-party resources. Of the 19 platforms, 8 had crisis management mechanisms, of which 5 were passive, that is, they offered information such as helplines to be used when necessary. Three of them adopted active management procedures, that is, the platforms would proactively send out alerts and offer assistance to users in need.
Only 8 platforms provided research-based references. All platforms had explicit intervention methods, with health education (n=17) being the most commonly used, followed by mindfulness meditation (n=6), CBT (n=5), social support (n=5), and mood diary (n=3), respectively. A total of 13 platforms provided users with human support during the intervention, 6 of which were given by professionals (doctors, psychotherapists, etc), 5 by nonprofessionals, and 2 by a combination of both sorts. Ten platforms offered users feedback. There were 10 platforms that assessed users’ mood, but only 2 of them specified the method of assessment (Edinburgh Postnatal Depression Scale and Patient Health Questionnaire-9). Regarding platform engagement styles, 4 platforms used only image-text presentation mode; 14 platforms integrated audio, video, animation, and other forms; and only 1 platform employed artificial intelligence technology, albeit ineffectively.
Ten platforms offered data export options to users. Six platforms provided users in need with referral information, 4 of which offered offline visits and 2 web-based visits. However, only 2 platforms were capable of directly integrating user data into existing health care systems.
We investigated the association between quality indices and platform survival time and found that platforms lasting over 10 years had a significantly higher tendency to use external resources from third parties than those lasting less than 10 years (
With a cohort of websites and apps that have been reported in peer-reviewed literature, we examined for the first time how internet-based PND intervention platforms supported by published evidence operated in the marketplace. We found that nearly half of these platforms were no longer accessible, with an average estimated operating time of 2.1 years. The results are consistent with the current state of the mobile health market, where every 2.9 days a clinical app related to depression becomes unavailable in the App store [
Compared with the original publications, many platforms adjusted the core elements of the interventions in actual operations. Changes in intervention methods were the most common adjustments, and the main adding features were those that increased user engagement, such as mood diary [
A thorough quality assessment of the 19 platforms currently accessible revealed some common problems across each dimension of the APA’s model. Only about one-third of platforms had been updated in the past 6 months, which is consistent with Spadaro et al’s [
This study has some limitations. Although this paper did not include a systematic review, we employed a systematic search process to identify eligible platforms. We were unable to obtain detailed information on platforms between literature publication and our evaluation (eg, how many times the platform has adjusted its functions and when the adjustments occurred). But in contrast to previous studies that only took snapshots of market available apps, this study provided a longitudinal perspective by focusing on a cohort of internet-based platforms supported by peer-reviewed literature and tracking their real-world status. This study did not examine subjective indicators of quality, such as ease of use. However, we have conducted a comprehensive, objective quality assessment of all platforms based on the APA’s model to obtain a complete understanding of the quality of internet-based platforms.
Internet-based intervention platforms for PND are in a period of rapid development, but this study shows that many evidence-based platforms have poor sustainability in the marketplace. Despite a variety of functionality adjustments in actual operation, most platforms still had quality concerns that were not discussed in the research design phase. The ultimate goal of intervention platform development is to be implemented in the real world. It is clear that feasibility or effectiveness studies within the research context alone are insufficient for this goal. Further research from the perspective of implementation science would be required to evaluate the long-term effectiveness of the platform in a real-world setting, identify the barriers and facilitators of the implementation, and inform platform design in the future.
Search terms in each database.
Quality assessment checklist.
Platform database.
The associations between functional adjustments and survival time of internet-based perinatal depression intervention platforms (n=19).
Quality assessment of internet-based perinatal depression intervention platforms against the American Psychiatric Association’s model (n=19).
The associations between quality indices and survival time of internet-based perinatal depression intervention platforms (n=19).
App Evaluation Model of the American Psychiatric Association
cognitive behavioral therapy
perinatal depression
The data sets generated and analyzed during this study are available from the corresponding author on reasonable request.
This work was supported by the National Natural Science Foundation of China (grants 81973059 and 82273643), the China Medical Board (grant 20-368) and the Fundamental Research Funds for the Central Universities of Central South University (grant 2022ZZTS0894). We would like to thank Professor KK Cheng for his comments on our manuscript and the HER Team from XiangYa School of Public Health in Central South University including Yuhua Qin, Yingwen Wang, Shiyu He, Yiqing Cai, and Wenye Zou for assistance in literature screening and data extraction.
WG conceived the study. ZZ and JP contributed equally to the literature search, data extraction and assessment, and the development of the manuscript under the supervision of WG. LL contributed to literature screening, platform identification, and assessment. All authors contributed to the critical review of the manuscript and approved the final draft. The corresponding author attests that all listed authors meet authorship criteria and that no others meeting the criteria have been omitted. WG is the guarantor of the study.
None declared.