From October 2020 to October 2021, a total of 131 patients with a diagnosis of SSD (74 residents and 57 outpatients) and 115 unaffected controls were recruited at 10 different centers in Northern Italy, as part of the DiAPAson project [19,23-26]. We included patients with an SSD diagnosis based on the Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition (DSM-5) [27], who were 20-55 years old, able to speak and write in Italian, and were undergoing treatment at psychiatric RFs or as outpatients at a department of mental health. We excluded patients who were deemed unable to provide informed consent or who had significant cognitive deficits (ie, a Mini-Mental State Examination corrected score lower than 24), a recent diagnosis of substance use disorder according to DSM-5 criteria [27], a history of clinically significant head injury, or degenerative neurological disease.

Unaffected control individuals were recruited by public advertisements and snowball sampling procedures and were matched by gender and age group (ie, 20-24, 25-29, 30-34, 35-39, 40-44, 45-49, and 50-55 years) with the clinical sample. At each study center, treating clinicians invited participants under their care to participate in the study. Participants were provided with detailed information about the study and had the opportunity to ask questions. Some of the assessment questionnaires were completed by the treating clinician, and research assistants (RA) helped the patients complete self-report questionnaires. ESM monitoring was preceded by a briefing session in which the RA gave instructions about the ESM notifications and how to effectively respond. The monitoring was followed by a debriefing session in which the same RA collected information on study acceptability and feasibility. During the debriefing session, outpatients and unaffected control individuals received €25 (US $27.38) for travel expenses.

Participants were assessed with a questionnaire recording psychiatric history and current treatment, an assessment of collaboration skills, the Specific Levels of Functioning Scale (SLOF) [28], the Brief Psychiatric Rating Scale (BPRS) [29,30], the Brief Negative Symptom Scale (BNSS) [31,32], and the World Health Organization Disability Assessment Schedule 2.0 (WHODAS 2.0) [33,34]. Self-esteem was assessed with the question, “I think I am at least as good as the others” derived from the Italian version of the Rosenberg Self-Esteem Scale [35].

Daily usage time and mood were measured using a smartphone-based ESM (ie, the participants completed a brief questionnaire about their current activities and mood on a smartphone 8 times a day from 8 AM to 12 AM for 7 consecutive days. Notifications were semirandomized (ie, were randomly sent within 8 scheduled time slots) and a reminder notification was sent after 15 minutes if there was no response to the initial prompt. The participant was allowed a maximum of 30 minutes to answer the notification.

This study was conducted in accordance with American Psychological Association’s [36] ethical standards for the treatment of human experimental volunteers and each participant provided consent in compliance with the tenets of the Declaration of Helsinki [37]. This study was approved by the ethical committee (EC) of the local institutions.

The study has been approved by the ECs of the 3 main participating centers: EC of IRCCS (Istituto di Ricovero e Cura a Carattere Scientifico) Istituto Centro San Giovanni di Dio Fatebenefratelli (July 31, 2019, number 211/2019), EC of Area Vasta Emilia Nord (September 25, 2019, number 0025975/19), and EC of Pavia (September 2, 2019, number 20190075685) and by the ECs of all participating sites. Data were deidentified in order to prevent the participants’ personal identities from being revealed.

Kolmogorov-Smirnov tests were performed as preliminary analyses to ensure no violation of the assumption of normality. Adherence with mobile ESM was measured by calculating the reaction rates to notifications (expressed as the percentage of answers), initially over a total of 56 notifications for 7-day monitoring, and then considering separate time slots (8 AM to 12 PM, 12 PM to 4 PM, 4 PM to 8 PM, and 8 PM to 12 AM) and days of the study (1-2, 3-5, and 6-7). No participants were excluded for not having reached a minimum of answered notifications in this study.

The descriptive statistics of sociodemographic and clinical variables consist of frequency tables (for categorical variables) and means and SDs (for continuous variables). To assess differences between groups, we used chi-square tests (or the Fisher exact test, which is more appropriate when smaller numbers are considered) and ANOVA (or the nonparametric Kruskal-Wallis test for variables not normally distributed). Bonferroni correction was applied for post hoc group comparisons.

For aim 1, we computed the percentage of adherence in the 3 different groups and for different daily hours and days, and we checked for any significant differences in rows (between groups) using a Kruskal-Wallis test, and differences in columns (within groups) using the Friedman test. For aim 2, we used binomial logistic regression models (separately for residents and outpatients), considering an adherence rate of ≥70% as the dependent variable to indicate a predictor of adherence. This cut-off was established in accordance with previous ESM studies that considered at least about 30% of answered notifications as acceptable [8,38].

All statistical analyses were performed using SPSS (version 27.0; IBM Corp). All statistical tests were 2-tailed; a P value of ≤.05 was considered significant.

The 3 groups were similar in their age and gender distribution (Table 1). Patients had lower levels of education and employment than control individuals, and the residents had more severe conditions: a longer illness duration (mean 19.3, SD 10.0 years), higher rates of psychiatric comorbidities (n=31, 41.9%), higher numbers of both antipsychotic (mean 1.7, SD 0.7) and nonantipsychotic medications (mean 1.6, SD 1.2), greater symptom severity (mean total BPRS score 47.0, SD 13.2), more severe negative symptoms (mean BNSS score 23.5, SD 14.6), and lower levels of social function (mean SLOF score 175.8, SD 20.7).

Between-group analyses (Table 2 and Figure 1) revealed that the number of participants to be excluded due to <30% of ESM notifications being answered was significantly higher among residents (n=17, 23%) than among outpatients (n=10, 17.5%) and unaffected control individuals (n=3, 2.6%). Moreover, residents had the lowest adherence rate among all 3 groups on weekdays (ie, Monday to Friday; mean 49.4%, SD 27.7%), in the 8 PM to 12 AM time slot (mean 26.3%, SD 25.7%), and in the first 2 (mean 53.5%, SD 28.8%) and the last 2 (mean 43.5%, SD 32.5%) days of monitoring. However, residents and outpatients showed similar adherence rates when considering the total adherence rate on weekends, in the 8 AM to 8 PM time slots, and in the central days of monitoring (ie, days 3, 4, and 5). Control group participants showed the highest adherence rate independent of daily time, day of monitoring, and day of the week.

Within-groups analyses (Table 2 and Figure 1) showed no differences in adherence rates among the 3 groups between weekends and weekdays. Again, not surprisingly, the 8 PM to 12 AM time slot had the lowest adherence rate among the 3 groups. Similarly, adherence rates were the lowest on the last 2 days between residents and control participants.

Logistic regression analyses (Table 3) revealed that a higher adherence rate (ie, ≥70%) was significantly predicted by higher collaboration skills (odds ratio [OR] 2.95, 95% CI 1.02-8.54; P=.046) among residents. Among outpatients, higher self-esteem (OR 3.39, 95% CI 1.13-10.14; P=.03) and lower severity of positive symptoms (OR 0.83, 95% CI 0.70-0.98; P=.04) predicted a higher adherence rate (ie, ≥70%).

In this study, we aimed at assessing the adherence rate to the ESM among patients with SSD (divided into resident and outpatient groups) and unaffected control individuals across daily hours and days of monitoring, and the factors associated with adherence in different patient groups. In the next sections, we discuss the main findings of our study and their implications for planning of real-time research with clinical populations with SSD.

This study has a number of limitations. First, participants had different levels of experience in the use of smartphones, and residents were often helped by health care workers, while outpatients were not; due to the high complexity of the study design and the number of RFs involved in the study, we were unable to assess the adherence rate in relation to the frequency and magnitude of support received by the staff. Second, residents did not receive financial reimbursement, unlike the outpatients and control participants, which may have affected their adherence to the procedure. Third, our findings may not be generalized to older individuals and patients with severe cognitive, motor, or visual deficits or those with substance use disorders. Fourth, our study was of comparatively short duration. Fifth, the study design was not sensitive to bedtime, afternoon naps, and specific rules on smartphone use and bedtime in the RFs. Sixth, we could not evaluate the validity of the answers given to notifications. However, the study has a number of strengths, including the very comprehensive clinical assessment, the relatively large sample size, and multicenter and controlled design.

This study adds to existing knowledge on the use of the ESM to measure fine-grained activities of patients with SSD in both residential and outpatient settings, and how the adherence of patients with SSD compares to that of unaffected control individuals. Such studies are relevant as they highlight the strengths and weaknesses of ESM’s study designs and allow for the development of recommendations and practical guidelines for implementing future studies or treatment plans that would optimize adherence rates and be acceptable for this specific population. Higher adherence to ESM assessments will, in turn, allow for the collection of reliable and accurate information about the daily experience of such clinical populations and the development of more specific and valid treatment plans for covering their needs.

Based on our results, the recommendations for both researchers and clinicians using the ESM include (1) setting ESM monitoring to waking hours, between 8 AM and 8 PM, and being sensitive to RFs’ rules and restraints, bedtime schedules, and habits of such populations; (2) beginning with several days of familiarization for patient groups and providing more training sessions prior to the start of the monitoring, which also include health care staff and caregivers; (3) limiting the number of days of monitoring; (4) selecting patients with more collaborative skills and excluding those with marked positive symptoms; and (5) as part of the preparation, attempting to improve participants’ self-confidence in the use of the technologies, guaranteeing adequate training and support before the start of the monitoring process. Finally, a major unanswered core issue in ESM research with this population is whether patients with SSD can provide reliable, accurate, and valid self-reports. Future studies should develop and apply accurate methods for assessing the validity of ESM answers in this population.