Participants were recruited between May and November 2015 from a public health center in Seoul, South Korea. Eligibility for study participation were as follows: adults (18 years of age or older) who voluntarily visited a center for nonorthopedic or neuromuscular causes, being capable of undertaking daily PA, having no plan for hospitalization during the study, and having access to their own mobile phones.

A 7-day study was conducted during the May and November sessions of 2015. Participants were given the option to participate in one or both sessions. On day 0, participants’ body composition and handgrip strength were measured. Participants were asked to wear the ActiGraph GT3X+ (ActiGraph LLC) over the right waist within an anterior axillary line for 7 consecutive days (days 1-6). They were instructed to always wear the device while awake, except for water activities such as swimming and showering. Out of 3 self-report measures of PA (ie, EMA, GPAQ, and BAR) were completed according to the following timeline: EMA was completed on 1 weekday and 1 weekend day; GPAQ was completed on the first day before participants were provided with the waist sensor; BAR was completed daily between days 1 and 6 (Figure 1).

Days that participants provided data through each instrument are colored in grey. Accelerometer data were provided from days 1 to 6. Participants were asked to wear a waist sensor every day, except during swimming and showering. EMA was performed on 1 weekday and 1 weekend day. GPAQ was completed on day 0, and BAR data were collected between days 1 and 6. Participants who participated in at least one session are denoted as overall participants.

The data cleaning process is illustrated in Figure 3. Of the 4408 EMA records (n=230), 348 records (n=2) were eliminated as participants provided less than 6 reports per day (ie, 6/8, less than 75%). In addition, 727 records were removed due to missing or uncategorized values, resulting in 3333 records (n=228). On the GPAQ, of the 333 records (n=235), 58 records were excluded based on the GPAQ analysis guideline [16], leaving 275 records (n=198).

To address repeated measures within participants, the multilevel analysis method was used with (1) step counts and total MVPA minutes from the accelerometer data and (2) activity-specific domains from the BAR as outcome measures. Multilevel modeling is suitable for addressing the hierarchical data structure inherent in the mixed-design study, which includes both within- and between-participant measures [31].

On the analysis with accelerometer data as outcome measures, both the BAR and EMA data were structured into 2 levels. Daily observations formed the lowest level of the hierarchy (level 1), nested within individuals (level 2). Data driven from 3 self-report instruments (EMA, GPAQ, and BAR), each with 6 PA domains (total MVPA; total MVPA excluding occupational; transport; leisure time exercise; occupational PA; and SB), were separately used to assess the association with objectively obtained accelerometer data (steps counts and total MVPA minutes). Both EMA and the BAR data were analyzed in 2 variations based on time intervals: daily and overall mean scores per participant. Filtered EMA, GPAQ, and BAR data were separately integrated with accelerometer data based on matching dates. The final analysis included 3 datasets each containing 2593 (EMA-accelerometer: n=218, 89 repeated), 270 (GPAQ-accelerometer: n=196, 77 repeated), and 2004 (BAR-accelerometer: n=234, 94 repeated) records.

Compared to the EMA method, where participants provide data at 2-hour intervals between 8 AM and 10 PM, the BAR captures PA continuously throughout the day, with reports recorded every 15 minutes. We further performed multilevel modeling using activity-specific domains from the BAR data as outcome measures to evaluate the validity of EMA against the more fine-grained reporting intervals of the BAR. Unlike accelerometer data, which measures PA only through step counts or total MVPA minutes, the BAR offers an additional advantage by allowing for domain-specific validation. For example, SB reported through EMA can be compared with SB reported through BAR. Similar time-slot matching methods were applied to integrate the BAR data with both EMA and GPAQ datasets. The final dataset sizes were 539 (EMA-BAR: n=221, with 73 repeated measures) and 198 (GPAQ-BAR: n=198, with 74 repeated measures).

Standardized coefficients were calculated for the comparisons between different instruments. Participants’ age and gender were included as covariates to account for differences in outcomes attributed to demographics. A sensitivity analysis on EMA data indicated that there is the difference in the validity of EMA data based on reported days, with reports from weekdays showing better match with outcome measures compared with those from weekend days. Therefore, for the analysis of EMA data, the type of the day (week vs weekend day) was further used as a covariate alongside participants’ age and gender. Total and repeated-only participant (those who enrolled in both May and November sessions of the study) groups were separately examined.

This study adhered to ethical guidelines for research involving human participants and received approval from the institutional review board at Seoul National University Hospital, Seoul, Korea 1812-129-997). All methods were carried out in accordance with the guidelines proposed in the Declarations of Helsinki. A total of 241 participants provided informed consent.

We conducted a comprehensive comparison of different assessment instruments (EMA, GPAQ, BAR, and accelerometer) to validate the EMA methodology for assessing PA. Compared to international surveillance methods for assessing PA, the smartphone-delivered EMA method demonstrated strengths across various PA metrics. This was evident both through analysis using objectively measured accelerometer data as reference measures and through analysis using well-validated, log-like recording systems as reference measures.

We observed consistent patterns in participants’ daily activity reports and their associations with accelerometer data collected via waist sensors. Generally, participants tended to overestimate total MVPA minutes spent compared to those tracked by wearable sensors. This aligns with previous research findings that reported overestimation of activity levels through self-reported measures over sensor-derived outcomes [32-34].

Comparisons among the 3 self-report instruments for modelling accelerometer data suggested that assessments with smaller time intervals yield stronger outcomes. The superior performance of the BAR and EMA over the GPAQ demonstrates the benefits of higher reporting frequencies and shorter recall periods. The results also indicate that daily-based measures on both BAR and EMA are more effective than the overall mean per participant, underscoring the appropriateness of multilevel modeling approaches. These findings have significant implications for research design, suggesting that instruments using shorter time intervals and daily reporting protocols are more suitable for capturing nuanced variations in PA patterns.

The practical advantages of EMA are particularly noteworthy in this context. In the current study, EMA allows up to 7 recordings per day, each covering the past 30 minutes, offering a compromise between accuracy and feasibility compared with the more demanding BAR, which requires participants to record data every 15 minutes. This balance makes EMA especially valuable in large-scale health research or clinical applications, where participant burden is a critical consideration. For instance, EMA’s ability to deliver reliable data with reduced respondent demands could improve adherence rates in long-term studies, such as those monitoring PA in individuals with chronic conditions like diabetes or cardiovascular disease. In clinical practice, this approach could help health care providers better tailor interventions by obtaining more precise and contextually relevant activity data.

Moreover, the flexibility of EMA offers opportunities to enhance the contextual understanding of PA. Unlike the BAR and GPAQ, EMA can capture additional contextual information, such as location, social settings, or emotional states during PA. This capability has practical applications in designing personalized behavioral interventions. For example, EMA prompts could be used to deliver real-time feedback or motivational messages when a participant is detected to be sedentary or transitioning to PA, leveraging data from wearable devices synced with smartphones. This adaptability positions EMA as a promising tool for both preventive health strategies and rehabilitation programs aimed at improving PA behaviors.

This study demonstrated a relatively high data completion rate, with most participants providing full sets of data across the different measuring instruments (Figure 3). However, future studies should focus on optimizing the temporal intervals of PA measurement to identify a balance between measurement accuracy and participant burden [35]. For example, investigating whether hourly prompts yield comparable validity to those at 30-minute intervals could inform the design of more efficient EMA protocols. Such refinements could enhance EMA’s utility across varied populations and contexts, including both high-demand clinical environments and large-scale epidemiological studies.

The recruitment for this study was conducted at a single public health center, where participants volunteered to take part in the study. While using this recruitment strategy was necessary to validate the Physical Activity Questionnaire [36] used in previous studies, a significant proportion of the participants were married females aged 50 years or older, indicating a potential skew in the overall participant demographic. This suggests that the tendencies observed in participants’ engagement in different types of activity, as well as the relatively low performance of occupational activity in modeling outcomes measures compared with other PA domains, may not be fully generalizable [32,37]. In addition, due to technological limitations, we did not prompt participants to report their PA through EMA at random intervals throughout the day. As a result, there is a possibility that the data may be skewed toward periods of more representative PA. Future studies should consider adopting methodologies that allow for more flexible timing of EMA prompts to address this limitation.

Overall, our comprehensive comparison of different assessment instruments underscores smartphone-delivered EMA as a valid and consistent method for assessing PA across various domains. While there is growing interest in using accelerometers for passive PA assessment, challenges associated with sensor-based measures highlight the continued need for self-report instruments in PA measurement. Smartphone-delivered EMA offers an additional advantage over traditional self-report tools by enabling real-time, adaptive monitoring of PA, while also allowing for more efficient data storage, retrieval, and use. In summary, the robustness of EMA in assessing various PA domains, coupled with its potential to collect contextual information, positions EMA as a valid and effective tool for the assessment of PA.