This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.
Diet and physical activity (PA) have a major impact on physical and mental health. However, there is a lack of effective strategies for sustaining these health-protective behaviors. A shift to a microtemporal, within-person approach is needed to capture dynamic processes underlying eating behavior and PA, as they change rapidly across minutes or hours and differ among individuals. However, a tool that captures these microtemporal, within-person processes in daily life is currently not present.
The APPetite-mobile-app is developed for the ecological momentary assessment of microtemporal, within-person processes of complex dietary intake, objectively recorded PA, and related factors. This study aims to evaluate the feasibility and usability of the APPetite-mobile-app and the validity of the incorporated APPetite-food record.
The APPetite-mobile-app captures dietary intake event-contingently through a food record, captures PA continuously through accelerometers, and captures related factors (eg, stress) signal-contingently through 8 prompts per day. Empirical data on feasibility (n=157), usability (n=84), and validity (n=44) were collected within the Eat2beNICE-APPetite-study. Feasibility and usability were examined in healthy participants and psychiatric patients. The relative validity of the APPetite-food record was assessed with a subgroup of healthy participants by using a counterbalanced crossover design. The reference method was a 24-hour recall. In addition, the energy intake was compared with the total energy expenditure estimated from accelerometry.
Good feasibility, with compliance rates above 80% for prompts and the accelerometer, as well as reasonable average response and recording durations (prompt: 2.04 min; food record per day: 17.66 min) and latencies (prompts: 3.16 min; food record: 58.35 min) were found. Usability was rated as moderate, with a score of 61.9 of 100 on the System Usability Scale. The evaluation of validity identified large differences in energy and macronutrient intake between the two methods at the group and individual levels. The APPetite-food record captured higher dietary intakes, indicating a lower level of underreporting, compared with the 24-hour recall. Energy intake was assessed fairly accurately by the APPetite-food record at the group level on 2 of 3 days when compared with total energy expenditure. The comparison with mean total energy expenditure (2417.8 kcal, SD 410) showed that the 24-hour recall (1909.2 kcal, SD 478.8) underestimated habitual energy intake to a larger degree than the APPetite-food record (2146.4 kcal, SD 574.5).
The APPetite-mobile-app is a promising tool for capturing microtemporal, within-person processes of diet, PA, and related factors in real time or near real time and is, to the best of our knowledge, the first of its kind. First evidence supports the good feasibility and moderate usability of the APPetite-mobile-app and the validity of the APPetite-food record. Future findings in this context will build the foundation for the development of personalized lifestyle modification interventions, such as just-in-time adaptive interventions.
Diet is a key contributor to both physical and mental health. Elevated BMI is a major risk factor for noncommunicable diseases, such as cardiovascular diseases [
Although factors and processes underlying eating behavior have been studied for many years [
It is not only diet that has a large impact on both physical and mental health. Physical activity (PA) represents another impactful, repeated-occurrence health behavior [
The repeated or continuous assessment of experiences, behaviors, or physiological processes in real life through smartphones or wearable devices is a highly promising approach for studying microtemporal, within-person processes [
Even though EMA studies do not allow causal conclusions, they offer insight into three important aspects of microtemporal, within-person processes: (1) temporal specificity (eg, Does diet influence mood to a greater extent than mood influences diet?), (2) situational specificity (eg, Is unhealthy eating more likely when being alone or with others?), and (3) person specificity (eg, Is stress more predictive for engaging in eating for some individuals compared with others?) [
Diet is a highly complex phenomenon that makes its assessment difficult. However, to avoid typical reporting biases that are present in traditional dietary assessment methods (eg, food frequency questionnaires), the number of studies using EMA to capture self-reported dietary intake or aspects of it in real time or near real time instead of retrospectively has rapidly grown in the last decade [
Despite the importance of taking possible interactions into account, most EMA studies focus on either the assessment of diet or PA. One study identified the need for an EMA tool to capture
In conclusion, there is a strong need for an EMA tool that allows capturing complex dietary intake, objectively measured PA, and a broad range of associated factors simultaneously in daily life to study microtemporal, within-person processes underlying these health-protective behaviors.
As no EMA tool allows the study of microtemporal, within-person processes of complex dietary intake, objectively measured PA, and related factors, we developed an EMA tool for the simultaneous assessment of these complex health behaviors and related factors in daily life: the APPetite-mobile-app (this term also covers the assessment of PA, although it is not performed by the APPetite-mobile-app itself but by an accelerometer).
The suitability of novel EMA tools for use in daily life should be evaluated. Therefore, feasibility, usability, and validity were examined empirically in this study. The following questions will be addressed: Is the APPetite-mobile-app a feasible and usable tool for the combined assessment of complex dietary intake, PA, and associated factors in daily life and a valid tool for the assessment of complex dietary intake in real time or near real time?
The APPetite-mobile-app was developed and run through movisensXS (version 1.4.7, movisens GmbH), a web-based platform for the development of EMA tools. It supports a broad range of sampling schemes, item formats, and multimedia records, allowing flexible and tailored study configurations. The APPetite-mobile-app is run through the movisensXS app (available for Android devices). If the mobile device has access to mobile data during the EMA assessment, participants’ entries will be uploaded instantly to the platform. In this way, compliance can be monitored throughout the EMA assessment, and a chat function allows direct messaging with participants. All participants received a study smartphone (Motorola Moto G 3rd generation), with access to mobile data. The movisensXS app was previously tested on this particular mobile device, ensuring its smooth functioning and increasing the standardization of the mobile-based assessment.
The APPetite-mobile-app uses event-, signal-, and time-contingent as well as continuous sampling (
Sampling strategy of the APPetite-mobile-app.
Food intake was recorded event-contingently through a food record. Participants were asked to enter foods and drinks as soon as possible after consuming them. Accordingly, participants are able to initiate the APPetite-food record at any time and capture their food intake in real time. This was chosen to minimize memory effects and record the exact time of food intake. In addition, this allows capturing food intake even during the night, when signal-contingent prompts are inappropriate. At 9 PM, a time-contingent prompt asks if all consumed foods and drinks of the day have been recorded, ensuring that no foods and drinks consumed on this day are missed.
The prompts are initiated signal-contingent at eight semirandom times per day between 8 AM and 10 PM. The minimum time between 2 prompts is 1 hour. Therefore, participants cannot predict the exact time of the next prompt, and the assessed situation is a better reflection of the participant’s real life. Participants were instructed to respond immediately to the prompt. However, if participants are unable to reply instantly, it is possible to postpone the prompt for 5, 10, 15, 20, or 25 minutes to avoid missing data and reduce the participants’ burden. If no reaction is registered, the prompt is deactivated and cannot be reactivated.
Continuous sampling through an accelerometer is used for the assessment of PA.
The APPetite-food record comprises a 6-step process: (1) selection of meal type, (2) entry of time of intake, (3) selection of consumed foods and drinks, (4) specification of consumed amounts, (5) presentation of reminder for commonly forgotten foods, and (6) indication of predominant reason for eating or drinking (
Screenshots of the 6-step process of the APPetite-food record.
Each prompt assesses the context, affect, stress, impulsivity, and food availability either since the last prompt or immediately before the prompt. In addition, the first prompt of a day captures sleep quality and quantity as well as stress expectancy. All prompt measures and items are described in
Move 3 sensors from movisens were used to objectively record PA. The accelerometer was worn on the nondominant wrist. Participants were asked to wear it at any time (also when sleeping) and only take it off when showering or performing water activities. The Move 3 sensor captures raw data on 3D acceleration, barometric air pressure, and temperature. Secondary parameters such as activity class, body position, steps, metabolic equivalents, and PA metrics can be extracted using the DataAnalyzer (movisens GmbH).
The feasibility of the APPetite-mobile-app was separately assessed for the EMA prompts, the APPetite-food record, and the accelerometer. The feasibility of the prompts is determined by prompt delivery, total number of answered prompts across all subjects, number of answered prompts per participant, compliance (percentage of complete prompts within received prompts), response latency (time from first prompt signal to answering), and the time needed to complete a single prompt. The food record’s feasibility was evaluated based on the number of recorded eating and drinking events per day, reporting latency (time between the meal and meal recording), and the time needed to record food intake per day. The amount of time wearing the accelerometer and compliance (percentage wearing the accelerometer within the 7-day assessment period) are measures of feasibility of the accelerometer.
The data were collected within the ongoing APPetite study. The APPetite study is part of the European Union Horizon2020 project Eat2beNICE and recruits participants from three existing studies: LORA (Longitudinal Resilience Assessment) study [
From November 2018 to March 2020, 161 participants were included in the APPetite study (140 LORA, 7 PROUD, and 14 BipoLife-A1). After the first in-person session, 3 LORA participants dropped out. Of these, 2 realized that they were unable to respond to prompts. The third person was mistakenly given a smartphone that was not coupled with the EMA protocol. Another person dropped out after the additional in-person session of the validation study for private reasons. Hence, EMA data of 157 participants are available for the evaluation of feasibility (see demographics in
Demographics of the total sample and the 3 cohorts (only individuals who completed the ecological momentary assessment were included; N=157).
Variables | Total (N=157) | LORAa (n=136) | PROUDb (n=7) | BipoLife-A1 (n=14) | |
|
|||||
|
Female | 100 (63.7) | 94 (69.1) | 1 (14.3) | 5 (35.7) |
|
Male | 57 (36.3) | 42 (30.9) | 6 (85.7) | 9 (64.3) |
Age (years), mean (SD) | 28.04 (7.22) | 28.08 (7.55) | 26.43 (2.51) | 28.5 (5.39) | |
BMI, mean (SD) | 24.71 (4.81) | 24.26 (3.87) | 24.98 (6.11) | 28.9 (9.13) |
aLORA: Longitudinal Resilience Assessment.
bPROUD: Prevention of Comorbid Depression and Obesity in Attention-Deficit/Hyperactivity Disorder.
The APPetite study consists of 2 in-person sessions, the EMA assessment, and a follow-up session from home. In the first in-person session, participants received detailed training on how to use the APPetite-mobile-app and the accelerometer. Participants received a smartphone with the APPetite-mobile-app and an accelerometer, including a wristband. Participants used the APPetite-mobile-app for 3 consecutive days (2 weekdays and 1 weekend day, not including the day of the first in-person session) and wore the accelerometer for 7 consecutive days (overlapping the 3 days of the APPetite-mobile-app assessment, not including the day of the first in-person session). During the 3 days of the app-based assessment, prompt compliance was tracked. If compliance fell below the threshold of 80%, a motivational message was sent to the participant. Participants who completed at least 80% (19/24) of the prompts were included in a raffle to win a €100 (US $121.74) voucher and a cooking class. Before the second in-person session, EMA data were checked, and questions regarding implausible prompt entries (eg, 8 AM as bedtime) and food records (eg, missing meals) were collected. These questions were reviewed in the second in-person session to resolve any uncertainties. Usability of the APPetite-mobile-app, reactivity, and representativity of the EMA assessment were assessed via questionnaires in the second in-person session.
Participants received €40 (US $48.7) after the second in-person session and €10 (US $12.17) after completing the follow-up. In addition, individual feedback on diet and PA was provided after the follow-up, which consisted of a web-based 24-hour recall from home.
Descriptive statistics were used to assess feasibility measures. We investigated whether compliance differed among the 3 cohorts, the 3 days, and between male and female participants. As compliance is not normally distributed, this is done using the following nonparametric tests: Kruskal-Wallis rank sum test, Friedman test, and Wilcoxon rank sum test. In addition, the Spearman rank correlation coefficient was calculated to investigate the association between compliance and age. The α level was set to .05. The analyses were performed using R 3.6.1 (R Core Team) with RStudio (RStudio, Public-benefit corporation).
Usability is assessed using the System Usability Scale (SUS; [
Data were collected within the APPetite study. However, SUS was subsequently added to this study (August 2019). Therefore, it is available only for a subsample of 84 participants (55 women and 29 men; 67 from LORA, 6 from PROUD, and 11 from BipoLife-A1). The mean age of the sample was 29.26 (SD 7.41) years, and the mean BMI was 24.82 (SD 5.26) kg/m2.
The SUS was completed during the second in-person session.
Total usability scores were calculated according to the study by Brooke [
The relative validity was assessed using a counterbalanced crossover design. Myfood24 Germany (Measure Your Food on One Day), a 24-hour recall, was chosen as the reference method. Myfood24 is a web-based, self-administered 24-hour dietary recall tool (refer to Koch et al [
Furthermore, energy intake is compared with total energy expenditure (TEE) based on the assumption that energy intake equals TEE in weight-stable individuals [
A total of 50 healthy participants from the LORA study (group 1: n=26; group 2: n=24) volunteered for the validation study. However, 6 participants from group 1 had to be excluded, as they did not complete all relevant parts within the predefined time schedule. Therefore, the evaluation of validity was based on data from 44 participants (33 women and 11 men)—20 from group 1 and 24 from group 2. This sample had a mean age of 28.64 (SD 8.13) years and a mean BMI of 23.8 (SD 3.62) kg/m2. The groups did not significantly differ in terms of sex (group 1: 15 women and 5 men; group 2: 18 women and 6 men;
Participants from the LORA cohort who agreed to participate in the APPetite study were asked whether they wanted to also participate in the validation study: recording their food intake through a 24-hour recall on 3 additional days. Of the participants who agreed, 26 were assigned to group 1 and 24 to group 2, following a counterbalanced crossover design. Hence, participants in group 1 completed three 24-hour recalls exactly a week before the APPetite-food record was used. In group 2, participants completed three 24-hour recalls exactly a week after the APPetite-food record was used. The same weekdays, the week before or after, were assessed. Both groups received the same training to familiarize themselves with the 24-hour recall and the APPetite-food record. Participants received €30 (US $36.52) to participate in the validation study.
Habitual energy and macronutrient intake assessed through the APPetite-food record was compared with habitual dietary intake assessed through the 24-hour recall.
Habitual energy and macronutrient intake from the two methods were compared at the group level using two-tailed paired
A total of 98.28% (3703/3768) of all scheduled prompts were delivered. The failure of prompt delivery was either due to technical problems or because the smartphone was switched off. Overall, 80.31% (3026/3768) of the prompts were answered completely. In total, 0.9% (34/3768) of prompts were registered as incomplete as a result of technical problems or extensive breaks during prompt completion. A total of 1.81% (68/3768) of prompts were dismissed, and 15.26% (575/3768) of prompts were ignored. The relatively large proportion of ignored prompts was, to some extent, a result of participants unintentionally leaving their smartphone at home or in another room. Furthermore, a number of participants reported that they had missed the first prompt or prompts of the day, as they were still sleeping.
Overall mean compliance (percentage of complete prompts within received prompts) was 81.73% (SD 21.65%). The compliance rate of 67.5% (106/157) of participants was above 80% (LORA: 94/136, 69.1%; PROUD: 4/7, 57%; and BipoLife-A1: 8/14, 57%). The mean compliance rate was 81.56% (SD 25.98%) on the first day, 83.28% (SD 23.55%) on the second day, and 79.97% (SD 25.8%) on the third day. The Friedman test showed no significant difference in compliance among the 3 days (
Compliance was highest in the LORA cohort and lowest in the PROUD cohort (see cohort means and SDs in
Female participants had, on average, a compliance of 83.95% (SD 19.02%). The mean compliance for male participants was 77.83% (SD 25.34%). The Wilcoxon rank sum test found no gender difference in compliance (
Participants responded to prompts after a mean of 189.32 seconds (SD 388.65). Responding to 70.54% (2157/3058) of all prompts was started within the first 60 seconds after the first prompt signal. The mean time needed to complete a single prompt was 122.63 seconds (SD 70.01). The prompt response latency and response duration for each of the 3 cohorts are shown in
Mean number and percentage of complete, incomplete, dismissed, and ignored prompts within received prompts for the total sample and each cohort.
Samples | Complete | Incomplete | Dismissed | Ignored | |||||
|
|||||||||
|
Values | 19.27 (5.32) | 0.22 (0.44) | 0.43 (1.07) | 3.66 (4.79) | ||||
|
Percentage | 81.73 (21.65) | 0.93 (1.89) | 1.82 (4.48) | 15.52 (20.08) | ||||
|
|||||||||
|
Values | 19.43 (5.04) | 0.23 (0.46) | 0.4 (1.05) | 3.5 (4.46) | ||||
|
Percentage | 82.48 (20.41) | 0.98 (1.95) | 1.71 (4.4) | 14.83 (18.7) | ||||
|
|||||||||
|
Values | 17.14 (7.73) | 0.43 (0.54) | 0.86 (1.46) | 5.57 (6.95) | ||||
|
Percentage | 71.43 (32.22) | 1.79 (2.23) | 3.57 (6.09) | 23.21 (28.95) | ||||
|
|||||||||
|
Values | 18.86 (6.76) | 0 (0) | 0.5 (1.09) | 4.29 (6.63) | ||||
|
Percentage | 79.59 (27.46) | 0 (0) | 2.08 (4.55) | 18.33 (27.8) |
aLORA: Longitudinal Resilience Assessment.
bPROUD: Prevention of Comorbid Depression and Obesity in Attention-Deficit/Hyperactivity Disorder.
Response latency and duration for a single prompt, reporting latency of the food record, and recording duration for the food record per day for the total sample and each cohort.
Variables | Total | LORAa | PROUDb | BipoLife-A1 | |
|
|||||
|
Latency | 189.32 (388.65) | 179.38 (375.82) | 242.24 (447.85) | 265.24 (469.8) |
|
Duration | 122.63 (70.01) | 119.13 (64.21) | 175.12 (144.31) | 134.01 (64.07) |
|
|||||
|
Latency | 58.35 (127.52) | 50.82 (115.8) | 147.02 (197.85) | 116.99 (190.16) |
|
Duration | 17.66 (8.66) | 17.8 (8.57) | 22.17 (10.86) | 14.26 (7.56) |
aLORA: Longitudinal Resilience Assessment.
bPROUD: Prevention of Comorbid Depression and Obesity in Attention-Deficit/Hyperactivity Disorder.
Dietary data of 8.8% (12/136) LORA, 14% (1/7) PROUD, and 14% (2/14) BipoLife-A1 participants had to be excluded, as the number of recorded meals or entered foods was evidently too low or entries were incomplete or implausible. In addition, 3 LORA participants had no food entry on 1 day. However, the remaining 2 days were recorded sufficiently well to be included. Included participants (n=142) recorded a total of 2969 eating and drinking events. In total, 3.03% (90/2969) entries were registered as incomplete, mainly due to technical problems. Participants entered on average 7.02 (SD 3.33) eating and drinking events per day (first day: mean 7.49, SD 3.14; range 2-17; no data available for 1 participant; second day: mean 7.08, SD 3.43; range 2-22; third day: mean 6.49, SD 3.37; range 2-17; no data available for 2 participants).
The mean latency from food intake to food recording was 58.35 (SD 127.52) minutes. Latency increased over the course of the 3 days (first day: mean 53.51, SD 72.01 min; second day: mean 69.5, SD 88.1 min; third day: 90.81, SD 116.12 min). The mean time to complete the food record of one day was 17.66 (SD 8.66) minutes. On the first day, participants took 21.01 (SD 9.68) minutes, on the second day they took 17.22 (SD 7.76) minutes, and on the third day they took 14.67 (SD 7.16) minutes. The cohort-specific food record latencies and recording durations are presented in
The accelerometer records of 2 participants stopped during the second day. It is unknown if this was due to technical problems or because participants connected the sensor to a computer that instantly stopped the recording. In total, 11 participants did not wear the sensor on at least one day or stopped wearing it before the end of the 7-day assessment period. On average, participants (N=157 including the abovementioned) wore the sensor for 6 days 3 hours and 57 minutes (mean 8876.96, SD 1815.36 min; range 771-10,403). Hence, the mean compliance was 88.07% (SD 18.01%).
The SUS total score was 61.9 (SD 17.79; range 17.5-97.5) out of 100. The SUS score of the LORA cohort (n=67) was 61.23 (SD 16.8; range 17.5-95). The lowest usability with an SUS score of 60 (SD 24.08; range 32.5-92.5) was rated by the PROUD cohort (n=6). The highest SUS score was found for the Bipolife-A1 cohort (n=11), with a score of 67.05 (SD 20.97; range 22.5-97.5). However, the 3 cohorts did not differ in the ratings of usability according to a one-way analysis of variance (
Female participants (mean 62.82, SD 17.36) scored usability on average marginally higher than male participants (mean 60.17, SD 18.77; t82=0.65 [two-tailed];
Habitual intake of energy, protein, fat, carbohydrates, sugar, and fiber assessed through the APPetite-food record and the 24-hour recall are shown in
With regard to possible order effects, both groups (APPetite-food record first and 24-hour recall first) showed higher energy intake assessed through the APPetite-food record (group 1: 8494 kJ/2029 kcal; group 2: 9327 kJ/2228 kcal) compared with the 24-hour recall (group 1: 7881.23 kJ/1882 kcal; group 2: 8086.01 kJ/1931 kcal).
Agreement between the two methods at the individual level was investigated through Bland-Altman plots for energy and macronutrient intake. Mean energy difference between the APPetite-food record and the 24-hour recall was 994.18 kJ (95% CI 370.8-1617.6). A normal distribution of the difference was observed.
Bland-Altman analyses for protein, fat, carbohydrate, sugar, and fiber intake can be found in
Mean habitual intake of energy and macronutrients from the APPetite-food record and the 24-hour recall; mean difference between the two methods; paired
Dietary intake per day | APPetite-food record, mean (SD) | 24-hour recall, mean (SD) | Mean difference (SD) | ||
Energy (kJ/day) | 8987.11 (2404.65) | 7992.93 (2002.61) | 994.18 (2050.52) | 3.21 (43) | .003a |
Energy (kcal/day) | 2146.42 (574.5) | 1909.16 (478.8) | 237.26 (489.94) | 3.21 (43) | .003 |
Protein (g/day) | 80.77 (27.6) | 69.68 (24.72) | 11.09 (22.63) | N/Ab | .004a |
Total fat (g/day) | 92.25 (32.54) | 76.95 (26.54) | 15.3 (29.41) | N/A | .002a |
Carbohydrate (g/day) | 228.81 (63.12) | 212.99 (52.65) | 15.82 (64.03) | 1.64 (43) | .11 |
Sugars (g/day) | 81.21 (32.6) | 76.97 (29.76) | 4.24 (31.42) | N/A | .40 |
Fiber (g/day) | 25.86 (9.22) | 23.3 (8.5) | 2.56 (7.43) | N/A | .04c |
a
bN/A: not applicable to Wilcoxon signed-rank test.
c
Bland-Altman plot assessing agreement between habitual energy intake in kJ per day captured by the APPetite-food record and the 24-hour recall (red line: mean difference=app–24-hour recall; dark red lines: 95% CI of mean difference; blue lines: lower and upper limits of agreement; dark blue lines: 95% CI of lower and upper limits of agreement).
Energy intake from the APPetite-food record was significantly lower than the TEE estimated from accelerometry on the first day (t43=5.33;
The APPetite-mobile-app was developed to capture complex dietary intake, objectively recorded PA, and related factors for studying microtemporal, within-person processes underlying eating behavior and PA in daily life. This study evaluated the feasibility and usability of the EMA tool as well as the validity of the APPetite-food record. The APPetite-mobile-app demonstrated good feasibility. Compliance with responding to prompts and wearing the accelerometer was above 80%, and reasonable response times and latencies were found for the prompts as well as the food record. Usability was rated moderate, with a mean SUS score of 61.9. Large differences in energy and macronutrient intake assessed with the APPetite-food record versus the 24-hour recall were found at the group and individual levels, whereby the APPetite-food record captured higher dietary intakes. Energy intake was assessed fairly accurately by the APPetite-food record on the group level on 2 of 3 days when compared with TEE. The comparison of habitual energy intake to mean TEE showed that the 24-hour recall underestimated energy intake to a larger degree than the APPetite-food record. These results indicate that the discrepancies between the two dietary assessment methods do not imply a lack of validity of the APPetite-food record; rather, they indicate a more accurate dietary assessment compared with the 24-hour recall and therefore provide the first evidence that the APPetite-food record is a valid tool for capturing complex dietary intake.
The good feasibility of EMA tools is crucial to ensure unbiased data collection and prevent systematic missing data. Compliance rates are an important indicator of feasibility. Although there is no official criterion indicating good compliance, Stone and Shiffman [
Furthermore, the results demonstrate that prompt compliance did not decrease over the course of 3 days. In other EMA studies that assessed more than 3 days, response rates declined substantially (eg, 40% from 63% on day 1 to 23% on day 7), even with only 4 prompts per day [
We found marginally lower prompt compliance rates in the clinical cohorts than in the healthy cohort. In a study by Porras-Segovia et al [
A mean of 2.04 (SD 1.17) minutes was needed to complete 1 prompt. In accordance with the high prompt compliance rate, the response duration of the prompts can be considered feasible. Responding to a prompt was initiated on average 3.16 (SD 6.48) minutes after the first prompt signal. Short prompt latencies are essential to guarantee the momentary nature of the response and should therefore be taken into account thoroughly. However, most studies have not reported prompt latencies [
Compliance with the food record cannot be directly determined, as it is not possible to differentiate between someone not recording a food item because of noncompliance or because of not actually consuming it. However, other quality measures could also be used. The time spent reporting daily dietary intake or the number of recorded eating and drinking occasions per day can be used for quality checks. On average, participants needed 17.6 minutes to complete the food record of 1 day. Other technology-based tools for assessing dietary intake show similar times to complete, ranging between 13 and 45 minutes [
The APPetite-food record was developed to record food intake in real time or near real time. Therefore, it is important to consider the amount of time between food intake and recording. Foods and drinks were recorded on average 58.35 minutes after intake. This shows that participants did not wait until the evening to record all eating and drinking occasions for 1 day. Hence, food intake was recorded in real time or near real time.
The food recording behavior of our participants suggests that the APPetite-food record is feasible. However, we noticed that the participants’ motivation was crucial for successfully capturing sufficient and accurate dietary data. Training is needed to ensure that participants understand the importance of food recording in real time or near real time. Furthermore, participants reported that receiving detailed dietary feedback at the end of the study increased their motivation to enter food intake accurately.
As expected from previous studies [
In addition to good feasibility, usability is an important criterion that should be considered when developing new EMA tools. The usability of the APPetite-mobile-app was rated as moderate, with an SUS score of 61.9 out of 100. In a previous study, the usability of the top 7 iPhone operating system and Android diet-tracking apps was assessed [
A food record was incorporated into the APPetite-mobile-app to capture complex dietary intake in real time or near real time. An evaluation of validity was needed to test whether the APPetite-food record accurately assessed dietary intake. Hence, the APPetite-food record was compared with a 24-hour recall and TEE estimated from nondominant wrist accelerometry.
With regard to relative validity, low agreement between habitual dietary intake measured by the APPetite-food record and the 24-hour recall was found at both the group and individual levels. At the group level, energy, protein, fat, and fiber intake from the APPetite-food record was significantly higher than the 24-hour recall. Wide LoA, which exceeded the predefined acceptable LoA, were found for energy, protein, fat, carbohydrate, sugar, and fiber intake at the individual level. One could argue that these discrepancies indicate a lack of validity in the APPetite-food record. However, even though 24-hour recalls are frequently used as the established reference method when assessing relative validity, the true intake remains unknown [
This interpretation is underlined by the comparison of energy intake and TEE estimated from accelerometry. Energy intake from the APPetite-food record was not significantly different from TEE on 2 of 3 days, indicating that the APPetite-food record assesses energy intake fairly accurately at the group level. However, the comparison with the mean TEE showed that both methods underestimated habitual energy intake. In this context, it must be mentioned that over one-third of the participants in the validation study (17/44, 39%) indicated that they are currently trying to lose weight. Therefore, the discrepancy between the TEE and the reported energy intake could, to some extent, be due to diet and weight loss. However, the 24-hour recall underestimated habitual energy intake to a greater extent. Inaccurate estimates of energy intake captured by 24-hour recalls have been reported in previous studies [
The results of the evaluation of validity indicate that the APPetite-food record might assess dietary intake more accurately than the 24-hour recall and capture daily energy intake fairly accurately at the group level. Nevertheless, both dietary assessment methods seem to underestimate habitual energy intake.
A common validation approach is the assessment of relative validity, which compares a novel tool to an established dietary assessment method. However, most of the available validation studies show methodological issues as they assess relative validity on overlapping days [
One limitation of our validation study is due to the fact that dietary intake varies from day to day. Bland and Altman call this case “method where true value varies” [
Many studies that use Bland-Altman agreement analyses to evaluate the validity of food records in real time or near real time have inaccuracies. To the best of our knowledge, our study is the only one that has a predefined acceptable LoA. These pre-established limits are necessary to avoid misleading interpretations. A consensus on the acceptable LoA for dietary intake is desirable. This will improve the comparability of the results from studies assessing relative validity. Furthermore, the use of established but biased dietary assessment methods, such as 24-hour recalls, to study relative validity should be questioned critically. New approaches to evaluate the validity of food records in real time or near real time are needed.
Our findings are limited because of the lack of control for possible weight changes during study participation. The comparison of TEE and energy intake is based on the assumption that energy expenditure is equal to energy intake. However, this assumption is valid only for weight-stable individuals.
Two further limitations concern the APPetite-food record itself: (1) nutritional values are generated manually, which is time-consuming and can be error-prone. Automated generation is preferable. (2) The APPetite-food record relies on self-reports of dietary intake. Self-reports are subjective and therefore more likely to be biased. To add a more objective component to the dietary assessment, photos of the foods and drinks consumed could be taken in addition to self-reports.
The strength of our assessment of feasibility and usability is that the sample of healthy participants was enriched with data from patients suffering from a mental disorder. Therefore, it was possible to show that the APPetite-mobile-app is equally feasible and usable in this population. This finding is particularly important as diet and PA play an important role in mental health. This opens up the possibility of studying microtemporal, within-person processes of diet, PA, and related factors in psychiatric patients, which is crucial for the understanding of the link among diet, PA, and mental health. However, the unequal sample sizes of the 3 cohorts limit the results. This is of concern in the context of cohort comparisons, as well as the interpretation of the means of the total sample. Furthermore, a selection bias could be present, as the participants were exclusively recruited from 3 existing study cohorts.
The development of novel EMA tools for assessing microtemporal processes of diet, PA, and related factors is required. Studies comparing these new EMA tools are needed to establish empirical evidence on which assessment approaches are most effective in the study of microtemporal processes. Future EMA studies should consider that participants’ motivation is the key to success, especially when complex dietary intake is assessed. Therefore, participants’ burden needs to be kept minimal, and incentives for prompt responding and food recording, such as dietary feedback and raffle inclusions, are essential.
New technologies and wearable sensors are a promising advancement in the area of dietary assessment in naturalistic settings, as they can passively detect eating behavior [
The APPetite-mobile-app is a promising tool for studying microtemporal, within-person processes of diet, PA, and related factors in real time or near real time and is, to the best of our knowledge, the first of its kind. First evidence supports that the APPetite-mobile-app is feasible and the APPetite-food record is a valid tool for capturing complex dietary intake. We hope this motivates other researchers to use EMA to capture complex dietary intake, PA, and associated factors in daily life, and it initiates a discussion about feasible, usable, and valid methods to assess these dynamics. Assessment strategies need to be developed, shared, and discussed to advance the research field. A solid empirical foundation regarding within-person, microtemporal associations of diet, PA, and associated factors is needed for the development of personalized lifestyle modification interventions, such as intensively adaptive interventions or just-in-time adaptive interventions [
APPetite-food record and nutritional value generation.
Reasons for food intake.
Prompt measures and items.
Bland-Altman analysis of protein, fat, carbohydrate, sugar, and fiber intake.
ecological momentary assessment
limits of agreement
Longitudinal Resilience Assessment
physical activity
Prevention of Comorbid Depression and Obesity in Attention-Deficit/Hyperactivity Disorder
System Usability Scale
total energy expenditure
This work was supported by the European Union’s Horizon 2020 Research and Innovation Program under grant agreement number 728018. The funding source had no involvement in the study design, data collection, interpretation of the findings, or writing of this manuscript.
UEP reports consultancy for Boehringer-Ingelheim. All other authors declare no conflicts of interest.