The development of Meal-Q was based on results from a population-based study in which 700 randomly selected Swedish participants reported, through either face-to-face interviews or telephone 24-hour recalls, on the food products they consumed for breakfast, lunch, dinner, and snacks (E Möller and S Christensen, personal written communication, August 2008). This dietary information guided the design of a meal- and Web-based FFQ called MaxMeal-Q. After a pretest of MaxMeal-Q in a randomly selected group of individuals (N=216), the shorter version, Meal-Q, was formed by omitting less commonly consumed food items and dishes. Subsequently, Meal-Q was included in the Validation of Methods Assessing diet and physical activity (VALMA) study. The reference methods were a 7-day WFR on the Web and DLW for estimation of energy expenditure [6].The Research Ethics committee at Karolinska Institutet approved the study.

After the validation study was completed, researchers from LifeGene, a large population-based cohort study [7], decided to use Meal-Q under the condition that the answering time be reduced. Therefore, we developed the shorter version, MiniMeal-Q, by omitting food items consumed on average with a low intake frequency and that contributed least to the total energy and nutrient intake. Yet, food items representing important food sources of certain nutrients were kept (eg, black pudding that contributes to iron intake). After a time test, LifeGene decided to use MiniMeal-Q. We validated MiniMeal-Q in the present study by truncating Meal-Q data to simulate MiniMeal-Q. The inherent dependence between Meal-Q and MiniMeal-Q should be taken into account when comparing their validity.

In April 2009, 180 healthy volunteer men and women aged 20 to 63 years were recruited to the VALMA study through public advertisement in Stockholm County, Sweden. Recruitment also took place at 3 universities including announcements among nutritionist students. Access to the Internet and an email address were prerequisites for eligibility, as well as not being on a weight-loss diet, not being pregnant, and not having given birth within the past 10 months. At an introductory meeting, participants were informed about the study and signed informed consent forms. Participants self-reported their height and weight, which was used to calculate body mass index (BMI).

After recruitment, the participants were divided into 3 age- and gender-balanced groups: group 1 (n=87), group 2 (n=53), and group 3 (n=40). Each group followed a 3-week study scheme shown in Figure 1.

All groups filled out Meal-Q and the WFR on the Web at their own choice of location (eg, at home) and group 3 was also given DLW. Groups 2 and 3 filled out a second Meal-Q after 3 weeks. Data from the first administered Meal-Q assessment and the WFR from all groups were compared for validity evaluation. The data from the first Meal-Q assessment was truncated for simulated analysis of MiniMeal-Q. The first and second Meal-Q assessments from groups 2 and 3 were compared for reproducibility evaluation. Information about education, occupation, and tobacco use (smoking and Swedish snuff) was collected in the first questionnaire. Answering time was automatically recorded, and directly after completion of the first Meal-Q, a short Web survey followed to evaluate its ease of use.

Total energy expenditure was determined in group 3 (n=40) using the DLW method [10] over 11 consecutive days (Figure 1). The details of this procedure have been described previously [11]. Briefly, on day 1 at the study site, each participant provided a 5-milliliter urine sample before receiving an oral DLW dose calculated according to body weight [12]. Subsequently, daily spot urine samples were collected for a total of 9 days. Participants were instructed to collect urine samples at a similar time each day (but not the first void of the day). All samples were kept refrigerated. On day 11, the 9 urine samples were returned to the study site and the eleventh urine sample was collected. All samples were shipped to the Medical Research Council, Human Nutrition Research, Cambridge, United Kingdom, for isotopic analysis, which has been previously described in detail [13]. Enrichments of ²H/¹H and ¹⁸O/¹⁶O in urine samples were determined by mass spectrometry. Following conversion to the universally accepted Vienna Standard Mean Ocean Water (VSMOW) / Standard Light Arctic Precipitation (SLAP) scale, total energy expenditure (TEE) was calculated by using standard equations [14-16]. CO₂ production (mole/day) was estimated using Schoeller et al’s correction for fractionation [15] and a respiratory quotient of 0.85. The respiratory quotient is based on omnivores with 30% to 35% energy contribution from fat and suitable to the VALMA population. The results of the CO₂ production were used to calculate the TEE of each participant by using the modified Weir equation [17].

Descriptive characteristics of study participants are presented as mean (SD) and as counts (%). Differences in BMI and age between study groups, between men and women, and between included and excluded participants were assessed using a 2-sample t test. Differences in education, nutrition background (studying or working in the nutrition field), and tobacco use were assessed using Fisher’s exact test. The level of statistical significance was set to alpha =.05.

Median and interquartile range (IQR) of crude energy and macronutrient intake was calculated and compared among Meal-Q, MiniMeal-Q, and the WFR. Energy intake from the questionnaires was also compared to TEE from DLW. Wilcoxon signed rank tests were used to determine differences between all methods. The median (IQR) answering time in minutes of each questionnaire was calculated and ease of use was evaluated from the Web survey. The between-person variance captured in the truncated MiniMeal-Q as compared to Meal-Q was calculated using linear regression.

For validity and reproducibility analyses, macronutrients were adjusted for total energy intake using the residual method [18]. Variables deviating from the normal distribution were transformed using the square, square root, or log transformation, as appropriate. Absolute agreement and potential difference in bias within the intake range were evaluated by plotting the differences between questionnaires and WFR or DLW against the average of the 2 methods, according to the method of Bland and Altman [19]. The degree of variation was represented by the limits of agreement, ie, ±2 SD of the mean difference. The ranking agreement and magnitude of misclassification when comparing questionnaires with the WFR and DLW was tested by dividing participants into quartile categories of energy and energy-adjusted macronutrient intake. Proportions of participants classified into the same, adjacent, and extreme quartiles were calculated. Because variables were normally distributed after energy adjustment and transformation, Pearson correlation coefficients were used to measure the degree of linear relationship between the questionnaires and the WFR and DLW. Deattenuated correlations corrected for within-person variation in the WFR were calculated using the formulas of Beaton et al [20] and Liu et al [21], and confidence intervals (CI) were produced using the method of Willett and Rosner [22]. Confidence intervals for correlations with DLW were obtained using the bootstrap method [23].

Reproducibility of Meal-Q was evaluated by comparing crude median energy and macronutrient intake between the first and second Meal-Q and by cross-classification of energy and energy-adjusted [18] quartiles of macronutrient intake. Intraclass correlation coefficients (ICCs) [24] were also computed using 1-way ANOVA with random effects. Statistical analyses were performed using STATA statistical software version 11.2 (StataCorp LP, College Station, TX, USA).

As shown in Table 1, most of the study participants were students or highly educated. One-third were working full time, and almost as many had a nutrition background. Few participants used tobacco. There was no statistically significant difference between groups or sexes regarding age, BMI, education, nutrition background, or smoking (but more men than women used Swedish snuff).

The median time to answer the Meal-Q and the MiniMeal-Q was 17 (IQR 11) and 7 (IQR 4) minutes, respectively. Most (92%) participants perceived Meal-Q as easy to fill out, 91% thought the questions were relevant, and 93% reported that food items and dishes were presented in a logical order. For MiniMeal-Q, the figures were 95%, 88%, and 91%, respectively. The overall mean grade of Meal-Q and MiniMeal-Q’s ease of use was 4.2 on a 5-point scale in which 5 was the best grade. The between-person variance captured by MiniMeal-Q as compared to Meal-Q ranged from 96% to 99% for energy and macronutrients.

Energy and macronutrient intake was higher in the WFR compared with both questionnaires, except for polyunsaturated fat assessed with Meal-Q (Table 2). In group 3 (n=39), the energy expenditure from DLW was higher than energy intake assessed by both questionnaires (P<.001, Wilcoxon’s signed rank test). The energy expenditure from DLW was 11,423 kJ (IQR 2777) and the energy intake assessed with Meal-Q and MiniMeal-Q were 7954 kJ (IQR 2736) and 7358 kJ (IQR 2718), respectively.

As shown in Figure 3, the Bland-Altman plots with DLW indicate that the WFR and both questionnaires underestimated energy intake for most participants. Compared to the WFR, the questionnaires had a larger underestimation, larger variance, and a weak trend of decreasing accuracy with increasing intakes.

The Bland-Altman plots of Meal-Q and the WFR in Figure 4 showed a negative mean difference for energy and all macronutrients. There was a trend of decreasing accuracy with increasing energy and polyunsaturated fat intake, and trends of increasing underestimation with increasing intakes for other macronutrients. Because of varying bias within the intake range, the proportion of participants outside the limits of agreement deviated somewhat from 5% for some plots. Bland-Altman plots for MiniMeal-Q and the WFR were very similar to those for Meal-Q and the WFR (see Multimedia Appendix 1).

Table 3 shows that the proportion of participants classified into the same or adjacent quartile for energy was 70% by Meal-Q and 67% by MiniMeal-Q as compared to the WFR. Correspondingly, the proportions for macronutrients ranged from 76% to 86% for both questionnaires. Quartile cross-classification of Meal-Q and DLW placed 77% into the same or adjacent quartile, and values were identical for MiniMeal-Q.

Pearson correlation coefficients (r) with the WFR and DLW were similar between Meal-Q and MiniMeal-Q (Table 4). Deattenuated and energy-adjusted correlations with the WFR ranged from r=0.18-0.73 for Meal-Q and from r=0.18-0.74 for MiniMeal-Q. Correlation with DLW was r=0.42 for Meal-Q and r=0.38 for MiniMeal-Q.

This study presents the validity and reproducibility of the new meal- and Web-based interactive Meal-Q, as well as a simulated validation of its shorter version, MiniMeal-Q. Both questionnaires were perceived as easy to use and had a short answering time. Trends of varying bias within the intake range were seen for energy and all macronutrients. Both questionnaires showed good ranking ability for carbohydrates, total fat, saturated fat, monounsaturated fat, and alcohol, whereas energy, protein, and polyunsaturated fat performed less well. Furthermore, Meal-Q showed fair reproducibility.

The Bland-Altman plots of the questionnaires versus WFR and DLW and the plot on WFR versus DLW showed a varying bias within the intake range. Energy and polyunsaturated fat both seemed to be underestimated and overestimated for both questionnaires at higher intake levels. For other macronutrients, the plots indicated that the questionnaires had difficulty assessing higher intakes. In contrast, quartile cross-classification with the WFR showed a fair ranking agreement for most of the nutrients, although a lower agreement was seen for energy and polyunsaturated fat. Similar rankings have been seen in 3 other validation studies of FFQs against food records [25-27] including a Web-based method [25]. In nutritional epidemiology, the association between diet and disease is commonly studied by ranking the dietary intake; therefore, absolute intake is often less important than good ranking order [18]. Hence, despite an underestimation of absolute intake, the ranking agreement for Meal-Q and MiniMeal-Q suggests they are useful in epidemiologic studies regarding most nutrients.

The correlations between the questionnaires and the WFR ranged from 0.18 for energy to 0.74 for alcohol. High correlations between FFQs and diet records are in the order of 0.6-0.7 and it is unlikely that correlations above 0.8 can be obtained [28]. A review of FFQs concluded that the mean correlation with food records of ≥6 days was 0.42 for energy, 0.57 for total fat, 0.53 for protein, 0.58 for carbohydrates, and 0.76 for alcohol [29]. In light of this literature, energy and protein seemed to perform less well, whereas other macronutrients showed correlations within expected ranges. A limitation of the FFQ methodology is the predefined number of food items, frequencies, and portion sizes, which could lead to a “flattened slope” effect in scatter plots [30]. This is a result of respondents consuming little food to unintentionally overreport, and for those consuming a lot to underreport. Correlations from such data would be artificially low. However, a truly small between-person variance would also give similar results [31]. Therefore, the low to moderate correlations seen in this study could reflect a limitation of the questionnaire design, but may also reveal a true small between-person variance. Bland and Altman have discouraged the use of correlation coefficients to evaluate validity because they do not measure agreement [19]. However, because the use of correlations in validation studies is widespread, we have included them to enable comparisons with other studies.

The DLW measurements in group 3 showed that the WFR, Meal-Q, and MiniMeal-Q underestimated energy intake by 17%, 30%, and 36%, respectively. Similar figures for food records and FFQs have been seen in other studies using DLW [32-34]. The Bland-Altman plots showed that the underestimation of energy was considerable for both questionnaires and the large variance indicated difficulties in precision. The underestimation and variance was much smaller for the WFR. Correlations with DLW were moderate for both questionnaires, although the CIs were wide because of the large variance. The correlations were similar to a study by Andersen et al [35], but slightly lower than that of Kroke et al [36]. Despite the underestimation and the large variance, quartile cross-classification with DLW showed a fair ranking agreement, similar to that found by Kroke et al [36].

The moderate to strong quartile cross-classifications of the first and second Meal-Q assessments suggest the questionnaire has fair reproducibility. Correlations between repeated administrations of FFQs in other studies have ranged r=0.5-0.8 [31], and Meal-Q showed similar results. The reproducibility might have been affected by the short time period between the Meal-Q assessments, because participants are less likely to have true changes in intake or response after a short compared to a longer period [37]. In addition, it is important to keep in mind that the reproducibility cannot reveal systematic errors, which can be masked in a high correlation between 2 questionnaires.

The high between-person variance captured by MiniMeal-Q as compared to Meal-Q indicates that it is possible to use a shorter questionnaire while still assessing a similar intake range and keeping the ranking ability. Because MiniMeal-Q originates from Meal-Q data and is also compared to the same reference methods, their results become highly related. Therefore, caution must be taken when comparing their assessments and relative validity.

Regular use of the Internet in Sweden is higher among young people compared to older people. Among those aged 16 to 44 years, 88% to 94% use the Internet daily, whereas the proportion among the age groups 45 to 54 years and 65 to 74 years are 82% to 83% and 38% to 49%, respectively [3]. However, access to the Internet is high for all age groups—more than 90% for the young and middle-aged and 67% to 78% for the oldest age group. It is worth noticing that problems with cognition might be an issue in very old age groups, although this would also hold true for dietary assessment using a paper-based questionnaire. Concerns could be raised regarding whether Web-based questionnaires produce different kinds of bias as compared to paper-based questionnaires. However, bias associated with Web-based data collection does not seem to differ from that of paper-based questionnaires as seen in a large Swedish feasibility study of more than 45,000 participants [38].

To estimate the validity of a dietary assessment method, 2 statistical assumptions should be fulfilled. First, the assessed dietary intake should be linearly related to true intake. Second, the measurement errors should be independent between the test and the reference method. In this validation study, variables were linearly correlated to the WFR, although energy, protein, and polyunsaturated fat had a weaker linear relationship. The questionnaires rely on memory and have predefined food items, frequencies, and portion sizes, whereas the WFR does not rely on memory, is open-ended, and has direct assessment of portion sizes. Nevertheless, the methods are linked to the same nutrient database and are likewise affected by social desirability, which could lead to an overestimated validity.

The strengths of this validation study include its large sample size and few dropouts. There was also high compliance to the questionnaires, the WFR and DLW. Using the DLW method is an additional strength that enabled an objective estimation of TEE for the evaluation of energy intake. The digital format of the questionnaires and the WFR also substantially reduced the risk of coding errors and missing data. The proportion of underreporters in this study (14/177, 8%) was notably lower compared with some other studies [39-42], even if studies have had proportions in the range of 2% to 85% [43]. The use of individual PAL values from each participant for the Goldberg cut-off is likely to have increased the sensitivity [44] and could be an explanation.

Due to time constraints, the study period had to be kept short. This could have given an overestimation of the validity because the questionnaires and the WFR assessments were performed within a short period of time. Furthermore, the WFR was only performed once. Ideally, several records with independent days spread over a longer time period would have reflected the habitual dietary intake better. However, corrections for within-person variance in the WFR were made to minimize day-to-day variation and energy adjustment was made to avoid variations in intake related to total energy intake. The DLW measurement should also preferably have been done repeatedly over a longer time period to reflect habitual energy intake; however, this was not possible. Furthermore, most of the participants were women and many were students with a nutrition background. Also, the self-selection of participants could have biased the sample in favor of more motivated participants who are more inclined to give accurate answers compared to a sample from the general population [45]. Nevertheless, dietary intake from the WFR was in-line with a Swedish national dietary survey using food records (n=1214) [42], suggesting that the WFR intake would be comparable to an assessment within a more general and less-selected population. Acknowledging the highly educated study sample, the answering time might have been longer in a less-educated population.

Meal-Q and MiniMeal-Q are 2 Web-based FFQs shown to be highly user-friendly. Despite their short answering time, they had an ability to rank most macronutrient intakes well compared with the reference methods. In addition, Meal-Q showed fair reproducibility.