Published on 07.12.15 in Vol 17, No 12 (2015): December
Evaluation of a Web-Based Food Record for Children Using Direct Unobtrusive Lunch Observations: A Validation Study
Background: High-quality, Web-based dietary assessment tools for children are needed to reduce cost and improve user-friendliness when studying children’s dietary practices.
Objective: To evaluate the first Web-based dietary assessment tool for children in Norway, the Web-based Food Record (WebFR), by comparing children’s true school lunch intake with recordings in the WebFR, using direct unobtrusive observation as the reference method.
Methods: A total of 117 children, 8-9 years, from Bærum, Norway, were recruited from September to December 2013. Children completed 4 days of recordings in the WebFR, with parental assistance, and were observed during school lunch in the same period by 3 observers. Interobserver reliability assessments were satisfactory. Match, omission, and intrusion rates were calculated to assess the quality of the recordings in the WebFR for different food categories, and for all foods combined. Logistic regression analyses were used to investigate whether body mass index (BMI), parental educational level, parental ethnicity or family structure were associated with having a “low match rate” (≤70%).
Results: Bread and milk were recorded with less bias than spreads, fruits, and vegetables. Mean (SD) for match, omission, and intrusion rates for all foods combined were 73% (27%), 27% (27%), and 19% (26%), respectively. Match rates were statistically significantly associated with parental educational level (low education 52% [32%] versus high 77% [24%], P=.008) and parental ethnicity (non-Norwegian 57% [28%] versus others 75% [26%], P=.04). Only parental ethnicity remained statistically significant in the logistic regression model, showing an adjusted odds ratio of 6.9 and a 95% confidence interval between 1.3 and 36.4.
Conclusions: Compared with other similar studies, our results indicate that the WebFR is in line with, or better than most of other similar tools, yet enhancements could further improve the WebFR.
J Med Internet Res 2015;17(12):e273
High-quality dietary assessment tools are essential when studying children’s dietary practices. Traditional tools, such as food frequency questionnaires, 24-hour recalls, and food records, can be used to assess dietary intake [- ]. In recent years, there has been a shift toward the use of Web-based dietary assessment tools among both adults and the younger age groups [ - ]; those aimed at children are mostly 24-hour recalls, or mixed methods combining food records and 24-hour recalls [ - ]. These new types of digital dietary assessment methods are highly needed [ ].
In comparison with paper-based dietary assessment tools, Web-based tools facilitate data handling and improve user-friendliness; they reduce the burden for both the participant and researcher and can enhance motivation [, ]. Therefore, we have recently adapted the Danish Web-based Dietary Assessment Software for Children (WebDASC) [ ] to Norwegian conditions and food culture to develop the Web-based Food Record (WebFR) for children and adolescents.
It is well established that assessment of dietary intake is associated with errors . Furthermore, assessing children’s intake is especially challenging due to their limited cognitive abilities [ ], and because they often need assistance from a caretaker [ ]. Consequently, validating dietary assessment tools that target children is important [ ].
Direct observation is considered to be an appropriate high-quality method for validation studies of dietary assessment tools, because it provides direct unbiased information regarding what is eaten [, ]. Hence, we aimed to evaluate the accuracy of children’s school lunch entries in the first Web-based dietary assessment tool for children in Norway, the WebFR, using direct unobtrusive observation as the reference method.
All the 4th graders (8-9 years old) from 4 elementary schools in Bærum, the fifth most populated municipality in Norway and a suburb of the capital city, were invited through the schools from September to December 2013. Convenience sampling was used; selected schools were in a short travel distance for the observers and had a highly cooperative school administration. Verbal and written information was given at plenary school meetings and in school classes to parents/guardians and children, respectively. To be included in the study, children needed an Internet access at their home, and their parents/guardians needed access to email. The final sample consisted of 117 of the 196 invited children (59.7%). The study was conducted in accordance with the Declaration of Helsinki. The Regional Ethical Committee in the South East of Norway found the study to fall outside their remit. Approval from the Norwegian Social Science Data Services was obtained, in addition to child assent and written parental consent from all participants.
The participants were instructed to record everything they ate and drank in the WebFR, for 4 consecutive days, including a weekend day. They were instructed to complete the recordings in the WebFR at home, with parental assistance, at the end of each recording day, after all meals were consumed. A practical demonstration was given at school in addition to written instructions on how to use the WebFR. During the days they recorded their diet, each child was observed once during school lunch. The children's weights and heights were also measured using standard procedures. After completing the study, the participants received a personal gift card with 2 cinema tickets.
The Web-Based Food Record
The WebFR is designed as a food record, yet including elements of a dietary recall, as recordings are completed by the end of each recording day. It is structured by meals with photos for portion-size assessments. It was adapted from the WebDASC by replacing its food lists with approximately 550 of the most commonly eaten foods and beverages in Norway, based on data from the latest Norwegian National Dietary Survey . In addition, distinctive products designed for children (eg, children’s yoghurt) were also included based on Norwegian sales statistics. The photo series included both new photos taken specifically for the purpose of the development of the WebFR and selected photo series from the WebDASC, which were found appropriate to exemplify foods in the WebFR food list. Experienced dietitians considered the appropriateness of all the portion sizes included in the WebFR. All text and audio files were translated and slightly altered. The design of the interface was also changed to some extent; yet, the basic structure and functions of the WebDASC remained untouched. When recording, the participant selects the items consumed for each meal from drop down lists, or by using a free text search field. For each item, the participant chooses the most appropriate picture from a photo series with 2-4 photos illustrating different portion sizes, and then selects the number of portions eaten ( ). Some of the photo series serve as proxies for other food items (eg, a glass of milk is illustrated by a glass of apple juice). When a food item is not found in the WebFR, it is possible to record it in an open field. A voice-assisted cartoon character guides the participant through the recordings. Prompts and pop-ups have been designed to ease recording in the WebFR and counter recall bias. Visuals of the WebFR are shown in and .
The observer team included 1 registered dietician and 2 master’s students in nutrition. The observations were performed in classrooms in which the children ate their home-packed lunches during regular school days. Each child was observed one time, during the same period as when they were instructed to record data in the WebFR. Each observer monitored a maximum of 3 children at the same time in an unobtrusive manner (ie, avoiding interaction with the participants and blinding the observations for participants). The children were already familiar with the presence of the observers prior to the observations, through instructional sessions.
The observers used a standardized form to take notes during their observations. To ensure complete recordings, observers were present in the classroom from before the children started eating to until they all had stopped eating. Immediately after each observation session, the observers categorized all observed food items into categories and portion sizes that corresponded to the information in the WebFR, with the aid of tablets containing the lists of categories, items, and all photos found in the WebFR. When the observed foods were not found in the WebFR, the observers described the food item in detail in text and chose the food category and portion size they considered most appropriate for the specific food item. After completion of the data collection, the observer team determined what constituted matches, omissions, and intrusions, using a strict definition; that is, a match was considered a match only when the child and observer clearly described the same item.
Observer training prior to data collection was conducted over a period of 3 weeks, based on the training protocol by Richter et al . Interobserver reliability (IOR), which was expressed as the proportion of agreeing observations between observer pairs, was assessed during the training period and continuously during data collection. The overall agreement of observed food items was 92%, ranging from 88% to 96% between the 3 observer pairs. Lower agreement was found for observed portion sizes, with an overall agreement of 81% and a range from 77% to 88% between observer pairs.
Variables for “matches,” “omissions,” and “intrusions” were created by comparing the observational data with the participants’ school lunch recordings in the WebFR. Matches are items that are both observed as eaten and recorded as eaten by the child; omissions are items that are observed as eaten but not recorded as eaten; and intrusions are items not observed as eaten, but recorded as eaten by the child.
Participants’ height and weight were measured according to standard procedures, without shoes and in light clothing, to the nearest millimeter and 0.1 kg, respectively, by trained personnel. A digital scale was used (TANITA TBF-300, Tanita Corporation, Tokyo, Japan), in the privacy of a separate room, for each participant. Age and sex-specific body mass index (ISO-BMI) cutoffs defining overweight and obesity among the study participants were applied .
Parents/guardians provided information in the written consent form regarding each participant’s sex and age, parental education level (low, intermediate, or high), parental ethnicity (at least one versus no parents/guardians of Norwegian origin), and family structure (mother and father of participant living in same household versus other).
MS Excel (version 2010, Microsoft, Redmond, WA, USA) was used to create all the variables. IBM SPSS (version 21.0, 2012, IBM Corp, New York, NY, USA) was used in all analyses, with the exception of the bias-reduced logistic regression analysis, for which the statistical package R (version 3.0.1, 2013, The R Foundation for Statistical Computing, Vienna, Austria) was used.
Descriptive statistics for the observed food items, recorded food items, matches, omissions, and intrusions were performed. The rates of matches, omissions, and intrusions were calculated for each participant both for all food items combined and at the food item category level (eg, “fruit, berries,” “bread products”). Definitions of these variables are in accordance with previous definitions developed by Baxter et al , but without using weighted values, ie, all food items were given equal statistical weight, and thus equal importance, in this study. “Coinciding omissions and intrusions” were also calculated, that is, cases in which a participant has an omission that corresponds to an intrusion within the same food category and within the same meal (eg, “apple” omitted and “pear” intruded during the same school lunch). The portion sizes of the omitted and intruded food items were counted using 4 different categories as follows: extra small (XS), small (S), medium (M), and large (L), based on the available photo series in the WebFR. Unclear observations of food items or portion sizes were excluded from analyses, such as the amount of water consumed from an opaque drinking bottle.
Univariate analyses were conducted to find possible differences in the match rates, omission rates, and intrusion rates as continuous variables, for all foods combined, with regard to the following variables: sex, BMI category, parental educational level, parental ethnicity, and family structure. Parametric tests were used when appropriate. Because the omission rate is the inverse of the match rate (match rate=100 - omission rate), testing for the match rate was therefore equivalent to testing for the omission rate.
A log transformation of the match rate variable was conducted; nevertheless, the assumptions for doing a multivariate linear regression were not present. Hence, match rates were further recoded to a dichotomous variable, which was defined as either a “low match rate” (≤70%) or “high match rate” (>70%). Logistic regression analyses were used to investigate the association between participant characteristics and the quality of the recordings in the WebFR (ie, low versus high match rate). Because of low cell counts, Logistf (bias-reduced logistic regression, Firth correction)  was also conducted in the statistical package R to compare the results with those from the logistic regression. The reporting in this study is in line with the STROBE guidelines [ ].
The characteristics of the study sample are shown in. A total of 15 of 117 participants (12.8%) were overweight or obese. Furthermore, the parental education level was high among 77 of 111 children (69.4%), and a great majority had at least one parent/guardian of Norwegian origin.
|ISO-BMI cutoff categories|
|Overweight or obese||15||12.8|
|Parental education levela|
|At least one parent/guardian of Norwegian origin||105||91.3|
|Both parents/guardians of ethnic origin other than Norwegian||10||8.7|
|Mother and father of participant living in same household||87||78.4|
aInformation from 111 participants was available for “parental education level.” Complete information on both parents/guardians was available from 108 participants; the 3 cases with missing information from 1 parent/guardian were included in the table based on the 1 available parent/guardian\'s educational level.
bBoth parents/guardians\' education was maximum high-school level.
cOne parent/guardian\'s education was maximum high-school level, and the second parent/guardian\'s education was at university-college or university level.
dBoth parents/guardians\' education was at the university college or university level.
eInformation from 115 participants was available for “parental ethnicity.”
fInformation from 111 participants was available for “family structure.”
shows omission rates and intrusion rates for all food items combined, and for categories of food items, listed in descending order from the most to the least frequently observed. The overall averages for the omission and intrusion rates were 27% and 19%, respectively. At the food category level, the average rates varied widely between categories; “bread products” and “milk” were both frequently eaten, and had the lowest omission rates at 5% and 6%, respectively. “Spreads” were eaten most frequently, but the degree of omissions was higher. Food items in the categories “fruit, berries” and “vegetables, salads” were the third and fourth most frequently eaten, also with a high degree of omissions. By contrast, “biscuits, buns, waffles, cakes, and candy” were eaten infrequently, but had the highest proportion of omissions (85%). For intrusions, “bread products” and “dinner leftovers” had the lowest rates, and “beverages, other” and “yogurt” the highest.
|Coinciding omissions and |
|Nd||Mean (SD)||Nd||Mean (SD)||Ne||n (%)|
|All food items||117||27 (27)||117||19 (26)||136||18 (13.2)|
|Spreads||93||29 (43)||79||17 (33)||41||7 (17.1)|
|Bread products||95||5 (22)||97||7 (26)||5||3 (60.0)|
|Fruit, berries||42||39 (48)||36||25 (44)||22||1 (4.5)|
|Vegetables, salads||33||45 (49)||23||21 (39)||23||0 (0.0)|
|Milk||49||6 (24)||52||12 (32)||3||1 (33.3)|
|Beverages, otherf||44||18 (39)||62||42 (50)||8||2 (25.0)|
|Dinner leftovers||17||33 (43)||14||7 (27)||7||0 (0.0)|
|Miscellaneous||17||44 (50)||12||21 (40)||8||1 (12.5)|
|Biscuits, buns, waffles, cakes, and candy||12||85 (31)||4||38 (48)||12||1 (8.3)|
|Yogurt||11||64 (50)||9||56 (53)||7||2 (28.6)|
aOmission rate = omissions/observed eaten food items × 100 = omissions/(omissions + matches) × 100. Omission rates were calculated for each participant within the different food categories. Participants who were not observed eating foods within a certain category (eg, “fruit, berries”) were excluded from the analyses for this category, regardless of what was recorded eaten.
bIntrusion rate = intrusions/recorded eaten food items × 100 = intrusions/(intrusions + matches) × 100. Intrusion rates were calculated for each participant within the different food categories. Participants who did not record eating foods within a certain category (eg, “fruit, berries”) were excluded from the analyses for this category, regardless of what was observed eaten.
cCases where a participant had an omission that corresponds to an intrusion, within the same food category and within the same meal. For example, “apple” omitted and “pear” intruded during the same school lunch. Formula used: coinciding omissions and intrusions/omissions × 100.
dNumber of participants included in analyses.
eNumber of food items included in analyses.
fOf all intruded “beverages, other” 96% are drinking water.
In addition,shows that 18 of all 136 omissions (13.2%) were “coinciding omissions and intrusions”; this proportion varied greatly within the different food item categories. Out of the small number of omissions in the “bread products” category, 3 out of 5 (60%) were coinciding omissions and intrusions, thus most of the omissions in this category were minor errors (eg, white bread replaced by whole grain bread). By contrast, the categories “spreads,” “fruit, berries,” and “vegetables, salads” had low proportions of coinciding omissions and intrusions, and thus most of the omissions in these categories were food items that the participants simply did not record.
Omissions and intrusions of large portion sizes are considered to be more severe than the omission or intrusion of small portion sizes. In, the proportion of different portion sizes for omissions and intrusions is presented for all food items combined, and for each food category. Although all types of portion sizes were omitted overall, there was a trend toward omitting smaller portions. This was not the case for overall intrusions, in which the medium- and large-portion sizes intruded more often than the smaller ones.
|Items||Nd||Proportion of different sizesc of omitted food items, n (%)||Nf||Proportion of different sizesc of intruded food items, n (%)|
|All food items||136||28 (20.6)||29 (21.3)||21 (15.4)||22 (16.2)||36 (26.5)||91||9 (9.9)||24 (26.4)||30 (33.3)||28 (30.8)||—|
|Spreads||41||7 (17.1)||7 (17.1)||12 (29.3)||3 (7.3)||12 (29.3)||22||2 (9.1)||10 (45.5)||6 (27.3)||4 (18.2)||—|
|Bread products||5||—||1 (20.0)||1 (20.0)||3 (60.0)||0 (0.0)||7||—||0 (0.0)||5 (71.4)||2 (28.6)||—|
|Fruit, berries||22||10 (45.5)||5 (22.7)||0 (0.0)||3 (13.6)||4 (18.2)||12||2 (16.7)||2 (16.7)||3 (25.0)||5 (41.7)||—|
|Vegetables, salads||23||5 (21.7)||9 (39.1)||5 (21.7)||1 (4.3)||3 (13.0)||7||1 (14.3)||4 (57.1)||1 (14.3)||1 (14.3)||—|
|Milk||3||0 (0.0)||0 (0.0)||0 (0.0)||2 (66.7)||1 (33.3)||6||0 (0.0)||0 (0.0)||2 (33.3)||4 (66.7)||—|
|Beverages, other||8||0 (0.0)||0 (0.0)||0 (0.0)||0 (0.0)||8 (100.0)||26||2 (7.7)||8 (30.8)||9 (34.6)||7 (26.9)||—|
|Dinner leftovers||7||1 (14.3)||2 (28.6)||2 (28.6)||1 (14. 3)||1 (14.3)||1||1 (100)||0 (0.0)||0 (0.0)||0 (0.0)||—|
|Miscellaneous||8||1 (12.5)||0 (0.0)||1 (12.5)||2 (25.0)||4 (50.0)||3||0 (0.0)||0 (0.0)||3 (100.0)||0 (0.0)||—|
|Biscuits, buns, waffles, cakes, and candy||12||4 (33.3)||5 (41.7)||0 (0.0)||2 (16.7)||1 (8.3)||2||1 (50.0)||0 (0.0)||1 (50.0)||0 (0.0)||—|
|Yogurt||7||—||0 (0.0)||0 (0.0)||5 (71.4)||2 (28.6)||5||—||0 (0.0)||0 (0.0)||5 (100.0)||—|
aItems observed eaten, but not recorded.
bItems recorded, but not observed eaten.
cPortion sizes were divided into the following categories: XS=extra small, S=small, M=medium, L=large, based on the photo series available for each food item.
dNumber of omitted food items included in analyses.
ePortion size not possible to observe with certainty, that is, when participants drank from dark-colored drinking bottles or milk cartons, or when participants ate a sandwich where spreads were partially hidden because it was placed in between 2 slices of bread.
fNumber of intruded food items included in analyses.
The very few omissions in the “bread products” and “milk” categories were mostly of large portion sizes, whereas the omitted portion sizes from “spreads” were mostly of medium sizes. By contrast, the majority of omitted items in the categories “fruit, berries” and “vegetables, salads” were of small portion sizes.
Along the same lines as the omissions, the few intrusions in the categories “bread products” and “milk” were all of medium or large sizes. In the categories “fruit, berries,” “vegetables, salads,” and “spreads,” intrusions occurred for all portion sizes.
Mean rates within subgroups are presented in. Children with normal weight tended to have lower omission rates, than their overweight or obese peers. The omission rates differed in a statistically significant fashion between the parental education levels; higher omission rates were associated with lower parental educational levels (P=.008). Furthermore, we found a statistically significant lower omission rate among children with at least one parent/guardian of Norwegian origin in comparison with children having both parents/guardians of ethnic origins other than Norwegian (P=.04). A lower omission rate was also observed among participants from homes in which the mother and father lived together, compared with children from homes with a different family structure.
|Variables||Total (N)||Match rate |
|Omission rate |
|Intrusion rate |
|Mean (SD)||Pd||Mean (SD)||Pd||Mean (SD)||Pd|
|Total participants (N)||117||73 (27)||27 (27)||19 (26)|
|Girls||64||71 (30)||29 (30)||22 (29)|
|Boys||53||76 (22)||24 (22)||16 (23)|
|ISO-BMI cutoff categories||.44||.44||.80|
|Normal weight||102||74 (27)||26 (27)||19 (26)|
|Overweight or obese||15||69 (27)||31 (27)||21 (28)|
|Parental education levele||.008||.008||.006|
|Lowf||12||52 (32)||48 (32)||40 (38)|
|Intermediateg||22||69 (31)||31 (31)||24 (32)|
|Highh||77||77 (24)||23 (24)||15 (21)|
|At least one parent/guardian of Norwegian origin||105||75 (26)||25 (26)||19 (26)|
|Both parents/guardians of other ethnic |
origin than Norwegian
|10||57 (28)||44 (28)||24 (27)|
|Mother and father of participant living in same household||87||75 (27)||25 (27)||20 (26)|
|Other||24||64 (29)||36 (29)||21 (31)|
aMatch rate = matches/observed eaten food items × 100 = matches/(omissions + matches) × 100. Match rates were calculated for each participant, for all food items combined.
bOmission rate = omissions/observed eaten food items × 100 = omissions/(omissions+ matches) × 100. Omission rates were calculated for each participant, for all food items combined.
cIntrusion rate = intrusions/recorded eaten food items × 100 = intrusions/(intrusions+ matches) × 100. Intrusion rates were calculated for each participant, for all food items combined.
dP value for comparison of groups. Analysis of variance and t test were used when applicable; if not, the nonparametric Mann-Whitney or Kruskal-Wallis test was used.
eInformation from 111 participants was available for “parental education level.” Complete information on both parents/guardians was available from 108 participants; the 3 cases with missing information from 1 parent/guardian were included in the table based on the 1 available parent/guardian\'s educational level.
fBoth parents/guardians\' education was maximum high-school level.
gOne parent/guardian\'s education was maximum high-school level, and the second parent/guardian\'s education was at the university college or university level.
hBoth parents/guardians\' education was at the university college or university level.
iInformation from 115 participants was available for “parental ethnicity.”
jInformation from 111 participants was available for “family structure.”
For intrusion rates, the differences between groups were not statistically significant, except for parental education wherein higher intrusion rates were associated with lower parental educational levels (P=.006).
The logistic regression model inshows that parental ethnic background and parental education level were the most important variables associated with a “low match rate” (≤70%). Although the “low educational level” effect was reduced when adjusting for other variables, this variable was still borderline significant. Parental ethnicity was the single most important variable associated with a low match rate; if both parents/guardians had an ethnic background other than Norwegian, the odds for a “low match rate” (≤70%) were higher than if at least one parent/guardian was of Norwegian ethnicity. However, the confidence intervals were wide. The results from the Logistf (bias-reduced logistic regression, Firth correction), analyzed due to low cell counts, were consistent with the results from the logistic regression model. Thus, the standard logistic regression was kept as the final model.
|Variables||n (%) of children||Odds ratio (95% CI)|
|With low match rate (≤70%)|
|Normal weight||96 (86.5)||36 (81.8)||1||1|
|Overweight or obese||15 (13.5)||8 (18.2)||1.9 (0.6-5.7)||1.6 (0.4-5.4)|
|Norwegian origin||101 (91.0)||36 (81.8)||1||1|
|Non-Norwegians||10 (9.0)||8 (18.2)||7.2 (1.5-35.9)||6.9 (1.3-36.4)|
|Parental education level|
|High||77 (69.4)||25 (56.8)||1||1|
|Intermediate||22 (19.8)||10 (22.7)||1.7 (0.7-4.6)||1.6 (0.6-4.5)|
|Low||12 (10.8)||9 (20.5)||6.2 (1.6-25.1)||3.8 (0.9-17.2)|
|Mother and father of participant living in same household||87 (78.4)||31 (70.5)||1||1|
|Other||24 (21.6)||13 (29.5)||2.1 (0.9-5.3)||2.0 (0.7-5.3)|
aAdjusted for all other variables in the model in a logistic regression analyses.
bISO-BMI cutoffs applied.
cBoth parents/guardians of ethnic origin other than Norwegian, compared with at least one parent/guardian of Norwegian origin (reference).
dFamily structure defined as everything else but “mother and father of participant living in same household” (ie, other) compared with “mother and father of participant living in same household” (reference).
We found that 8-9-year-old children on average had a match rate of 73%, an omission rate of 27%, and an intrusion rate of 19%, when comparing parental-assisted entries of school lunch data in a WebFR with unobtrusive observations. Mean omission and intrusion rates for different food categories varied greatly. Lower parental educational levels and a non-Norwegian background were associated with less accurate recordings, but this must be interpreted with caution because of the low numbers in these subgroups.
Comparisons With Other Work
Only a few other validation studies of Web-based 24-hour recalls/records for children have used observation during school meals as a reference method. Among these studies are the one on the Automated Self-Administered 24-hour Recall-Kids-2012 (ASA24-Kids-2012) among 9-11-year olds by Diep et al  in the United States, the Food Intake and Physical Activity of School Children (CAAFE) study among 7-10-year olds by Davies et al [ ] in Brazil, the Portuguese Self-Administered Computerised 24-h Dietary Recall (PAC24) study among 7-10-year olds by Carvalho et al [ ] in Portugal, the WebDASC study among 8-11-year olds by Biltoft-Jensen et al [ ] in Denmark, and the study of the modified Self-Administered Children and Infant Nutrition Assessment (SACINA) used among 6-8-year olds by Hunsberger et al [ ] in Sweden.
Our results are not directly comparable with these validation studies, partly because the rates of matches, omissions, and intrusions were not calculated in the same way as they were in our study. Nonetheless, we assert that it is possible to interpret the direction of the findings; in the CAAFE and ASA24-Kids-2012 studies, lower agreement between the recordings in the Web-based assessment tool and observations of school lunch were reported than in our study. The CAAFE study had average rates of 44% matches, 30% omissions, and 26% intrusions , whereas the ASA24-Kids-2012 study had average rates of 37% matches, 35% omissions, and 27% intrusions [ ]. Parental assistance during recordings was encouraged and accomplished for most participants in our study. This was not an option in the CAAFE and ASA24-Kids-2012 studies, although children could ask simple questions to the researchers during recording in these studies. The lack of parental/adult assistance may partly explain differences in the results between studies, and we argue that children at this age need help when recording, which is also proposed elsewhere [ , , - ]. Furthermore, a low parental educational level can somewhat explain the dissimilar results between studies; a low educational level was associated with poorer recordings in both the CAAFE study and our study, where 64% and 11% had a low parental educational level, respectively. In addition, in the CAAFE and ASA24-Kids-2012 studies, the children recorded entries after a 24-hour time lag, whereas in our study, the children were instructed to record their data within the same evening. This difference may explain why the match rate was higher in our study because it is demonstrated that omissions and intrusions in children’s dietary recalls increase as a function of time [ ].
The PAC24 study shows results that are more in line with our results, despite the lack of parental/adult assistance during recording . An explanation could be the broad definition of matches applied in the PAC24 study, in contrast to our study in which matches were defined in a stricter manner.
Because the WebFR is a Norwegian version of the Danish WebDASC, we expected the results to be consistent with the findings from the WebDASC validation study . Surprisingly, Biltoft-Jensen et al [ ] found 82% matches, 3% omissions, and 14% intrusions for total foods and beverages, which are remarkably better than in our study, and their rates of omissions were very low in comparison with our 27% omission rate. This large discrepancy cannot be explained by the fact that their calculations were performed slightly differently than those in our study. Age affects children’s dietary reports [ , ]. Thus, we argue that age may partly explain the differences between the studies because the children were on average a year older in the WebDASC study than those in this study. In addition, we suggest that reactivity may have been a larger problem in the Danish version than in ours because of their interactive observation style; children were instructed to unpack their packed lunches, separate items, open up sandwiches, and place them on a plate before a photograph was taken. In addition, questions were asked regarding food trading and earlier snacking from their packed lunch [ ].
A very high reporting accuracy was reported in the small validation study (n=25) of SACINA by Hunsberger and co-workers; in their study, overall food matches ranged from 86% to 98% . Although children in this study were only to recall 1 lunch meal eaten the previous day, assisted by a dietitian using the Web-based SACINA instrument providing photos and portion estimates, we believe this cannot explain why the accuracy was so contrastingly high compared to other studies.
Baxter et al [, - ] compared 9-year-old children’s dietary reports in the form of traditional recalls (not Web-based) with direct observations of school meals in several studies. In some of these studies, the rates of omissions and intrusions were calculated and presented in a way that is comparable with our study; the results demonstrate that the omission and intrusion rates varied widely, and that reporting accuracy was reduced when the time intervals between eating and reporting increased. Our results are consistent with or better than the majority of the studies by Baxter et al for “same day recalls.”
Only a few studies report the rates of omissions and intrusions for selected food subcategories that are comparable with our findings. Vegetables and sweets were reported as the most often omitted food items in the PAC24 study, whereas beverages were the most commonly intruded item . This is in line with the high omission rates for “vegetables and salads” and “biscuits, buns, waffles, cakes, and candy,” and the high intrusion rate for “beverages, other” found in our study. Nevertheless, Biltoft-Jensen et al [ ] reported remarkably lower omissions for fruits and vegetables than in our study. Once more, we argue that reactivity may have been a large problem in Biltoft-Jensen et al’s study, and may explain the discrepancy between the studies.
To our knowledge, we are the first to report on “coinciding omissions and intrusions,” and by doing so we add important knowledge as to whether the omissions and intrusions represent major errors, and not just slightly imprecise recordings. The food category “spreads” had a high omission rate, and most of the omissions were major errors, not “coinciding omissions and intrusions.” This discovery has already led us to improve the WebFR, by including tailor-made prompts for “spreads.”
Taking the portion sizes of the omitted and intruded food items into account is important because it provides a better understanding of whether these omissions and intrusions are of great concern or not. We observed high omission rates in the food categories “fruit, berries” and “vegetables, salads”; however, the portion sizes of these categories were mostly small in contrast to the portion sizes of omitted “spreads.” Thus, we argue that the omissions of spreads are more troublesome than the omissions of fruits and vegetables in our WebFR.
Lower parental education levels have been associated with a higher degree of misreporting among children in the form of underreporting, or both underreporting and overreporting [- ]. As already described, both our findings and the results of the CAAFE study show that lower parental education was associated with more recording errors [ ]. However, the picture is a bit unclear when searching the literature for associations between ethnic backgrounds and omissions or intrusions in dietary assessment tools for children. Baxter et al [ , ] reported that there were no statistically significant differences in rates with regard to ethnicity. Yet, Baranowski et al [ ] reported that Hispanic children described more problems when using the Food Intake Recording Software System, an early version of the ASA24-Kids-2012, than other children. Our study is coherent with the latter; although we must emphasize that there were a limited number of participants who had both parents of non-Norwegian origin. A possible explanation for our findings may be that having an ethnicity different from the majority is a proxy for having a different food culture and perhaps literary difficulties, which may be a barrier, even though participants can enter any type of food into the WebFR using the open field option. Furthermore, although audio files were included to assist those with lower reading skills, the WebFR still requires knowledge of the Norwegian language, and thus errors during recording may occur more often among persons with language difficulties. Consequently, we suggest that children and/or parents or guardians with language difficulties should be identified and given extra instructions on how to use the WebFR in future studies. They may benefit from direct personal contact with someone from the research crew, to ensure that they understand what to do.
Studies indicate that underreporting among children increases as BMI increases [, , ]. Nevertheless, the odds for a “low match rate” (≤70%) were only slightly higher and not significantly different for overweight/obese than normal weight children in our study. We believe this nonsignificant result may be explained by lack of power.
Strengths and Limitations
The use of direct unobtrusive observations is one of the strengths of this study, because these provide exact information about what is consumed, without affecting the recordings [, ]. Furthermore, we have demonstrated that our observations of home-packed school lunches were satisfactory; the overall consistency between observers for food items was 92%, which is considered sufficient [ , , ], and in line with other IOR assessments conducted during observations of home-packed lunches [ ]. Lower agreement was seen for the observed portion sizes. The combination of thorough training sessions prior to the study start and IOR assessments before and continuously during the study was important, and it provides a premise for high internal validity.
The small number of individuals in some of the subgroups is a limitation of this study, as the preferable adjustment for cluster effects (school level) proved infeasible due to lack of established statistical methods. Hence, the point estimates in the logistic regression analysis should be interpreted with caution.
For practical reasons, observations were restricted to school lunches and to children in 4th grade (8-9 years). Thus, a limitation is that we do not know whether our findings can be extrapolated to other meals or age groups. In addition, our participants had more highly educated parents/guardians and were less overweight or obese, than the average Norwegian population in which 29% have higher education , and 16% of third graders are overweight or obese [ ]. However, the proportion of participants with non-ethnic-Norwegian parents/guardians was 20% (see ), which is higher than the 14% average in the general Norwegian population [ ]. Despite these differences, this study was performed in a similar setting to the one intended for later use, that is, the next national dietary survey among children in Norway. This contributes to its external validity in a positive manner.
We have demonstrated that 8-9-year-old children had a mean match rate of 73% when recording their food intake from school lunch, with parental assistance, in a WebFR. Some children had difficulties recording, but the mean results were better than what have been reported in most validation studies of other Web-based dietary assessment tools among children. The WebFR could be improved further by including additional prompts for high omission rate foods. We suggest that children and their parents/guardians with language difficulties should be given extra support and information about how to use the WebFR in future studies.
The authors thank Susanne Strohmaier for assistance with the statistical analyses.
Conflicts of Interest
Multimedia Appendix 1
Selected screenshots from the Web-based Food Record (WebFR).WMV File (Windows Media Video), 3MB
- McPherson R, Hoelscher D, Alexander M, Scanlon K, Serdula M. Dietary assessment methods among school-aged children: Validity and reliability. Prev Med 2000 Aug;31(2):S11-S33. [CrossRef]
- Thompson FE, Subar AF. Dietary assessment methodology. In: Coulston AM, Boushey CJ, Ferruzzi MG, editors. Nutrition in the Prevention and Treatment of Disease. 3rd edition. San Diego, CA: Academic Press; 2008.
- Thompson FE, Subar AF, Loria CM, Reedy JL, Baranowski T. Need for technological innovation in dietary assessment. J Am Diet Assoc 2010 Jan;110(1):48-51 [FREE Full text] [CrossRef] [Medline]
- Forster H, Fallaize R, Gallagher C, O'Donovan CB, Woolhead C, Walsh MC, et al. Online dietary intake estimation: The Food4Me food frequency questionnaire. J Med Internet Res 2014;16(6):e150 [FREE Full text] [CrossRef] [Medline]
- Christensen SE, Möller E, Bonn SE, Ploner A, Wright A, Sjölander A, et al. Two new meal- and web-based interactive food frequency questionnaires: Validation of energy and macronutrient intake. J Med Internet Res 2013 Jun;15(6):e109 [FREE Full text] [CrossRef] [Medline]
- Adamson AJ, Baranowski T. Developing technological solutions for dietary assessment in children and young people. J Hum Nutr Diet 2014 Jan;27 Suppl 1:1-4. [CrossRef] [Medline]
- Biltoft-Jensen A, Trolle E, Christensen T, Islam N, Andersen LF, Egenfeldt-Nielsen S, et al. WebDASC: A web-based dietary assessment software for 8-11-year-old Danish children. J Hum Nutr Diet 2014 Jan;27 Suppl 1:43-53. [CrossRef] [Medline]
- Baranowski T, Islam N, Douglass D, Dadabhoy H, Beltran A, Baranowski J, et al. Food Intake Recording Software System, version 4 (FIRSSt4): A self-completed 24-h dietary recall for children. J Hum Nutr Diet 2014 Jan;27 Suppl 1:66-71 [FREE Full text] [CrossRef] [Medline]
- Vereecken C, Covents M, Maes L, Moyson T. Formative evaluation of the dietary assessment component of Children's and Adolescents' Nutrition Assessment and Advice on the Web (CANAA-W). J Hum Nutr Diet 2014 Jan;27 Suppl 1:54-65. [CrossRef] [Medline]
- Foster E, Hawkins A, Delve J, Adamson AJ. Reducing the cost of dietary assessment: Self-completed recall and analysis of nutrition for use with children (SCRAN24). J Hum Nutr Diet 2014 Jan;27 Suppl 1:26-35. [CrossRef] [Medline]
- Moore HJ, Hillier FC, Batterham AM, Ells LJ, Summerbell CD. Technology-based dietary assessment: Development of the Synchronised Nutrition and Activity Program (SNAP). J Hum Nutr Diet 2014 Jan;27 Suppl 1:36-42. [CrossRef] [Medline]
- Davies VF, Kupek E, de Assis MA, Natal S, Di Pietro PF, Baranowski T. Validation of a web-based questionnaire to assess the dietary intake of Brazilian children aged 7-10 years. J Hum Nutr Diet 2015 Jan;28 Suppl 1:93-102. [CrossRef] [Medline]
- Livingstone MB, Robson PJ, Wallace JM. Issues in dietary intake assessment of children and adolescents. Br J Nutr 2004 Oct;92 Suppl 2:S213-S222. [Medline]
- Livingstone MB, Robson PJ. Measurement of dietary intake in children. Proc Nutr Soc 2000 May;59(2):279-293. [Medline]
- Simons-Morton BG, Baranowski T. Observation in assessment of children's dietary practices. J Sch Health 1991 May;61(5):204-207. [Medline]
- Baglio ML, Baxter SD, Guinn CH, Thompson WO, Shaffer NM, Frye Francesca HA. Assessment of interobserver reliability in nutrition studies that use direct observation of school meals. J Am Diet Assoc 2004 Sep;104(9):1385-1392 [FREE Full text] [CrossRef] [Medline]
- The Norwegian Directorate of Health. Norkost 3. A nationwide dietary survey among 18-70 year old men and women in Norway-11 [In Norwegian]. Oslo, Norway: The Norwegian Directorate of Health; 2012. URL: http://www.lhl.no/Documents/norkost-3-is-2000%5b1%5d.pdf [accessed 2015-10-07] [WebCite Cache]
- Richter SL, Vandervet LM, Macaskill LA, Salvadori MI, Seabrook JA, Dworatzek PD. Accuracy and reliability of direct observations of home-packed lunches in elementary schools by trained nutrition students. J Acad Nutr Diet 2012 Oct;112(10):1603-1607. [CrossRef] [Medline]
- Cole TJ, Bellizzi MC, Flegal KM, Dietz WH. Establishing a standard definition for child overweight and obesity worldwide: International survey. BMJ 2000 May 6;320(7244):1240-1243 [FREE Full text] [Medline]
- Baxter SD, Thompson WO, Litaker MS, Frye Francesca H A, Guinn CH. Low accuracy and low consistency of fourth-graders' school breakfast and school lunch recalls. J Am Diet Assoc 2002 Mar;102(3):386-395 [FREE Full text] [Medline]
- Firth D. Bias reduction of maximum likelihood estimates. Biometrika 1993;80(1):27-38. [CrossRef]
- von EE, Altman DG, Egger M, Pocock SJ, Gøtzsche PC, Vandenbroucke JP. The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: Guidelines for reporting observational studies. Lancet 2007 Oct 20;370(9596):1453-1457. [CrossRef] [Medline]
- Diep CS, Hingle M, Chen T, Dadabhoy HR, Beltran A, Baranowski J, et al. The Automated Self-Administered 24-Hour Dietary Recall for Children, 2012 Version, for Youth Aged 9 to 11 Years: A validation study. J Acad Nutr Diet 2015 Oct;115(10):1591-1598. [CrossRef] [Medline]
- Carvalho MA, Baranowski T, Foster E, Santos O, Cardoso B, Rito A, et al. Validation of the Portuguese self-administered computerised 24-hour dietary recall among second-, third- and fourth-grade children. J Hum Nutr Diet 2015 Dec;28(6):666-674. [CrossRef] [Medline]
- Biltoft-Jensen A, Bysted A, Trolle E, Christensen T, Knuthsen P, Damsgaard CT, et al. Evaluation of Web-based Dietary Assessment Software for Children: Comparing reported fruit, juice and vegetable intakes with plasma carotenoid concentration and school lunch observations. Br J Nutr 2013 Jul 14;110(1):186-195. [CrossRef] [Medline]
- Hunsberger M, Pena P, Lissner L, Grafström L, Vanaelst B, Börnhorst C, et al. Validity of self-reported lunch recalls in Swedish school children aged 6-8 years. Nutr J 2013;12:129 [FREE Full text] [CrossRef] [Medline]
- Foster E, Hawkins A, Simpson E, Adamson AJ. Developing an interactive portion size assessment system (IPSAS) for use with children. J Hum Nutr Diet 2014 Jan;27 Suppl 1:18-25. [CrossRef] [Medline]
- Baranowski T, Islam N, Baranowski J, Martin S, Beltran A, Dadabhoy H, et al. Comparison of a web-based versus traditional diet recall among children. J Acad Nutr Diet 2012 Apr;112(4):527-532 [FREE Full text] [CrossRef] [Medline]
- Burrows TL, Martin RJ, Collins CE. A systematic review of the validity of dietary assessment methods in children when compared with the method of doubly labeled water. J Am Diet Assoc 2010 Oct;110(10):1501-1510. [CrossRef] [Medline]
- Baxter SD, Thompson WO, Davis HC, Johnson MH. Impact of gender, ethnicity, meal component, and time interval between eating and reporting on accuracy of fourth-graders' self-reports of school lunch. J Am Diet Assoc 1997 Nov;97(11):1293-1298. [CrossRef] [Medline]
- Baxter SD, Smith AF, Litaker MS, Guinn CH, Shaffer NM, Baglio ML, et al. Recency affects reporting accuracy of children's dietary recalls. Ann Epidemiol 2004 Jul;14(6):385-390. [CrossRef] [Medline]
- Baxter SD, Thompson WO, Litaker MS, Guinn CH, Frye Francesca HA, Baglio ML, et al. Accuracy of fourth-graders' dietary recalls of school breakfast and school lunch validated with observations: In-person versus telephone interviews. J Nutr Educ Behav 2003;35(3):124-134 [FREE Full text] [Medline]
- Baxter SD, Guinn CH, Royer JA, Hardin JW, Mackelprang AJ, Smith AF. Accuracy of children's school-breakfast reports and school-lunch reports (in 24-h dietary recalls) differs by retention interval. Eur J Clin Nutr 2009 Dec;63(12):1394-1403 [FREE Full text] [CrossRef] [Medline]
- Baxter SD, Smith AF, Guinn CH, Thompson WO, Litaker MS, Baglio ML, et al. Interview format influences the accuracy of children's dietary recalls validated with observations. Nutr Res 2003 Nov;23(11):1537-1546. [CrossRef]
- Baxter SD, Thompson WO, Smith AF, Litaker MS, Yin Z, Frye Francesca HA, et al. Reverse versus forward order reporting and the accuracy of fourth-graders' recalls of school breakfast and school lunch. Prev Med 2003 May;36(5):601-614. [Medline]
- Lioret S, Touvier M, Balin M, Huybrechts I, Dubuisson C, Dufour A, et al. Characteristics of energy under-reporting in children and adolescents. Br J Nutr 2011 Jun;105(11):1671-1680. [CrossRef] [Medline]
- Rangan AM, Flood VM, Gill TP. Misreporting of energy intake in the 2007 Australian Children's Survey: Identification, characteristics and impact of misreporters. Nutrients 2011 Feb;3(2):186-199 [FREE Full text] [CrossRef] [Medline]
- Murakami K, Miyake Y, Sasaki S, Tanaka K, Arakawa M. Characteristics of under- and over-reporters of energy intake among Japanese children and adolescents: The Ryukyus Child Health Study. Nutrition 2012 May;28(5):532-538. [CrossRef] [Medline]
- Baranowski T, Islam N, Baranowski J, Cullen KW, Myres D, Marsh T, et al. The food intake recording software system is valid among fourth-grade children. J Am Diet Assoc 2002 Mar;102(3):380-385. [Medline]
- Baxter SD, Smith AF, Litaker MS, Guinn CH, Nichols MD, Miller PH, et al. Body mass index, sex, interview protocol, and children's accuracy for reporting kilocalories observed eaten at school meals. J Am Diet Assoc 2006 Oct;106(10):1656-1662 [FREE Full text] [CrossRef] [Medline]
- Fisher JO, Johnson RK, Lindquist C, Birch LL, Goran MI. Influence of body composition on the accuracy of reported energy intake in children. Obes Res 2000 Nov;8(8):597-603. [CrossRef] [Medline]
- Baxter SD, Hardin JW, Smith AF, Royer JA, Guinn CH, Mackelprang AJ. Twenty-four hour dietary recalls by fourth-grade children were not influenced by observations of school meals. J Clin Epidemiol 2009 Aug;62(8):878-885 [FREE Full text] [CrossRef] [Medline]
- Smith AF, Baxter SD, Hardin JW, Guinn CH, Royer JA, Litaker MS. Validation-study conclusions from dietary reports by fourth-grade children observed eating school meals are generalisable to dietary reports by comparable children not observed. Public Health Nutr 2007 Oct;10(10):1057-1066 [FREE Full text] [CrossRef] [Medline]
- Simons-Morton BG, Forthofer R, Huang IW, Baranowski T, Reed DB, Fleishman R. Reliability of direct observation of schoolchildren's consumption of bag lunches. J Am Diet Assoc 1992 Feb;92(2):219-221. [Medline]
- Statistics Norway. Population's Level of Education. Oslo, Norway: Statistics Norway; 2013 Oct 1. URL: http://www.ssb.no/en/utdanning/statistikker/utniv/ [accessed 2015-10-07] [WebCite Cache]
- Norwegian Institute of Public Health. Division of Epidemiology. Child Growth in Norway 2008-2010-2012. Height, Weight and Waist Circumference among 3rd Graders [In Norwegian]. Oslo, Norway: Norwegian Institute Of Public Health; 2014. URL: http://www.fhi.no/dokumenter/b8fd82d8a0.pdf [accessed 2015-10-07] [WebCite Cache]
- Statistics Norway. Immigrants and Norwegian-Born to Immigrant Parents in 13 Municipalities [In Norwegian]. Oslo, Norway: Statistics Norway; 2014. URL: https://www.ssb.no/befolkning/artikler-og-publikasjoner/_attachment/190485?_ts=147aaa8bd38 [accessed 2015-10-07] [WebCite Cache]
|ASA24-Kids-2012: Automated Self-Administered 24-hour Recall-Kids-2012|
|BMI: body mass index|
|CAAFE: the Food Intake and Physical Activity of School Children|
|IOR: interobserver reliability|
|ISO-BMI: age and sex-specific body mass index|
|PAC24: the Portuguese Self-Administered Computerised 24-h Dietary Recall|
|SACINA: Self-Administered Children and Infant Nutrition Assessment|
|WebDASC: Web-based Dietary Assessment Software for Children|
|WebFR: Web-based Food Record|
Edited by G Eysenbach; submitted 13.08.15; peer-reviewed by Å Svensson, V Fernandes Davies; comments to author 03.09.15; revised version received 15.09.15; accepted 02.10.15; published 07.12.15
©Anine Christine Medin, Helene Astrup, Britt Marlene Kåsin, Lene Frost Andersen. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 07.12.2015.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.