Published on in Vol 18, No 6 (2016): Jun

Original Paper

1UCD Institute of Food and Health, University College Dublin, Dublin, Ireland

2Creme Global, Trinity Technology and Enterprise Campus, Grand Canal Quay, Dublin, Ireland

3Human Nutrition Research Centre, Institute of Cellular Medicine, Newcastle University, Newcastle Upon Tyne, United Kingdom

4Hugh Sinclair Unit of Human Nutrition and Institute for Cardiovascular and Metabolic Research, University of Reading, Reading, United Kingdom

5Department of Human Biology, NUTRIM, School for Nutrition and Translational Research in Metabolism, Maastricht University Medical Centre, Maastricht, Netherlands

6Department of Nutrition, Food Science and Physiology, Centre for Nutrition Research, University of Navarra, Pamplona, Spain

7ZIEL Research Center of Nutrition and Food Sciences, Biochemistry Unit, Technische Universität München, München, Germany

8Department of Nutrition and Dietetics, Harokopio University, Athens, Greece

9National Food & Nutrition Institute (IZZ), Warsaw, Poland

10Department of Nutrition, Institute of Basic Medical Sciences, University of Oslo, Oslo, Norway

Corresponding Author:

Lorraine Brennan, PhD

UCD Institute of Food and Health

University College Dublin




Phone: 353 1 716 ext 2811

Fax:353 1 716147


Background: Despite numerous healthy eating campaigns, the prevalence of diets high in saturated fatty acids, sugar, and salt and low in fiber, fruit, and vegetables remains high. With more people than ever accessing the Internet, Web-based dietary assessment instruments have the potential to promote healthier dietary behaviors via personalized dietary advice.

Objective: The objectives of this study were to develop a dietary feedback system for the delivery of consistent personalized dietary advice in a multicenter study and to examine the impact of automating the advice system.

Methods: The development of the dietary feedback system included 4 components: (1) designing a system for categorizing nutritional intakes; (2) creating a method for prioritizing 3 nutrient-related goals for subsequent targeted dietary advice; (3) constructing decision tree algorithms linking data on nutritional intake to feedback messages; and (4) developing personal feedback reports. The system was used manually by researchers to provide personalized nutrition advice based on dietary assessment to 369 participants during the Food4Me randomized controlled trial, with an automated version developed on completion of the study.

Results: Saturated fatty acid, salt, and dietary fiber were most frequently selected as nutrient-related goals across the 7 centers. Average agreement between the manual and automated systems, in selecting 3 nutrient-related goals for personalized dietary advice across the centers, was highest for nutrient-related goals 1 and 2 and lower for goal 3, averaging at 92%, 87%, and 63%, respectively. Complete agreement between the 2 systems for feedback advice message selection averaged at 87% across the centers.

Conclusions: The dietary feedback system was used to deliver personalized dietary advice within a multi-country study. Overall, there was good agreement between the manual and automated feedback systems, giving promise to the use of automated systems for personalizing dietary advice.

Trial Registration: NCT01530139; (Archived by WebCite at

J Med Internet Res 2016;18(6):e150



Diets that are low in saturated fatty acids (SFAs), sugar, and salt and high in fruit, vegetables, and fiber are considered the healthy choice and have been shown to reduce the risk of noncommunicable diseases (NCD) [1-4]. However, despite numerous campaigns and policies to promote healthy eating, NCD burden has continued to rise, globally, over the past decade with increased contribution from nutrition-related risk factors [5]. As a result, there is the need for effective strategies to promote healthy dietary habits and to help consumers to achieve the necessary dietary changes.

Intensive inter-person counseling and interventions have been shown to improve dietary behaviors [6], although the potential feasibility and effectiveness of such methods across large populations is limited by both expense and accessibility [1,7]. Given the global increases in Internet availability, and the increasing utilization of the Internet as a method for delivering behavioral changes [8-11], the use of Web-based dietary assessment tools to provide personalized nutrition (PN) advice (based on dietary intake alone) at an individual level could provide a more cost-effective approach for improving dietary behaviors than current generic dietary advice and inter-person counseling [1,12-14].

Numerous studies have shown that personalized feedback advice is more effective than generic information for changing health behaviors including dietary intake [8,15-20]. Web-based dietary assessment instruments (eg, food frequency questionnaires—FFQs, 24-hour recalls, food diaries) can be the basis for the development of tools with the ability to provide users with personalized or tailored dietary feedback advice based on their self-reported dietary habits [12]. Such dietary feedback advice often illustrates the adequacy of the individual’s nutrient intakes in comparison with recommended intakes, intakes of peers of the same age or gender or previously attained intakes [7,16,19-21].

Although Web-based dietary assessment tools provide a potential starting point for delivering personalized dietary advice to large populations, they are usually 1-way systems that are designed to collect dietary intake data in a cost-effective manner. To limit measurement error and increase data completeness, Web-based dietary assessment tools can be preprogrammed with range and plausibility checks, probing questions and encoded to ensure all questions are answered [22]. Despite programming these check elements, it is essential that dietary assessment systems are coherent, comprehensive, structured, and clear, particularly when a consistent approach for composing and delivering dietary feedback advice is required. Algorithms can be developed to provide feedback in a systematic manner and have been applied in many clinical or health care domains to link personal data to knowledge systems with the aim of developing diagnoses or treatment options. In the same way, by comparing nutrient intakes with dietary recommendations, it is possible to develop algorithms that generate appropriate feedback messages, which can be stored in a message archive [23,24]. Algorithms can also be programmed into dietary assessment tools, facilitating the automatic generation of feedback advice [8]. Web-based dietary assessment tools, which generate automatic feedback advice, have the potential to improve dietary habits across large population groups, while minimizing researcher burden, reducing costs, and saving time.

The objectives of this paper were to report on the development of a Web-based dietary feedback system for the delivery of consistent personalized dietary advice in a multicenter study and to examine the impact of automating the feedback system on dietary advice delivery.

Development of the Feedback System

The dietary feedback system was developed, for manual use by researchers, for the delivery of personalized dietary advice in the pan-European Food4Me Proof-of-Principle (PoP) study. The Food4Me PoP study that aimed to deliver Web-based personalized dietary and physical activity (PA) advice was designed to emulate a real-life Internet-delivered, PN service [25]. As described in detail elsewhere, 1607 participants were randomized to 1 of the 4 groups receiving different levels of PN advice: Level 0 (control group) receiving conventional, non-PN advice; Level 1 PN advice based on dietary intake and PA data alone; Level 2 PN advice based on dietary intake, PA, and phenotypic data; Level 3 PN advice based on dietary intake, PA, phenotypic, and genotypic data [25]. The aim was to recruit a total of 1540 participants (220 participants per center) to allow for a potential 20% dropout rate, planned using a priori power calculation [25]. Power calculations were conducted in Minitab and based on glucose and omega-3 fatty acid concentrations within European adult populations [25]. The study had ethical approval from the corresponding committees of all participating centers (Germany, Greece, Ireland, the Netherlands, Poland, Spain, and the United Kingdom) and was conducted from August 2012 to March 2014. The Food4Me trial was registered as a randomized control trial (NCT01530139) at

The dietary feedback system, described in this paper, was used to deliver PN advice to participants randomized to receive Level 1 PN only (n=414). All participants of the Food4Me study received dietary feedback advice via email without face-to-face contact with researchers [25]. Thus, the feedback system was designed and developed to ensure that delivery of personalized dietary feedback was consistent across all the 7 countries. The key stages in the development of the dietary feedback system are illustrated in Figure 1 and described in detail in the following section.

Figure 1. Stages in the development of the dietary feedback system.
View this figure
Development of a Gradation System for Coding of Nutrient Intakes

Dietary intake data were collected throughout the Food4Me study using the recently developed and validated Web-based Food4Me FFQ [26,27]. Previous evidence has shown the Food4Me FFQ to have good agreement with the EPIC-Norfolk FFQ and moderate agreement with a 4-day weighed food record for the assessment of both nutrient and food group intakes, rendering it a useful tool for ranking individuals based on nutrient and food group intakes [26,27]. The Institute of Medicine (IOM) dietary reference intakes were used as a basis for developing a gradation system to categorize nutrient intakes automatically after completion of the Web-based Food4Me FFQ by participants [28,29]. Institute of Medicine reference values were used because (1) they were the most up-to-date values available and (2) recommendations vary across Europe with a need to find standardized reference intakes. Nutrient gradations were calculated based on IOM estimated average requirements (EARs) and tolerable upper intake levels (upper limits—ULs). Lower cutoff values were calculated as the EAR minus 2 standard deviations. Institute of Medicine–recommended daily allowance or World Health Organization recommendations were used to calculate the gradations when IOM EARs were not available [30,31]. The gradations (EAR, UL, and lower cutoff point) for each nutrient were preprogrammed into the Web-based Food4Me FFQ by Creme Global (Dublin, Ireland) facilitating a visual ranking system for nutrient intakes (Multimedia Appendix 1). Where applicable, gradations incorporated IOM recommendations for age and gender. Color coding and labeling of nutrient intakes were integrated into the Web-based Food4Me FFQ nutritional outputs to enable rapid visual assessment of the nutrient intake as “very low,” “low,” “recommended,” “high,” or “very high,” which were labeled and color-coded as red, amber, green, amber, and red, respectively.

Development of a System for Prioritizing Dietary Feedback Advice

Seventeen nutrients were selected for inclusion in the dietary feedback system viz “protein,” “carbohydrate,” “total fat,” “MUFA,” “PUFA,” “SFA,” “salt,” “omega 3,” “fiber,” “calcium,” “iron,” “vitamin A,” “folate,” “thiamin,” “riboflavin,” “vitamin B12,” and “vitamin C”. In addition, based on patient-centered models for facilitating dietary change, which emphasize that patients should focus on only a few goals at a time [25], 3 nutrient-related goals (target nutrients) were selected for particular emphasis in the feedback report. To select these 3 nutrient-related goals for subsequent dietary advice, the 17 nutrients were split into 3 groups (Multimedia Appendix 2). Group 1 consisted of all the “fat-related” nutrients: SFA, omega-3, total fat, monounsaturated fatty acid, and polyunsaturated fatty acid (PUFA). Group 2 included folate, dietary fiber, salt, vitamin B12, riboflavin, thiamin, protein, and carbohydrate. Group 3 consisted of calcium, iron, vitamin C, and vitamin A. A ranking system was embedded in the methodology for identifying target nutrients, so that nutrients at the top of each group received highest priority (ie, nutrients of higher public health concern). Generally, the highest priority nutrient, flagged “red” from each group was chosen as the nutrient-related goal, if no “red” nutrients were available, those flagged “amber” were chosen. In cases when only 2 nutrients were flagged “red” or “amber,” a third ‘green’ (recommended) nutrient was given with a positive message to maintain nutrient intake.

Development of Decision Trees to Link Nutritional Intake Data to Feedback Messages

Sixteen dietary decision trees were manually developed to provide dietary feedback advice (Level 1 PN). All decision trees were developed to link nutrient intakes generated automatically by the Web-based Food4Me FFQ to a library of feedback messages. With the exception of the decision trees for carbohydrate, unsaturated fat, dietary fiber, and salt, all decision trees were generated to account for nutritional intake from both foods and supplements. Decision trees were based on the IOM gradation system with branches developed for “low,” “recommended,” and “high” intakes of each nutrient. The decision trees for SFA and salt were more complex and involved identifying the 2 main food groups contributing to the intake of the nutrient thereby enabling further personalization of dietary advice. For example, if a participant’s diet were identified as high in SFAs, researchers would follow the decision tree to identify the 2 highest contributing food groups of 7 potential candidate groups.

All decision trees included branches linking to individual feedback messages. In total, the archive of dietary feedback messages consisted of 92 messages, which were developed using a variety of reputable sources including British Dietetic Association, Food Safety Authority Ireland, and British Nutrition Foundation [32-36]. For each decision tree, the feedback messages consisted of practical food-based tips to improve nutrient intake. An example of the decision tree for vitamin C and corresponding feedback messages are presented in Figure 2, with further examples of SFA and salt given in Multimedia Appendix 3. Protocols for the feedback system were standardized across the 7 countries, and all feedback messages were translated into the language of each recruitment country. To assess the utility and applicability of the decision trees and corresponding feedback messages, during the Food4Me study, a record was kept by researchers at all the 7 centers. These entries were amalgamated and categorized by nutrient.

Figure 2. Vitamin C decision tree and corresponding feedback messages.
View this figure
Development of Feedback Reports for Dietary Feedback Advice

Template feedback reports were developed to ensure that dietary advice was delivered in a consistent format across all countries. Standard operating procedures were compiled and nutritionists or dietitians in all centers received training in composing the reports. All feedback reports began with a short message of encouragement from the researcher, which also highlighted the main areas for improvement. This message was followed by 4 key sections:

  • 1: How your diet compares to recommendations
  • 2: Your physical characteristics
  • 3: Your nutrient profile
  • 4: Your personalized nutrition advice

Section 1 included a table comparing the participant’s average number of portions of 5 food groups: “fruit and vegetables,” “whole grains,” “dairy products,” “oily fish,” and “red meat” with guideline amounts. The guideline amounts were derived from amalgamating the national dietary advice in each of the 7 centers to create 1 common set of Food4Me dietary guidelines. Section 2 detailed the participant’s height and weight and compared the participant’s body mass index (BMI) and PA with recommendations. Section 3 graphically illustrated the participant’s intake of each of the 17 nutrients as “good, no change recommended,” “improvement recommended,” and “improvement strongly recommended” on a gradation scale comparing intakes with IOM recommendations. Section 4 detailed personalized behavioral goals including a table listing the participant’s 3 nutrient-related goals, along with dietary sources and the feedback message(s) from the corresponding decision tree. A list of all the food groups and nutrients for which personalized feedback was given can be found in Multimedia Appendix 4, with an example feedback report presented in Multimedia Appendix 5.

Automation of the Dietary Feedback

Nutrient intake analysis was automated for use in the Food4Me study, with the nutrient-related goals and feedback messages derived manually (using the priority system and decision trees) by Food4Me researchers using their own judgment to overrule the feedback system when appropriate. After completion of the Food4Me intervention study, the feedback system was automated by Creme Global (Dublin, Ireland). Therefore, a comparison of the automated and manual-based approaches was possible. The system was automated by firstly capturing the manual decision process for selecting priority nutrients, based on nutrient intake analysis, in a computer algorithm (written in the PHP programming language). Second, the decision trees for selection of feedback messages were encoded in table-like data structures and stored in a relational database (MySQL). Additional computer algorithms for traversing these data structures were then developed (also using PHP), which enabled feedback messages to be generated automatically from nutrient intake analysis.


The first part of the analysis examines the manual use of the feedback system during the Food4Me PoP study. Descriptive statistics were computed to describe the general characteristics of participants across each of the 7 centers using general linear model analysis with least significant difference post hoc. Nutrient-related goal selection frequency was examined across each of the 7 countries. All statistical analyses were conducted using IBM SPSS Statistics version 20, and P<.05 was considered statistically significant.

The second part of the analysis compares the dietary feedback provided by researchers during the Food4Me PoP study with feedback generated automatically by the computerized algorithms and messaging system for participants in Level 1 of the intervention. The level of agreement between the manual (researcher) and automated systems was assessed for both nutrient-related goal selection and feedback advice using baseline data for Level 1 participants, as illustrated in Figure 3. Agreement between the manual and automated systems, in selecting the 3 nutrient-related goals, was investigated by comparing each individual nutrient-related goal (1, 2, and 3) selected by both systems (manual vs automated) for each participant in the 7 countries. To evaluate the agreement for nutrient-related goal selection, we also examined whether the same 3 nutrients were selected as the 3 nutrient-related goals (in any order 1, 2, or 3) by both systems.

As outlined in Figure 3, to compare if the same feedback advice message(s) were derived from each decision tree by both systems, we selected only participants given the same nutrient-related goals (1, 2, and 3) by both the manual and automated systems and cross-compared the messages given by both systems for each nutrient-related goal (1, 2, and 3). Agreement was categorized as either “complete agreement,” “complete disagreement,” or “partial agreement” (data not shown). Overall agreement for feedback advice message was computed by summing the number of participants categorized into the 3 agreement groups for each nutrient-related goal across the countries.

Figure 3. Steps taken to compare the level of agreement between the manual and automated systems for nutrient-related goals and feedback advice. The agreement between the 2 systems in selecting the 3 nutrient-related goals was assessed for each participant. Nutrient-related goal agreement between the 2 systems was also evaluated for all 3 nutrient-related goals in random order. Where there was agreement between the 2 systems for nutrient goal selection, agreement in the feedback advice message(s) selected was then assessed between the two systems. In Figure 3, a represents complete agreement, 100% match between advice messages given by the manual and automated systems; b represents complete disagreement, 0% match between the advice messages given by the 2 systems; c represents partial agreement, some agreement, and some disagreement between the advice messages given by the 2 systems, partial agreement was only applicable to the SFA and salt decision trees as multiple advice messages were given in the feedback to participants.
View this figure

The Study Population

A total of 414 participants, from the Level 1 group, across the 7 countries were available for inclusion in the analysis. After removing dropouts immediately after being randomized (n=41) and participants who did not have nutrient-related goals recorded at baseline (n=4), 369 participants were included in the analysis (Germany, n=52; Greece, n=51; Ireland, n=55; Netherlands, n=56; Poland, n=49; Spain, n=54; and the United Kingdom, n=52). There were no significant differences in age for men and women; however, self-reported BMI was significantly lower for women compared with men (P=.002). When comparing demographic characteristics across all the countries, no significant differences were observed for bodyweight or BMI; however, significant differences were observed for age (P=.02) and height (P<.001), as summarized in Table 1.

Table 1. Demographic characteristics of the study population in total and across countriesa.
CountryDemographic characteristics

Gender—female (%)Age (y)Height (m)Weight (kg)BMI (kg/m2)
All countries (n=369)57.539.7 ± 12.91.71 ± 0.1074.19 ± 16.6125.21 ± 5.02
Germany (n=52)51.942.6 ± 14.9b,d1.75 ± 0.09b73.48 ± 13.5523.95 ± 3.62
Greece (n=51)58.838.1 ± 10.5b,c1.69 ± 0.10c76.02 ± 19.3726.46 ± 6.52
Ireland (n=55)56.438.9 ± 12.0b,c,d1.70 ± 0.10c72.90 ± 16.1325.07 ± 4.76
Netherlands (n=56)50.043.0 ± 15.4d1.76 ± 0.10b77.70 ± 16.2925.08 ± 4.37
Poland (n=49)71.436.2 ± 11.1c1.69 ± 0.07c71.46 ± 16.6124.83 ± 4.99
Spain (n=54)51.941.8 ± 11.0b,d,e1.69 ± 0.10c74.83 ± 16.9426.09 ± 4.98
United Kingdom (n=52)63.537.1 ± 13.4c,e1.70 ± 0.76c72.60 ± 17.0524.94 ± 5.36

aValues are means ± standard deviations.

b,c,d,eMeans with different superscripts denote significant differences (analysis of variance with least significant difference post hoc)

Manual Use and Efficacy of the Dietary Feedback System

The top 3 most frequently selected nutrient-related goals (1, 2, and 3) across each of the countries are summarized in Table 2. Saturated fatty acid was most frequently selected as nutrient-related goal 1 across all centers except for Spain where PUFA was most frequently selected as nutrient-related goal 1 in feedback given to 35% (19/54) of participants. Salt was most frequently selected as nutrient-related goal 2 in all 7 centers, with the exception of Germany, where both folate and salt were most frequently selected. As summarized in Table 2, greater variation was observed across the 7 countries for nutrient-related goal 3. As a result of the prioritization process, salt was most frequently selected as both nutrient-related goals 2 and 3, in both Spain and the United Kingdom.

The number of times each of the 17 nutrients was selected as a nutrient-related goal (1, 2, or 3) across each of the 7 countries is summarized in Table 3. Overall, SFA, salt, and dietary fiber were the top 3 most frequently targeted nutrient-related goals provided to 72%, 72%, and 39% of participants, respectively. Saturated fatty acid was the most frequently chosen nutrient-related goal overall for Germany, Ireland, and Poland. Salt was the most frequently selected nutrient-related goal overall for the Netherlands, Spain, and the United Kingdom, and in Greece, both dietary fiber and salt were most frequently selected.

A summary of the key issues encountered during the Food4Me study regarding the use and applicability of the decision trees and feedback advice messages is summarized in Table 4. Most of these issues were related to the SFA and salt decision trees, owing to their more complex design.

Table 2. Top 3 most frequently selected nutrient-related goals (1, 2, and 3) at baseline in the 7 countriesa.
CountryNutrient-related goal 1Nutrient-related goal 2Nutrient-related goal 3
Germany (n=52)SFAb(54%)Folate or salt (29%)Carbohydrate or calcium (15%)
Greece (n=51)SFA (41%)Salt (39%)Dietary fiber (41%)
Ireland (n=55)SFA (56%)Salt (53%)Dietary fiber (15%)
Netherlands (n=56)SFA (39%)Salt (38%)Dietary fiber 21%)
Poland (n=49)SFA (53%)Salt (39%)Calcium (19%)
Spain (n=54)PUFAc(35%)Salt (35%)Salt (24%)
United Kingdom (n=52)SFA (48%)Salt (46%)Salt (21%)

aPercentages indicate the percentage of participants who received the nutrient as the nutrient-related goal.

bSFA: saturated fatty acid.

cPUFA: polyunsaturated fatty acid.

Table 3. Number of participants having nutrient-related goals (1, 2, or 3) at baseline across the 7 countriesa.
United Kingdom
Total fat0532381
Dietary fiber14312019142420
Vitamin C2132421
Vitamin A0122421
Vitamin B121000001
Total number of nutrient-related goalse156153155168147161c153

aRiboflavin and thiamin were not given as nutrient-related goals in any of the 7 centers and are not presented in Table 3.

bSFA: saturated fatty acid.

cMUFA: monounsaturated fatty acid.

dPUFA: polyunsaturated fatty acid.

eTotal number of nutrient-related goals in each country, calculated as number of participants (N) multiplied by 3. In Ireland, Spain, and the United Kingdom, total number of nutrient-related goals is less than this calculation as several participants were only given 2 nutrient-related goals in Ireland (n=10), Spain (n=1), the and United Kingdom (n=3).

Table 4. Issues with dietary decision trees or feedback messages.
Issue with decision trees or feedback messagesDecision trees affected
Algorithm did not capture all food groups contributing to the nutrient intake.SFAa
Sometimes, the feedback messages are not specific to the participant’s diet, that is, the participant’s main sources of the nutrient may not have been identified in the feedback message.SFA, salt, total fat
Feedback messages are repeated for different food groups.Salt
Messages are not always relevant for vegetarians.SFA, salt, omega 3

aSFA: saturated fatty acid.

Comparison of Advice Generated by Manual and Automated Systems

The level of agreement between the manual and automated systems for the selection of the 3 nutrient-related goals in all the countries is summarized in Table 5. Good agreement was observed between both methods in all the countries for nutrient-related goals 1 and 2 with average agreement of 92% and 87%, respectively. Agreement between the 2 methods ranged from 100% (the United Kingdom) to 82% (Greece) for nutrient-related goal 1 and from 98% (Spain) to 80% (the Netherlands) for nutrient-related goal 2. Lower agreement was observed between the manual and automated systems for nutrient-related goal 3, mean 63% across centers. For nutrient-related goal 3 selection, agreement was highest for Spain (85%) and lowest for Greece (45%). Agreement between the 2 systems for all 3 nutrient-related goals in random order ranged from 83% (Spain) to 47% (Greece), averaging 66% across all countries.

Having selected the priority nutrients, the next stage was the selection of feedback messages. The level of agreement between the manual and automated systems for the feedback advice messages is summarized in Table 6. Complete agreement between the 2 systems for feedback advice ranged from 90% (Greece and Ireland) to 82% (Germany and Poland), with an average of 87% across the 7 countries.

Table 5. Level of agreement between manual and automated systems for baseline nutrient-related goals selection in all 7 countriesa.
CountryNutrient-related goal 1Nutrient-related goal 2Nutrient-related goal 3Random order agreementb

n (%)n (%)n (%)n (%)
Germany (n=52)50 (96)47 (90)33 (63)36 (69)
Greece (n=51)41 (82)42 (82)23 (45)24 (47)
Ireland (n=55)c48 (87)45 (82)31 (56)33 (60)
Netherlands (n=56)50 (89)45 (80)34 (61)36 (64)
Poland (n=49)44 (90)43 (88)35 (71)37 (76)
Spain (n=54)c53 (98)53 (98)46 (85)45 (83)
United Kingdom (n=52)c52 (100)48 (92)31 (60)33 (63)

aLevel of agreement between the manual and automated systems for each individual nutrient-related goal and for all 3 nutrient-related goals in random order.

bAgreement for all 3 nutrient-related goals in random order derived by calculating the agreement when the same 3 nutrients were selected as the 3 nutrient-related goals by both systems regardless if they were given as nutrient-related goal 1, nutrient-related goal 2, or nutrient-related goal 3.

cLower agreement observed for nutrient related-goal 3 and overall agreement as a minority some participants received only 2 nutrient-related goals from the manual system in Ireland (n=10), Spain (n=1), and the United Kingdom (n=3).

Table 6. Level of agreement between manual and automated systems for baseline feedback advice message selection in all 7 countries.
CountryComplete agreementaComplete disagreementbAgreement plus disagreementc

n (%)n (%)n (%)
Germany (n=130)107 (82)3 (2)20 (16)
Greece (n=107)95 (90)0 (0)12 (11)
Ireland (n=124)112 (90)3 (3)9 (7)
Netherlands (n=129)114 (88)0 (0)15 (12)
Poland (n=122)100 (82)1 (1)21 (17)
Spain (n=152)134 (88)5 (3)13 (9)
United Kingdom (n=131)113 (86)3 (2)15 (11)

aComplete agreement, 100% match between advice messages given by the manual and automated systems.

bComplete disagreement, 0% match between the advice messages given by the 2 systems.

cPartial agreement, some agreement, and some disagreement between the advice messages given by the 2 systems, partial agreement was only applicable to the SFA and salt decision trees as multiple advice messages were given in the feedback to participants.

Principal Findings

This paper presents a novel system for providing consistent personalized dietary advice, automatically generated from dietary intake data submitted via the Internet by participants in a multi-country study. To our knowledge, this paper is one of the first to describe, in detail, the steps involved in developing a dietary feedback system to translate food and nutrient intake data into automatically generated personalized feedback advice and to compare the automatically generated advice with advice manually provided by nutrition researchers.

Development of the feedback system consisted of a 4-step process: designing a gradation system to categorize nutrient intakes, creating a priority system to enable 3 nutrients to be further selected as nutrient-related goals for particular emphasis in the feedback report, constructing decision trees to link nutrient intakes to feedback advice, and finally, developing feedback report templates. The findings of this paper demonstrate that automation of the feedback system is feasible and more superior compared with manual use of the system by result of removal of human error.

The feedback system described in this paper is unique as it was developed to provide personalized feedback on intakes of 5 food groups and 17 nutrients, 3 of which were prioritized and selected as “nutrient-related goals” for subsequent targeted dietary advice. As the feedback system was developed to deliver relevant PN advice for all European adults, it was important that the system considered all nutrients for which changes in intake would be recommended to improve the health of significant proportions of the adult population. To address this requirement, the system that we developed included key food groups and a large number of nutrients so that feedback could be generated and personalized for a complete nutritional profile, rather than limited personalized information on one or several nutrient(s) as previously applied in other tailoring studies [20,37,38]. Providing extensive information, guaranteed that the dietary feedback would include personalized advice, which could be translated into food groups and nutrients of importance for a healthy diet (including fruit and vegetables, SFA, salt, fiber, whole grains, red meat and dairy products, omega-3 fatty acids, vitamin C, and iron) [39]. Several studies have briefly alluded to the application of a similar multistep process, linking nutrient intake data to feedback messages [1,13,21,24,37,40,41]. However, most of these studies have focused mostly on tailoring advice for selected nutrients or food groups only (eg, SFA, fruit and vegetable intakes) [1,6,13,16,21,24,37,42-44], with the exception of the advice generated for adolescents by Maes et al [40] which included fiber, vitamin C, calcium, iron, and fat and the reports developed by Kannan et al [23] that provided personalized feedback for 13 nutrients.

In addition to providing personalized food-based messages, feedback was also displayed graphically in reports by comparison of the individual’s nutrient intakes with dietary recommendations, similar to many other studies [1,6,15,16,42,45]. Nutrient-related goal selection was relatively consistent across the countries, with SFA and salt being the most frequently assigned nutrient-related goals targeted, although some North–South gradient differences are observed. This result was expected given the prevalence of diets high in SFAs and salt across Europe [39,46,47].

Because the feedback system had not been tested previously in a multicenter cohort, the decision trees were used manually for the delivery of dietary advice within the intervention study. Manual use of the decision trees within the study combined with rigorous recording of the issues identified by the nutritionists or dietitians delivering the intervention facilitated evaluation of the advice and identification of aspects of the system, which could be improved. The decision trees were developed to comprise essential components such as contribution from supplements and food groups (SFA and salt decision trees only). To guarantee feedback advice is fully relevant and appropriately personalized at an individual level, suggested improvements include further expanding the decision trees to integrate specific food items and incorporate additional food groups (SFA and salt decision trees), for example, branches for consumption of commonly consumed food items high in SFAs (eg, ice cream and quiche) could be added to the SFA and total fat decision trees.

Automated technologies are increasingly used across many disciplines and have been shown to be as effective as human (face-to-face) systems for 3-month weight loss and exercise interventions [8,48]. Furthermore, Emerencia et al [49] recently developed an automated personalized system for schizophrenia patients providing advice similar to that given by clinicians. Although several studies have generated computer-automated dietary feedback, to our knowledge, this paper is among the first to examine whether it is possible to fully automate a dietary feedback system designed to provide PN advice to enhance healthy eating. We observed high agreement between the manual and automated systems, averaging at 92% and 87%, for nutrient-related goals 1 and 2, respectively. Good agreement was also observed for feedback advice selection averaging 87%. However, although our results show potential for future automation of dietary feedback tools, some disagreement was observed between the manual and automated systems.

Nutrient goal selection disagreement averaged 8%, 13%, and 37% across the 7 centers for nutrient-related goals 1, 2, and 3, respectively. Reasons for this disagreement included: researchers overruling the priority system (eg, if they thought it was inappropriate to give one of the nutrient-related goals selected using the priority system); researchers misreporting the nutrient-related goal in the report (eg, the correct nutrient-related goal was given in the report but misreported in the Internet); and researcher error in selecting the nutrient-related goal (eg, giving an adequate nutrient by mistake). The level of disagreement between the 2 systems was much greater for nutrient-related goal 3, especially for Greece, primarily, a result of overruling the priority system. Overruling did not always mean that the original selection was inappropriate, only that the researcher thought another option was more relevant. For feedback advice selection, complete disagreement was negligible (2%), but partial agreement was higher (12%), and was confined to the SFA and salt decision tree messages only. Owing to their more complex design, the feedback advice for these nutrients included 2-4 individual messages, which were more prone to researcher misreporting, error or overruling the decision trees to give additional messages. There was also a degree of ambiguity in the prioritization rules and the SFA and salt decision tree rules. For example, in relation to prioritization, no rule was set for selecting a positive third nutrient when only 2 risk nutrients were identified, thereby increasing the level of disagreement for nutrient goal selection between the manual and automated systems. Similarly, for both the SFA and salt decision trees, no rules were established for which food groups should be selected when several contributed equally to the nutrient intake.

Although we observed some disagreement between the manual and automated systems, overall, the agreement between the systems was excellent. Much of the disagreement we observed between the 2 systems would be reduced by automation of the feedback system, which would eliminate researcher error and researcher misreporting, and further clarification of the decision rules for both the prioritization process and SFA and salt decision trees. Furthermore, despite researchers overruling the priority nutrients, both the systems selected nutrients of high priority.

Technological innovations have evolved the delivery and management of health care through the increasing use of mobile phone health and computerized health care apps [50,51], with the personalized health field developing rapidly [52]. Consequently, our feedback system has potential to become a cost-effective approach to improve dietary behaviors at an individual level, which could be delivered by public or private health care providers. The present system was designed to provide advice on usual intake rather than actual intake; the challenge in the future will be to provide advice to consumers at the point of purchase or consumption.

Strengths and Limitations

The strengths of this study include the adequate sample size and multicenter recruitment in 7 European countries. In addition, to facilitate behavior change, the system was designed to incorporate numerous behavioral change techniques including goal setting, action planning, and barrier identification [11,25]. However, the system did not take into account motivation for change, and this could be a potential limitation. Feedback that is personalized to current dietary intake and stage of change may be considered as being more motivational and could have greater efficacy in promoting sustained dietary changes than advice personalized on dietary intake data only [14]. Another potential limitation of the study is that the feedback system was tested in a group of individuals who volunteered to receive PN advice and may therefore not be applicable to other population groups. However, the dietary and anthropometric characteristics of participants in the Food4Me study were broadly similar to those of the wider population of European adults so that the study tested the utility of the system to select appropriate feedback for the likely most common dietary changes. In addition, we did not follow-up with participants regarding their opinions about the advice they received, for example, if they thought it was relevant to them, appropriate, or useful, and this would have been extremely useful information to have captured for the future progression of PN.


We developed a Web-based dietary feedback system that was capable of delivering consistent personalized dietary advice to adult European participants in the multicenter Food4Me study. Outcomes from comparison of the manual and automated feedback systems provide confidence that such an automated dietary feedback system can be developed and implemented across multiple countries with the potential to contribute to scalable and cost-effective interventions to improving dietary behaviors and health across large populations.


The Food4Me study was supported by the European Commission under the Food, Agriculture, Fisheries and Biotechnology Theme of the 7th Framework Programme for Research and Technological Development, grant number 265494.

Authors' Contributions

The authors’ responsibilities were as follows: HF, MCW, LB, and EG carried out the analyses and drafted the manuscript; CCM, HF, CBO, CW, CG, CFMM, ALM, RF, SNC, RSC, SK, LT, CPL, MG, AS, IMFG, MCW, ERG, LB, and JCM contributed to the standardized operating procedures for the study; EJD and RO contributed to the design and technical aspect of the Web-based server for automating the feedback system; CCM, SNC, RSC, HF, CW, CBO, CFMM, AM, RF, SK,CM, LT, CPL, MG, AS, MCW, and JCM conducted the intervention; YM, IT, CAD, ERG, LB, JAL, JAM, WHMS, HD, MJG, and JCM contributed to the research design of the Food4Me study. All authors contributed to a critical review of the manuscript during the writing process and approved the final version to be published.

Conflicts of Interest

None declared.

Multimedia Appendix 1

Nutrient gradations.

PDF File (Adobe PDF File), 597KB

Multimedia Appendix 2

Level 1 priority nutrients and rationale.

PDF File (Adobe PDF File), 294KB

Multimedia Appendix 3

Example decision trees and messages.

PDF File (Adobe PDF File), 244KB

Multimedia Appendix 4

Nutrients and food groups for which personalized feedback was given.

PDF File (Adobe PDF File), 250KB

Multimedia Appendix 5

Example feedback report.

PDF File (Adobe PDF File), 447KB

  1. Wright JL, Sherriff JL, Dhaliwal SS, Mamo John C L. Tailored, iterative, printed dietary feedback is as effective as group education in improving dietary behaviours: results from a randomised control trial in middle-aged adults with cardiovascular risk factors. Int J Behav Nutr Phys Act 2011;8:43 [FREE Full text] [CrossRef] [Medline]
  2. Desroches S, Lapointe A, Ratté S, Gravel K, Légaré F, Turcotte S. Interventions to enhance adherence to dietary advice for preventing and managing chronic diseases in adults. Cochrane Database Syst Rev 2013;2:CD008722. [CrossRef] [Medline]
  3. McCarthy M. Higher sugar intake linked to raised risk of cardiovascular mortality, study finds. BMJ 2014;348:g1352. [Medline]
  4. Dhurandhar NV, Thomas D. The link between dietary sugar intake and cardiovascular disease mortality: an unresolved question. JAMA 2015 Mar 3;313(9):959-960. [CrossRef] [Medline]
  5. Swinburn B, Sacks G, Vandevijvere S, Kumanyika S, Lobstein T, Neal B, INFORMAS. INFORMAS (International Network for Food and Obesity/non-communicable diseases Research, Monitoring and Action Support): overview and key principles. Obes Rev 2013 Oct;14 Suppl 1:1-12. [CrossRef] [Medline]
  6. Block G, Block T, Wakimoto P, Block CH. Demonstration of an E-mailed worksite nutrition intervention program. Prev Chronic Dis 2004 Oct;1(4):A06 [FREE Full text] [Medline]
  7. Probst YC, Tapsell LC. Overview of computerized dietary assessment programs for research and practice in nutrition education. J Nutr Educ Behav 2005;37(1):20-26. [Medline]
  8. Tate DF, Jackvony EH, Wing RR. A randomized trial comparing human e-mail counseling, computer-automated tailored counseling, and no counseling in an Internet weight loss program. Arch Intern Med 2006;166(15):1620-1625. [CrossRef] [Medline]
  9. Juan W, Gerrior S, Hiza H. MyPyramid Tracker assesses food consumption, physical activity, and energy balance status interactively. J Nutr Educ Behav 2006;38(6 Suppl):S155-S157. [CrossRef] [Medline]
  10. van GL, van EP, Boon B, Borsboom G, Visscher T, Oenema A. Results from an online computer-tailored weight management intervention for overweight adults: randomized controlled trial. J Med Internet Res 2012;14(2):e44 [FREE Full text] [CrossRef] [Medline]
  11. Celis-Morales C, Lara J, Mathers JC. Personalising nutritional guidance for more effective behaviour change. Proc Nutr Soc 2015 May;74(2):130-138. [CrossRef] [Medline]
  12. Anton SD, LeBlanc E, Allen HR, Karabetian C, Sacks F, Bray G, et al. Use of a computerized tracking system to monitor and provide feedback on dietary goals for calorie-restricted diets: the POUNDS LOST study. J Diabetes Sci Technol 2012 Sep;6(5):1216-1225 [FREE Full text] [Medline]
  13. Springvloet L, Lechner L, Oenema A. Planned development and evaluation protocol of two versions of a web-based computer-tailored nutrition education intervention aimed at adults, including cognitive and environmental feedback. BMC Public Health 2014;14:47 [FREE Full text] [CrossRef] [Medline]
  14. Forster H, Walsh MC, Gibney MJ, Brennan L, Gibney ER. Personalised nutrition: the role of new dietary assessment methods. Proc Nutr Soc 2016 Feb;75(1):96-105. [CrossRef] [Medline]
  15. Ezendam Nicole P M, Brug J, Oenema A. Evaluation of the Web-based computer-tailored FATaintPHAT intervention to promote energy balance among adolescents: results from a school cluster randomized trial. Arch Pediatr Adolesc Med 2012 Mar;166(3):248-255. [CrossRef] [Medline]
  16. Kroeze W, Oenema A, Campbell M, Brug J. The efficacy of Web-based and print-delivered computer-tailored interventions to reduce fat intake: results of a randomized, controlled trial. J Nutr Educ Behav 2008;40(4):226-236. [CrossRef] [Medline]
  17. Lustria Mia Liza A, Noar SM, Cortese J, Van Stee Stephanie K, Glueckauf RL, Lee J. A meta-analysis of web-delivered tailored health behavior change interventions. J Health Commun 2013;18(9):1039-1069. [CrossRef] [Medline]
  18. Moore TJ, Alsabeeh N, Apovian CM, Murphy MC, Coffman GA, Cullum-Dugan D, et al. Weight, blood pressure, and dietary benefits after 12 months of a Web-based Nutrition Education Program (DASH for health): longitudinal observational study. J Med Internet Res 2008;10(4):e52 [FREE Full text] [CrossRef] [Medline]
  19. Patrick K, Calfas KJ, Norman GJ, Rosenberg D, Zabinski MF, Sallis JF, et al. Outcomes of a 12-month web-based intervention for overweight and obese men. Ann Behav Med 2011 Dec;42(3):391-401. [CrossRef] [Medline]
  20. Sternfeld B, Block C, Quesenberry CP, Block TJ, Husson G, Norris JC, et al. Improving diet and physical activity with ALIVE: a worksite randomized trial. Am J Prev Med 2009 Jun;36(6):475-483. [CrossRef] [Medline]
  21. Oenema A, Tan F, Brug J. Short-term efficacy of a web-based computer-tailored nutrition intervention: main effects and mediators. Ann Behav Med 2005 Feb;29(1):54-63. [CrossRef] [Medline]
  22. Illner A, Freisling H, Boeing H, Huybrechts I, Crispim SP, Slimani N. Review and evaluation of innovative technologies for measuring diet in nutritional epidemiology. Int J Epidemiol 2012 Aug;41(4):1187-1203 [FREE Full text] [CrossRef] [Medline]
  23. Kannan S, Schulz A, Israel B, Ayra I, Weir S, Dvonch TJ, et al. A community-based participatory approach to personalized, computer-generated nutrition feedback reports: the healthy environments partnership. Prog Community Health Partnersh 2008;2(1):41-53 [FREE Full text] [CrossRef] [Medline]
  24. Brug J. Dutch research into the development and impact of computer-tailored nutrition education. Eur J Clin Nutr 1999 May;53 Suppl 2:S78-S82. [Medline]
  25. Celis-Morales C, Livingstone KM, Marsaux Cyril F M, Forster H, O'Donovan CB, Woolhead C, Matthews John N S, Saris Wim H M, et al. Design and baseline characteristics of the Food4Me study: a web-based randomised controlled trial of personalised nutrition in seven European countries. Genes Nutr 2015 Jan;10(1):450 [FREE Full text] [CrossRef] [Medline]
  26. Forster H, Fallaize R, Gallagher C, O'Donovan CB, Woolhead C, Walsh MC, et al. Online dietary intake estimation: the Food4Me food frequency questionnaire. J Med Internet Res 2014;16(6):e150 [FREE Full text] [CrossRef] [Medline]
  27. Fallaize R, Forster H, Macready AL, Walsh MC, Mathers JC, Brennan L, et al. Online dietary intake estimation: reproducibility and validity of the Food4Me food frequency questionnaire against a 4-day weighed food record. J Med Internet Res 2014;16(8):e190 [FREE Full text] [CrossRef] [Medline]
  28. Institute of Medicine. Dietary Reference Intakes for energy, Carbohydrate, Fibre, Fat, Fatty acids, Cholesterol, Protein, and Amino acids. 2005   URL: [accessed 2016-01-25] [WebCite Cache]
  29. Institute of Medicine. Dietary Reference Intakes Tables and Applications. 2011   URL: [accessed 2012-01-18] [WebCite Cache]
  30. World Health Organisation. Protein and Amino acid requirements in Human Nutrition. 2007   URL: [accessed 2012-01-18] [WebCite Cache]
  31. World Health Organisation. Food and Agriculture Organisation of the United Nations (FAO) report of an expert consultation on fats and fatty acids in human nutrition. 2010.   URL: [accessed 2012-01-20] [WebCite Cache]
  32. Public Health Agency. Enjoy Healthy Eating. 2012.   URL: [accessed 2016-01-25] [WebCite Cache]
  33. British Dietetic Association. Food Fact Sheets. 2013.   URL: [accessed 2016-01-25] [WebCite Cache]
  34. British Dietetic Association. Weight Wise Eating well what?s a healthy diet?. 2013.   URL: [accessed 2016-01-25] [WebCite Cache]
  35. British Nutrition Foundation. Healthy eating. 2013.   URL: [accessed 2016-01-25] [WebCite Cache]
  36. Food Saftey Authority Ireland. Scientific Recommendations for Healthy Eating Guidelines in Ireland. 2011.   URL: [accessed 2016-01-25] [WebCite Cache]
  37. Ambeba EJ, Ye L, Sereika SM, Styn MA, Acharya SD, Sevick MA, et al. The use of mHealth to deliver tailored messages reduces reported energy and fat intake. J Cardiovasc Nurs 2015;30(1):35-43. [CrossRef] [Medline]
  38. Springvloet L, Lechner L, de VH, Candel Math J J M, Oenema A. Short- and medium-term efficacy of a Web-based computer-tailored nutrition education intervention for adults including cognitive and environmental feedback: randomized controlled trial. J Med Internet Res 2015;17(1):e23 [FREE Full text] [CrossRef] [Medline]
  39. Capacci S, Mazzocchi M, Shankar B, Macias JB, Verbeke W, Pérez-Cueto Federico J A, et al. Policies to promote healthy eating in Europe: a structured review of policies and their effectiveness. Nutr Rev 2012 Mar;70(3):188-200. [CrossRef] [Medline]
  40. Maes L, Cook TL, Ottovaere C, Matthijs C, Moreno LA, Kersting M, et al. Pilot evaluation of the HELENA (Healthy Lifestyle in Europe by Nutrition in Adolescence) Food-O-Meter, a computer-tailored nutrition advice for adolescents: a study in six European cities. Public Health Nutr 2011 Jul;14(7):1292-1302. [CrossRef] [Medline]
  41. Kerr DA, Pollard CM, Howat P, Delp EJ, Pickering M, Kerr KR, et al. Connecting Health and Technology (CHAT): protocol of a randomized controlled trial to improve nutrition behaviours using mobile devices and tailored text messaging in young adults. BMC Public Health 2012;12:477 [FREE Full text] [CrossRef] [Medline]
  42. Robroek Suzan J W, Bredt FJ, Burdorf A. The (cost-)effectiveness of an individually tailored long-term worksite health promotion programme on physical activity and nutrition: design of a pragmatic cluster randomised controlled trial. BMC Public Health 2007;7:259 [FREE Full text] [CrossRef] [Medline]
  43. Smeets T, Kremers S P J, Brug J, de VH. Effects of tailored feedback on multiple health behaviors. Ann Behav Med 2007 Apr;33(2):117-123. [CrossRef] [Medline]
  44. Raats MM, Sparks P, Geekie MA, Shepherd R. The effects of providing personalized dietary feedback. A semi-computerized approach. Patient Educ Couns 1999 Jun;37(2):177-189. [Medline]
  45. Oenema A, Brug J. Feedback strategies to raise awareness of personal dietary intake: results of a randomized controlled trial. Prev Med 2003 Apr;36(4):429-439. [Medline]
  46. Brown IJ, Tzoulaki I, Candeias V, Elliott P. Salt intakes around the world: implications for public health. Int J Epidemiol 2009 Jun;38(3):791-813 [FREE Full text] [CrossRef] [Medline]
  47. Linseisen J, Welch AA, Ocké M, Amiano P, Agnoli C, Ferrari P, van der Schouw Y T, et al. Dietary fat intake in the European Prospective Investigation into Cancer and Nutrition: results from the 24-h dietary recalls. Eur J Clin Nutr 2009 Nov;63 Suppl 4:S61-S80. [CrossRef] [Medline]
  48. King AC, Hekler EB, Castro CM, Buman MP, Marcus BH, Friedman RH, et al. Exercise advice by humans versus computers: maintenance effects at 18 months. Health Psychol 2014 Feb;33(2):192-196. [CrossRef] [Medline]
  49. Emerencia A, van der Krieke Lian, Sytema S, Petkov N, Aiello M. Generating personalized advice for schizophrenia patients. Artif Intell Med 2013 May;58(1):23-36. [CrossRef] [Medline]
  50. Meier CA, Fitzgerald MC, Smith JM. eHealth: extending, enhancing, and evolving health care. Annu Rev Biomed Eng 2013;15:359-382. [CrossRef] [Medline]
  51. Weinstein RS, Lopez AM, Joseph BA, Erps KA, Holcomb M, Barker GP, et al. Telemedicine, telehealth, and mobile health applications that work: opportunities and barriers. Am J Med 2014 Mar;127(3):183-187. [CrossRef] [Medline]
  52. Clifton L, Clifton DA, Pimentel Marco A F, Watkinson PJ, Tarassenko L. Gaussian processes for personalized e-health monitoring with wearable sensors. IEEE Trans Biomed Eng 2013 Jan;60(1):193-197. [CrossRef] [Medline]

BMI: body mass index
EAR: estimated average requirement
FFQ: food frequency questionnaire
IOM: Institute of Medicine
NCD: noncommunicable diseases
PA: physical activity
PN: personalized nutrition
PoP: Proof-of-Principle
PUFA: polyunsaturated fatty acids
SFA: saturated fatty acid
UL: upper limit

Edited by G Eysenbach; submitted 15.02.16; peer-reviewed by A Minihane, J de Vries; comments to author 10.03.16; revised version received 01.04.16; accepted 04.04.16; published 30.06.16


©Hannah Forster, Marianne C Walsh, Clare B O'Donovan, Clara Woolhead, Caroline McGirr, E.J. Daly, Richard O'Riordan, Carlos Celis-Morales, Rosalind Fallaize, Anna L Macready, Cyril F M Marsaux, Santiago Navas-Carretero, Rodrigo San-Cristobal, Silvia Kolossa, Kai Hartwig, Christina Mavrogianni, Lydia Tsirigoti, Christina P Lambrinou, Magdalena Godlewska, Agnieszka Surwiłło, Ingrid Merethe Fange Gjelstad, Christian A Drevon, Yannis Manios, Iwona Traczyk, J Alfredo Martinez, Wim H M Saris, Hannelore Daniel, Julie A Lovegrove, John C Mathers, Michael J Gibney, Eileen R Gibney, Lorraine Brennan. Originally published in the Journal of Medical Internet Research (, 30.06.2016.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on, as well as this copyright and license information must be included.