Published on 28.04.15 in Vol 17, No 4 (2015): April
Using a 3D Virtual Supermarket to Measure Food Purchase Behavior: A Validation Study
Background: There is increasing recognition that supermarkets are an important environment for health-promoting interventions such as fiscal food policies or front-of-pack nutrition labeling. However, due to the complexities of undertaking such research in the real world, well-designed randomized controlled trials on these kinds of interventions are lacking. The Virtual Supermarket is a 3-dimensional computerized research environment designed to enable experimental studies in a supermarket setting without the complexity or costs normally associated with undertaking such research.
Objective: The primary objective was to validate the Virtual Supermarket by comparing virtual and real-life food purchasing behavior. A secondary objective was to obtain participant feedback on perceived sense of “presence” (the subjective experience of being in one place or environment even if physically located in another) in the Virtual Supermarket.
Methods: Eligible main household shoppers (New Zealand adults aged ≥18 years) were asked to conduct 3 shopping occasions in the Virtual Supermarket over 3 consecutive weeks, complete the validated Presence Questionnaire Items Stems, and collect their real supermarket grocery till receipts for that same period. Proportional expenditure (NZ$) and the proportion of products purchased over 18 major food groups were compared between the virtual and real supermarkets. Data were analyzed using repeated measures mixed models.
Results: A total of 123 participants consented to take part in the study. In total, 69.9% (86/123) completed 1 shop in the Virtual Supermarket, 64.2% (79/123) completed 2 shops, 60.2% (74/123) completed 3 shops, and 48.8% (60/123) returned their real supermarket till receipts. The 4 food groups with the highest relative expenditures were the same for the virtual and real supermarkets: fresh fruit and vegetables (virtual estimate: 14.3%; real: 17.4%), bread and bakery (virtual: 10.0%; real: 8.2%), dairy (virtual: 19.1%; real: 12.6%), and meat and fish (virtual: 16.5%; real: 16.8%). Significant differences in proportional expenditures were observed for 6 food groups, with largest differences (virtual – real) for dairy (in expenditure 6.5%, P<.001; in items 2.2%, P=.04) and fresh fruit and vegetables (in expenditure: –3.1%, P=.04; in items: 5.9%, P=.002). There was no trend of overspending in the Virtual Supermarket and participants experienced a medium-to-high presence (88%, 73/83 scored medium; 8%, 7/83 scored high).
Conclusions: Shopping patterns in the Virtual Supermarket were comparable to those in real life. Overall, the Virtual Supermarket is a valid tool to measure food purchasing behavior. Nevertheless, it is important to improve the functionality of some food categories, in particular fruit and vegetables and dairy. The results of this validation will assist in making further improvements to the software and with optimization of the internal and external validity of this innovative methodology.
J Med Internet Res 2015;17(4):e107
- virtual reality;
- user-computer interface;
- software validation;
- nutrition policy;
- public health
Supportive environments are essential for people to make healthier food choices and there is a growing call to implement structural interventions such as fiscal policies and front-of-pack labeling to create a healthier food environment [- ]. Supermarkets form a key setting for health-promoting interventions [ , ] because they have a central position within the food chain [ , ] and they are where people buy most of their food (87% of households in New Zealand buy foods from supermarkets weekly or more often [ ]). Although structural interventions in the retail setting have clear potential, high-quality evidence on their effectiveness is limited [ , ]. Evidence from randomized controlled trials (RCTs) is particularly lacking. Although examples of high-quality supermarket trials exist [ - ], experimental studies have mostly been conducted in settings with a limited number of food choices and most lacked objective outcome measures [ ].
The main reason for the dearth of high-quality supermarket trials is that they are complex and costly to conduct. This is especially true for RCTs (eg, it would be problematic to raise prices on soft drinks [simulating a tax] in some supermarkets but not in others). Furthermore, opposition and extensive lobbying from the food industry against several proposed intervention strategies is another important reason for the lack of experimental studies .
In order to find a solution to the complexities of undertaking supermarket intervention research, we developed a research tool that can be used to study the effects of interventions in a virtual reality setting: the Virtual Supermarket (see). Virtual environments are computer-generated models, which participants can experience and interact with intuitively in real time [ ]. Gaming technology is used to simulate a real supermarket shopping experience where study participants purchase virtual food items. Photographs of real foods are used to compose virtual food products and prices are displayed on virtual shelf labels. Currently, 2 versions of this Virtual Supermarket exist: the original Dutch version [ ] and a New Zealand version (developed using a similar methodology) that is the subject of this study. Other examples of virtual supermarkets can be found in the scientific literature; however, to our best knowledge, these have different functionalities [ ], are developed for other purposes (eg, rehabilitation) [ ], or have not published study results yet [ ].
To obtain valid study results, it is important that the Virtual Supermarket simulates a real supermarket as closely as possible. Here, one of the most important traits is the level of presence people experience . Presence is defined as “a psychological state of ‘being there’ mediated by an environment that engages our senses, captures our attention, and fosters our active involvement” [ ] and reflects “the subjective experience of being in one place or environment, even if one is physically located in another” [ ]. Factors that are hypothesized to contribute to a sense of presence are (1) involvement, (2) sensory fidelity, (3) adaptation/immersion, and (4) interface quality factors. Second, a key question is whether peoples’ virtual behavior reflects their real-life behavior accurately. Studies using the Dutch Virtual Supermarket have already shown some promising results [ , ]. However, to date, validation of the software has been solely based on self-report (questionnaire) data. Therefore, the aim of this study was to validate the NZ Virtual Supermarket by comparing virtual and real-life food purchasing behavior using objective outcome measures and also to evaluate the level of presence that people experience. Results will inform further development of the software and guide the development of future experiments using the Virtual Supermarket.
The New Zealand Virtual Supermarket
The Virtual Supermarket was developed in Unity (a game development system) and represents a 3-dimensional (3D) computer simulation of a real shopping experience (). More details about the software can be found elsewhere [ , ]. The Dutch Virtual Supermarket is available online [ ] (the NZ version is currently undergoing modification).
The NZ Virtual Supermarket was designed using the Auckland branch of a market leader as a model. NZ food prices and product range were obtained from a 2011 field survey of packaged foods available for sale in Auckland supermarkets  and online supermarkets. Foods were categorized into 18 food groups and further divided into 91 categories, 141 subcategories, and 31 sub-subcategories (eg, dairy—cheese—hard cheese—high salt cheese). The Virtual Supermarket includes a representative number of products in each food category, with at least 1 product in each subcategory, adding up to a total of 1445 unique products ( ) (an average NZ supermarket offers approximately 9000 nutritionally unique food products). Within each category, we selected the top-selling products using the Australian Grocery Guide annual report 2010 sales data [ ]. In addition, we ensured sufficient variety of special food items such as organic, diet, or gluten-free varieties and included home or discount brands where possible.
In the Virtual Supermarket, participants can navigate around the store with a shopping trolley using their computer cursor keys and select (or remove) groceries with a mouse click. There is an option to view a list of selected groceries, including the price and total amount of money spent. Cash registers are located at the end of the supermarket where participants can check out and complete the study. The software underwent some testing by our study team prior to this validation study, but no formal usability testing was performed.
|Food group and category||Products, n|
|Wine (including cask wine, champagne, dessert wine, red, rose wine, sparkling, special releases, white wine, other liquor)||57|
|Beer and cider||15|
|Bread and bakery|
|Rolls, bagels, and flat breads||21|
|Fresh bread (in-store bakery)||4|
|Sweet treats and cakes||28|
|Cakes, muffins, and pastry||26|
|Baking and cooking|
|Herbs and spices||40|
|Bread crumbs and coating||4|
|Beverages (including coffee, plain; cordial bases; electrolyte drink; energy drink; fruit and vegetable juices; hot drink mixes; ice tea drinks; soft and flavored drinks; tea, plain; water)||115|
|Cereal and cereal products|
|Bran and breakfast cereals||27|
|Yogurt/ yogurt drinks||20|
|Deli and chilled foods|
|Fresh pasta, sauces, soups, and pesto||4|
|Antipasto and continental meats||12|
|Fresh meals and sandwiches||3|
|Edible oils and oil emulsions|
|Butter and margarine||11|
|Fish and fish products|
|Fish with pastry||1|
|Fresh fruit and vegetables|
|Fruit and vegetables other|
|Fruit (including dried, dried fruit and nut mixes, frozen fruit, fruit bars, fruit in juice/syrup, other)||40|
|Vegetables (including baked beans, corn [can], legumes [can], tomatoes [can], canned [other], dried legumes, frozen potato products, frozen vegetables)||61|
|Jam and spreads||11|
|Nuts and seeds||22|
|Fresh meat and seafood, meat products|
|Beef and veal (unprocessed)||5|
|Sausages and small good||5|
|Other desserts (including frozen juice and ice blocks, fruit crumbles, sponges/pudding, fruit pies, jelly, pavlova/meringues, sorbets, other desserts)||21|
|Sauces, spreads and seasonings|
|Peanut butter and sweet spreads||9|
|Relished and pickles||12|
|Crisps and snacks||32|
Study participants were asked to shop on 3 occasions in the Virtual Supermarket over 3 consecutive weeks and to collect their real supermarket grocery till receipts for that same period. Virtual and real shops were conducted in the same weeks. Three shopping occasions were included to provide a representative picture of usual food purchases and to test learning effects. Participants were instructed to conduct their weekly virtual shop prior to their real-life shopping trips. This order was chosen because it is unlikely that people adapt their real-life purchases to their choices made in the Virtual Supermarket; however, when real shops are conducted first, it is possible that people could use their real till receipts as a basis for their food purchases in the Virtual Supermarket. The study design and procedures were approved by the University of Auckland Human Participants Ethics Committee (Nov 7, 2012; ID8691).
Participants were recruited from the general NZ population using advertisements placed in supermarkets, a research participant recruitment website (researchstudies.co.nz), local newspapers, mailing lists and via Māori (indigenous New Zealanders) and Pacific networks within the University. Inclusion criteria were adult (≥18 years), main household shopper, able to communicate in English, access to a computer and Internet, and have an email address. Exclusion criteria included another person in the household already participating in the study.
Participants completed the study at home or on any computer with an Internet connection. Following registration and consent, an email was sent with study instructions and a URL (link to Web address) to download the Virtual Supermarket. In addition to a hard copy illustrated manual, we recorded an instructional video on the Virtual Supermarket. For all 3 virtual shopping occasions, participants were instructed to purchase food for their household for the coming week as they planned to do in real life. It was emphasized that they should buy all food groceries they would need for their household in the coming week, including food they were planning to buy for a special occasion (eg, for a birthday). Likewise, participants were instructed not to buy foods in the Virtual Supermarket if they were not planning to buy these in real life (ie, because they had a sufficient stock). The shopping budget in the Virtual Supermarket was set to the same level as participants’ real-life household food shopping budget for that week. After completion of each virtual shop, participants were asked to collect their real supermarket till receipts for that week (a reminder message appeared on their computer screen). Participants were instructed to collect all real-life food shop till receipts, including from supermarkets, speciality stores, convenience stores, and markets, but not from fast food, restaurants, or takeaways. Along with the till receipts, participants completed a questionnaire about their shopping habits that week (eg, “In the past week, did you make any special food purchases that you would not have normally; for example, because there was a sale/promotion or because of a special occasion?”) and asking how many till receipts were missing. Participants received email and short message service (SMS) text message reminders throughout the study to remind them to complete their virtual shops and collect their till receipts. Participants who completed all 3 virtual shops and returned their grocery till receipts received a NZ $20 supermarket voucher.
The primary outcome of this study was the proportional expenditures (NZ$) on each food group (18 food groups, see) for both the Virtual Supermarket and real supermarket (eg, percentage of total expenditures on vegetables). We measured proportional expenditure instead of absolute expenditures because we were interested in shopping patterns (eg, did participants buy similar quantities of milk and bread) which are better reflected by proportional expenditures (absolute supermarket prices tend to vary substantially over time). Secondly, we also looked at the number of products purchased in each food category and compared the Virtual Supermarket and real supermarket proportions (eg, percentage of all purchased products that are vegetables). Product purchases in the Virtual Supermarket were measured by till receipts automatically generated at the end of each shop. Real-life food purchases were measured by grocery till receipts.
Secondary outcomes included self-reported level of presence measured by the Presence Questionnaire Items Stems (version 4.0) . This questionnaire includes a total of 29 items that are scored on a 7-point Likert scale. Examples of questions include “How responsive was the Virtual Supermarket to actions that you initiated (or performed)?” and “How much did your experiences in the Virtual Supermarket seem consistent with your real world experiences?” In addition, we measured potential effect modifiers and background variables including:
- Capability or tendency to be involved or immersed. People can experience different levels of involvement both in virtual tasks and in common activities (focus). This concept was measured with the items that relate to the factor “focus” (7 questions) from the Immersive Tendency Questionnaire Item Stems [ ] (ie, “how mentally alert do you feel at the present time?”)
- Participant characteristics including age, gender, household size, ethnicity, responsibility for household grocery shopping, highest educational qualification, work status, household income, supermarket shopping habits, computer use, and Internet access.
We aimed to recruit 96 participants. This sample size was estimated to provide 85% power at 5% level of significance (2-sided) to detect a minimal absolute difference of 1% in percentage of food products or expenditure on each food category between weekly virtual and real supermarket purchases. This assumed a standard deviation of 3% based on data from previous research  and an estimated 20% loss to follow-up rate. Because this was a validation study, the sample size was not adjusted for multiple comparisons.
Comparing Virtual and Real Supermarkets
Data from the virtual and real-life grocery till receipts were entered into SPSS version 20 (IBM Corp, Armonk, NY, USA) and coded within the matching 18 food groups. Data were then imported into SAS version 9.2 (SAS Institute Inc, Cary, NC, USA) for further analysis. For each weekly shopping visit, proportional expenditure (NZ$) and the proportion of products purchased were calculated for each food group. First, alignment of the virtual and the real purchases was described using line graphs displaying mean purchases for all food groups for all participants with a least 1 valid virtual shop (raw data). Second, the resemblance between the virtual and real supermarket was tested using repeated measures mixed models. Both supermarket type and shopping visit were fitted as fixed effects. Their interaction was tested and dropped if it was not statistically significant. A random effect at participant level was also fitted to account for correlation between repeated visits on the same participant, and attrition over time assuming the data were missing at random. Model-adjusted means were estimated for the virtual and real supermarkets, and mean differences were tested. Analyses were conducted on the total sample (ie, including all participants who completed at least 1 virtual shop, N=86), followed by complete-case analysis. This second phase included participants who completed all 3 virtual shops and returned grocery till receipts for all 3 weeks. Also, the till receipts had to be valid and we excluded those who reported having failed to return the majority of their receipts (n=8). To determine missing till receipts, we asked about the number of shops participants conducted during the week, whether these were small or bigger shopping episodes, and how many till receipts they collected. If participants reported having conducted a large shop, this till receipt had to be available for the participant’s data to be included in analysis. If participants did multiple smaller shops, at least 60% of receipts had to be available. This resulted in a total of 52 participants (60%, 52/86 for the complete-case analysis).
After each virtual shopping occasion, participants were asked to complete the presence and focus questionnaires. Because there was no difference in these scores over time, the mean total presence and focus score over all completed shops was derived for each participant (typical value). These mean values were divided into 3 groups (low/medium/high) for descriptive analysis. Linear mixed models were used to test the predicted effect of focus on presence (continuous score). Sensitivity analysis was conducted to test whether participants with a higher presence score performed better in the Virtual Supermarket within the complete cases by selecting a subset of participants with a high sense of presence using the median score (median 100) as a cut-off point.
A total of 309 participants registered for this study via phone or email. The first 135 participants were phoned to check their eligibility; 4 were not eligible because they were unavailable during the study period. The remainder were sent the baseline questionnaire and consent forms, with 123 participants enrolled in the study. Of these, 86 (69.9%, 86/123) participants completed shop 1, 79 (64.2%, 79/123) completed shop 2, 74 (60.2%, 74/123) completed shop 3, and 60 (48.8%, 60/123) returned their till receipts (see).
shows the characteristics of the participants who completed at least 1 virtual shop (N=86), completed all 3 virtual shops and returned valid till receipts (N=52), and completed all 3 virtual shops and returned valid till receipts plus had a high presence score (N=25). Results show that there were no clear differences in these sets of participants, except that participants in group 3 (high presence) were somewhat younger (mean 34.0, SD 8.5 years compared to mean 38.7, SD 12.3 years for all 86 participants), and included fewer retired persons and more students.
|Participant characteristics||All participantsa N=86||Complete casesb n=52||High presencec n=25|
|Age (years), mean (SD)||38.7 (12.3)||38.0 (11.2)||34.0 (8.5)|
|Gender (female), n (%)||66 (76.7)||41 (79)||20 (80)|
|Household size, n (%)|
|1 person||9 (10.5)||7 (14)||3 (12)|
|2 persons||32 (37.2)||19 (37)||10 (40)|
|3 persons||15 (17.4)||9 (17)||6 (24)|
|≥4 persons||28 (32.6)||16 (31)||6 (16)|
|Missing||2 (2.3)||1 (2)||—|
|Ethnicity, n (%)|
|NZ European||45 (52.3)||25 (48)||12 (48)|
|Māori||3 (3.5)||2 (4)||1 (4)|
|Pacific Islander||3 (3.5)||1 (2)||1 (4)|
|Chinese||6 (7.0)||4 (8)||1 (4)|
|Other (Australian, English, etc)||29 (33.7)||20 (38)||10 (36)|
|Responsibility for household grocery shopping, n (%)|
|Mainly responsible||51 (59.3)||32 (62)||14 (56)|
|Mostly responsible||17 (19.8)||7 (14)||3 (12)|
|Shared responsibility||17 (19.8)||13 (25)||8 (32)|
|Highest educational qualification, n (%)|
|None||4 (4.7)||2 (4)||—|
|Secondary school||11 (12.8)||5 (10)||3 (12)|
|Undergraduate||26 (30.2)||19 (37)||9 (36)|
|Postgraduate||34 (39.5)||20 (39)||11 (44)|
|Other||4 (4.7)||3 (6)||1 (4)|
|Declined answer||7 (8.2)||3 (6)||1 (4)|
|Work status, n (%)|
|Full time paid work||34 (39.5)||22 (42)||12 (48)|
|Part time paid work||14 (16.3)||9 (17)||4 (16)|
|Retired||6 (7.0)||3 (6)||0 (0)|
|Student||11 (12.8)||6 (12)||4 (16)|
|Homemaker||11 (12.8)||7 (14)||4 (16)|
|Beneficiary||2 (2.3)||0 (0)||0 (0)|
|Other||7 (8.2)||5 (10)||1 (4)|
|Total household income before taxd(NZ $), n (%)|
|<$20,000||5 (5.8)||4 (8)||2 (8)|
|$20,001-$40,000||11 (12.8)||7 (13)||3 (12)|
|$40,001-$60,000||11 (12.8)||5 (10)||1 (4)|
|$60,001-$80,000||16 (18.6)||9 (17)||7 (28)|
|≥$80,001||39 (45.4)||20 (39)||11 (44)|
|Don’t know/declined answer||4 (4.7)||2 (4)||1 (4)|
|Food purchases at supermarket, n (%)|
|All food||9 (10.5)||6 (12)||4 (16)|
|Most food||62 (72.1)||36 (70)||17 (68)|
|Half of food||12 (14.0)||9 (17)||3 (12)|
|Some food||2 (2.4)||1 (2)||1 (4)|
|Grocery shopping frequency, n (%)|
|< Once a week||10 (11.6)||5 (10)||2 (8)|
|Once a week||31 (36.0)||16 (31)||8 (32)|
|Twice a week||29 (33.7)||22 (42)||10 (40)|
|≥3 times a week||15 (17.5)||9 (17)||5 (20)|
|Frequency of using a computer, n (%)|
|Daily||83 (96.5)||51 (98)||24 (96)|
|Several days per week||2 (2.3)||1 (2)||1 (4)|
|Internet at home (yes), n (%)||85 (98.8)||51 (98)||24 (96)|
|Type of operating system used for this study, n (%)|
|Windows XP||11 (12.8)||6 (12)||4 (16)|
|Windows 7||41 (47.7)||26 (50)||11 (44)|
|Windows (don’t know version)||20 (23.3)||11 (21)||7 (28)|
|MacOS X 10.5/10.6||5 (5.8)||3 (6)||1 (4)|
|MacOS X 10.7/10.8||6 (7.0)||5 (10)||1 (4)|
|MacOS X (don’t know version)||2 (2.3)||1 (2)||1 (4)|
a. Includes all participants who completed at least 1 shop in the Virtual Supermarket.
b. Includes all participants who completed all 3 shops in the Virtual Supermarket and returned valid till receipts.
c. Includes participants who completed all 3 shops in the Virtual Supermarket and returned valid till receipts and had a high presence score.
d. Median annual income in New Zealand from wages and salaries was NZ $43,888 in June 2013 (per person)
and show the proportion of money spent / proportion of items purchased for each of the 18 food groups, respectively, for both the virtual and the real supermarket and for each of the 3 shopping occasions (n=86). Results suggest that the purchasing patterns across the food groups were comparable. The 4 food groups with the highest relative expenditures were the same for the virtual and real supermarkets (all 3 shops): fruit and vegetables (14.3% Virtual Supermarket and 17.4% real supermarket), bread and bakery (10.0% Virtual Supermarket and 8.2% real supermarket), dairy (19.1% Virtual Supermarket and 12.6% real supermarket), and meat and fish (16.5% Virtual Supermarket and 16.8% real supermarket) ( ). Overall, total mean expenditures in the Virtual Supermarket were NZ $84.96 (SD 46.88 compared to NZ $125.15 (SD 74.15) in the real supermarket. On average, participants set their budget in the Virtual Supermarket at NZ $121.19 (SD 65.01) and spent 71.4% (SD 25.6) of this budget.
|Food category||% Total expenditures||% Total number of items|
|Means||Difference estimate||P||Means||Difference estimate||P|
|Virtual estimatea||Real estimate||Virtual estimate||Real estimate|
|Bread and bakery||10.0||8.15||1.89||.02||11.7||10.3||1.44||.08|
|Baking and cooking||1.10||0.56||0.54||.16||0.98||0.85||0.39||.17|
|Cereal and cereal products||5.64||7.14||–1.50||.06||5.46||7.58||–2.13||.01|
|Deli and chilled foods||0.31||0.07||0.25||.15||0.28||0.07||0.21||.29|
|Edible oils and emulsions||3.28||3.16||0.12||.84||2.46||2.35||0.11||.81|
|Fish and fish products||2.82||2.44||0.38||.42||2.88||2.65||0.23||.69|
|Fruit and vegetables fresh||14.3||17.4||–3.12||.04||28.9||22.9||5.94||.002|
|Fruit and vegetables other||5.46||7.36||–1.90||.009||6.57||8.64||–2.07||.02|
|Fresh meat, meat products, fresh fish||16.5||16.8||–0.31||.85||9.64||9.77||–0.13||.89|
|Sauces, spreads, and seasonings||3.38||4.56||–1.18||.05||3.29||4.79||–1.50||.01|
a Results from repeated measures mixed models fitted to evaluate the percentage of price (outcome 1) and percentage of items (outcome 2), respectively, with supermarket type (virtual or real) and shop (1, 2, or 3) as fixed effects, registration number as random effect, and shop as repeated effect.
Difference Between Virtual and Real Purchases
shows the estimated means and difference for the virtual and real purchases for all 18 food categories and separately as a proportion of total expenditures (outcome 1) and proportion of total items (outcome 2) for the complete cases (n=52). Results for the total sample (N=86) are shown in and show comparable outcomes to the results presented subsequently. Mixed models revealed that the effect of shopping occasion (shop 1, 2, or 3) was only significant for the convenience foods category. This means that the similarity between virtual and real purchases was stable over time and, therefore, the results are presented at an aggregate level. Six food categories showed significant differences in purchases between the virtual and real shop, including bread and bakery (difference estimate 1.9%, P=.02); dairy (6.5%, P<.001); fruit and vegetables, fresh (3.1%, P=.04); fruit and vegetables, other (1.9%, P=.009); snack foods (2.6%, P<.001); and eggs (1.6%, P=.01). Similar results were observed for difference in proportion of items purchased.
Focus and Presence
Previous research indicated that people with a higher focus might perceive stronger feelings of presence in virtual environments. Outcomes of the linear mixed models confirmed that focus was strongly positively associated with perceived presence in the Virtual Supermarket (estimate 1.05, P<.001).
With regard to presence, results showed that a large majority of participants (≥95% across the 3 shops) reported a medium-to-high feeling of presence in the Virtual Supermarket (). Participants scored particularly high for the subscales adaptation/immersion (ie, perceiving oneself to be enveloped by, included in, and interacting with an environment) and interface quality. According to theory, we would expect that people with a higher presence would perform better in the Virtual Supermarket compared to people with a lower perception of presence. Sensitivity analysis on a subset of participants with a high presence score (n=25) revealed greater similarity in proportional expenditures between virtual and real purchasing behavior. The total number of categories with significant differences between virtual and real purchases reduced from 6 to 4 in this subset of participants (the categories “fruit and vegetables” and “eggs” showed no longer significant differences).
|Presence score and subscales for 3 groups||Mean (SD)||n (%)|
|Total presence (range 29-203)||104 (22.0)|
|Low (0-67)||3 (4)|
|Medium (68-134)||73 (88)|
|High (≥134)||7 (8)|
|Subscale 1: involvement (range 12-84)||40.0 (11.1)|
|Low (0-28)||12 (15)|
|Medium (29-56)||67 (81)|
|High (≥57)||4 (5)|
|Subscale 2: sensory fidelity (range 6-42)||18.0 (4.8)|
|Low (0-14)||19 (23)|
|Medium (15-28)||62 (75)|
|High (≥29)||2 (2)|
|Subscale 3: adaptation/immersion (range 8-56)||31.2 (6.9)|
|Low (0-18)||1 (1)|
|Medium (19-37)||64 (77)|
|High (≥38)||18 (22)|
|Subscale 4: interface quality (range 3-21)||14.9 (3.2)|
|Low (0-7)||1 (1)|
|Medium (8-14)||33 (40)|
|High (≥15)||49 (59)|
|Focus (range 7-49)||29.7 (5.7)|
|Medium (17-32)||51 (61)|
|High (≥33)||32 (39)|
a Mean represents the typical value for each participant over all recorded virtual shops.
b Questionnaire data for 3 participants were missing due to technical issues.
This study shows that consumer purchasing patterns in the Virtual Supermarket were comparable to those in real supermarkets and confirms that the Virtual Supermarket is a valid tool to measure food purchasing behavior in a supermarket setting. Furthermore, a large majority of participants experienced a medium-to-high sense of presence in the Virtual Supermarket. The categories “fruits and vegetables” and “dairy” showed the strongest difference between virtual and real purchases. Results will be useful to guide further improvement of the software.
This validation study examined differences in purchasing patterns between virtual and real shopping occasions across a total of 18 different food categories. Results demonstrate that the 4 food groups with the highest relative expenditures were the same for the virtual and real shopping occasions (eg, fresh fruit and vegetables, bread and bakery, dairy, and meat and fish). Likewise, the 4 food groups with the lowest proportional expenditures were the same (eg, other miscellaneous, deli and chilled foods, baking and cooking goods, and baby care products). These results indicate that the overall purchasing patterns were similar in the virtual and real supermarket and that proportional purchases within food groups were comparable.
One key limitation of the Virtual Supermarket is that participants did not use real money and, therefore, could spend more than they would in real life. For this reason, participants were asked to set a realistic shopping budget and the software did not allow them to overspend. Results showed no patterns of overspending in the Virtual Supermarket compared to the real supermarket; for some food categories, people did spend more in the Virtual Supermarket (dairy, bread and bakery, and eggs) but for others they spent less (fruit and vegetables, and snack foods). Unexpectedly, total absolute expenditures were substantially lower in the Virtual Supermarket compared to the real supermarket. On average, participants spent approximately 70% of their shopping budget in the Virtual Supermarket (which is reasonable); however, in approximately 10% of the shopping occasions, participants spent less than 30% of their budget. One plausible explanation for the relatively low expenditures is that the Virtual Supermarket holds no sales promotions. A 2002 study by Aguirregabiria  revealed that typical sales promotions (which are approximately a 20% discount) increase weekly sales rates by 500%. In addition, choices are often influenced by other in-store promotions, such as end-of-aisle and merchandising displays [ ], none of which are available in our Virtual Supermarket.
We could include promotions in future versions of the Virtual Supermarket; however, when designing RCTs it is important to realize that these features might influence the outcome of interest. One key advantage of the Virtual Supermarket compared to real supermarkets is that it enables creation of a “neutral” shopping environment with completely controllable experimental and control conditions enabling effective testing of the effects of changing only one element (eg, price) without interference of other effects (eg, sales signs). Also, it can be used nationwide because it does not require participants to come to a specific research location. Depending on the specific study aims, it is important to consider the balance between making the Virtual Supermarket as realistic as possible (external validity) and designing it to deliver a valid study outcome (internal validity). A major advantage of the Virtual Supermarket is that it is modifiable and researchers can design the Virtual Supermarket in accordance with specific study needs.
In addition to overall shopping patterns, this study compared proportional purchases within 18 food categories. Results demonstrated that 6 of the food categories differed significantly in proportion of total expenditures between the virtual and real shops. These categories were bread and bakery; dairy; fruit and vegetables, fresh; fruit and vegetables, other; snack foods; and eggs. Although this finding is important, it needs to be interpreted with some caution. Most importantly, we made multiple comparisons between groups, which increased the likeliness of finding significant results. A more conservative level of significance of P=.01 might be appropriate, in which case only 3 categories (dairy, processed fruit and vegetables, and snack foods) would have shown relevant significant differences. Furthermore, our focus was on proportional purchases, meaning that if one food category showed clear differences, other food categories would be affected as well (eg, if participants spent 17% on meat in the Virtual Supermarket there was only 83% left for other categories). The reason why we focused on relative purchases and not absolute purchases is that the prices in the Virtual Supermarket and the real supermarket were expected to be different due to promotional and seasonality effects in the real supermarket. Finally, it is important to consider that our findings are quite conservative. Our conclusions are based on comparing shopping behavior in 2 different environments. However, it also can be expected that 2 shops in the same environment can be slightly different (eg, people do not buy exactly the same each time). Our analysis did not account for this “natural” variation in shopping behavior, which further strengthens our finding that the differences between the Virtual Supermarket and the real supermarket were relatively minor.
Keeping this in mind, food categories that showed the most important differences between the virtual and real shopping occasions were fresh fruit and vegetables, dairy, and snack foods. For fresh fruit and vegetables, participants spent significantly more in real life compared to the virtual shops. One important explanation for this finding is that many participants misunderstood the interface in the Virtual Supermarket for purchasing loose items. After clicking on a fruit or vegetable product, participants were asked to enter the number of grams they wanted to buy (eg, 100 grams of apples). However, we observed that a large number of participants (n=29) misinterpreted this question and entered the number of items they wanted to buy (eg, 2 apples). The software registered this automatically as grams, meaning that the till receipt would show 2 grams of apples. This error caused a significant underrepresentation of fruit and vegetable expenditures in the Virtual Supermarket and if these participants were excluded from analysis the proportional expenditures on fruit and vegetables increased from mean of approximately 14% (SD 14) to a mean of approximately 15% (SD 15). This effect might also partly explain the observed underspending in the Virtual Supermarket as mentioned previously. Therefore, improving the interface for loose items will be a key priority when updating the Virtual Supermarket. Looking at snack foods, the relative underspending within this category in the Virtual Supermarket could be due to a number of reasons. First, snack foods are normally heavily promoted which leads to increased sales [- ]. This effect was not captured in the Virtual Supermarket. Also, supermarkets tend to provide more shelf space for snacks foods, which is expected to increase sales [ ]. The Virtual Supermarket used a representative product selection based on the number of different types of products/brands available for sale in each food category in NZ (eg, different types of soft drinks), not accounting for the exact number of individual products that is normally displayed on the shelves (eg, supermarkets often display a large number of the same Coca-Cola bottles covering a whole aisle; the Virtual Supermarket did not account for this). To improve the resemblance with real supermarkets, it might be important to measure shelf space for all products in real supermarkets and design the Virtual Supermarket accordingly. However, there is a large variation in design between different supermarkets making it hard to determine the best representation; also, this requires cooperation from supermarkets which might be difficult to achieve.
Another limitation of this study was the relatively large number of participants that dropped out due to software problems. In particular, a large number of participants reported that the software could not be run on their (mostly older) computers. Since completion of this validation study, we have found a solution to this by improving the way the software processes the individual product images. Therefore, for future studies, the NZ Virtual Supermarket will be more compatible with older computers and laptops. Nevertheless, the high dropout could bias our results and participants that completed the study might have been better using the software than an average user. Another problem was that we were not able to recruit an ethnically diverse sample and, therefore, are not able to draw conclusions on the acceptability and usability of the Virtual Supermarket among Māori (indigenous) and Pacific people. To ensure diversity in future studies, it would be important to use targeted recruitment strategies for these populations. Nevertheless, our sample was relatively diverse with regard to other characteristics, including education level, age, work status, and income, and there were no signs of differential dropout.
To our knowledge, this is one of the first studies making a direct comparison between virtual and real shopping behavior. One other example is a 2004 study by Grewe et al  who validated a virtual reality (VR) shopping task to assess real-life memory and spatial navigation in patients with epilepsy. The authors found that the VR tool was valid and generalizable compared to a real-life shopping task. Furthermore, Sharpe et al [ ] validated fast food and soft drink purchases made in an online simulated road trip against actual purchasing and consumption of a lunch meal from McDonalds 1 week later. This study found strong evidence that virtual and real purchases were similar and that participants were consistent in the type of foods they chose in the virtual task [ ]. In other health-related domains, there is more evidence for the realism and performance of virtual environments. For example, VR is frequently used to improve surgical technical skills [ ], as a therapeutic instrument in clinical psychology [ , ], and in the treatment of several phobias [ ]. Also, in the field of computer sciences, there is a wealth of research into human computer interaction [ ]. Linking this evidence to the Virtual Supermarket, there is good reason to assume that people experience a high presence because of the high realism (eg, it requires very little imagination to recognize the software as a supermarket), the realistic movement (pushing a shopping trolley, 3D features), and the fact that it is easy to locate oneself due to clear landmarks (eg, aisles) [ ]. One way to further improve presence is through the use of avatars. People tend to feel a higher feeling of presence when there is a virtual representation of oneself in the virtual world and when other users recognize them (other avatars) [ ]. It would be interesting to explore the use of avatars further and to test the impact of social interactions in the Virtual Supermarket.
Another way to improve the feeling of presence is by improving software graphics and functionality. With improving technology, virtual environments are becoming increasingly realistic and we aim to improve the Virtual Supermarket alongside improving technology. Although we realize that virtual worlds will never be the same as real life, they do provide a clear opportunity to conduct research that would otherwise be nearly impossible. In addition, the Virtual Supermarket offers the opportunity to provide much needed independent rigorous experimental evidence to inform (controversial) food policy .
This study shows that the Virtual Supermarket is a valid and useful tool to study food-purchasing behavior. The results of this validation will provide valuable guidance regarding further improvements to the software and to the optimization of the internal and external validity of this innovative methodology.
This work was supported by a University of Auckland Faculty Research Development Fund grant. The lead author is supported by a Fellowship from the National Heart Foundation (grant 1570). We acknowledge Professor Tony Blakely for his help in critically reviewing the manuscript and our research assistant Rita George for her help with the completion of study procedures. The Virtual Supermarket was developed by SURFsara Amsterdam and funded by the VU University Amsterdam.
Conflicts of Interest
Multimedia Appendix 1
Means and difference for Virtual Supermarket and real supermarket purchases for the 18 food categories (total sample, n=86).PDF File (Adobe PDF File), 72KB
- Gortmaker SL, Swinburn BA, Levy D, Carter R, Mabry PL, Finegood DT, et al. Changing the future of obesity: science, policy, and action. Lancet 2011 Aug 27;378(9793):838-847 [FREE Full text] [CrossRef] [Medline]
- Swinburn BA, Sacks G, Hall KD, McPherson K, Finegood DT, Moodie ML, et al. The global obesity pandemic: shaped by global drivers and local environments. Lancet 2011 Aug 27;378(9793):804-814. [CrossRef] [Medline]
- Roberto CA, Pomeranz JL, Fisher JO. The need for public policies to promote healthier food consumption: A comment on Wansink and Chandon (2014). Journal of Consumer Psychology 2014 Jul;24(3):438-445. [CrossRef]
- Costa JC, Claro RM, Martins APB, Levy RB. Food purchasing sites. Repercussions for healthy eating. Appetite 2013 Nov;70:99-103. [CrossRef] [Medline]
- Ni Mhurchu C, Vandevijvere S, Waterlander W, Thornton LE, Kelly B, Cameron AJ, INFORMAS. Monitoring the availability of healthy and unhealthy foods and non-alcoholic beverages in community and consumer retail food environments globally. Obes Rev 2013 Oct;14 Suppl 1:108-119. [CrossRef] [Medline]
- Hawkes C. Dietary implications of supermarket development: a global perspective. Development Policy Review 2008;26(6):657-692. [CrossRef]
- Vorley B, UK Food Group. Food, Inc: Corporate Concentration from Farm to Consumer. London: International Institute for Environment and Development; 2003. URL: http://www.ukfg.org.uk/docs/UKFG-Foodinc-Nov03.pdf [accessed 2014-08-10] [WebCite Cache]
- Health Sponsorship Council. In fact: Shopping behaviours of New Zealand households. 2012 Feb. URL: http://archive.hsc.org.nz/sites/default/files/publications/In%20Fact-Shopping%20behaviours-fnl-120301.pdf [accessed 2014-08-10] [WebCite Cache]
- Glanz K, Hoelscher D. Increasing fruit and vegetable intake by changing environments, policy and pricing: restaurant-based research, strategies, and recommendations. Prev Med 2004 Sep;39 Suppl 2:S88-S93. [CrossRef] [Medline]
- Escaron AL, Meinen AM, Nitzke SA, Martinez-Donate AP. Supermarket and grocery store-based interventions to promote healthful food choices and eating practices: a systematic review. Prev Chronic Dis 2013;10:E50 [FREE Full text] [CrossRef] [Medline]
- Ni Mhurchu C, Blakely T, Jiang Y, Eyles HC, Rodgers A. Effects of price discounts and tailored nutrition education on supermarket purchases: a randomized controlled trial. Am J Clin Nutr 2010 Mar;91(3):736-747 [FREE Full text] [CrossRef] [Medline]
- Waterlander WE, de Boer MR, Schuit AJ, Seidell JC, Steenhuis IHM. Price discounts significantly enhance fruit and vegetable purchases when combined with nutrition education: a randomized controlled supermarket trial. Am J Clin Nutr 2013 Apr;97(4):886-895 [FREE Full text] [CrossRef] [Medline]
- Hersey JC, Wohlgenant KC, Arsenault JE, Kosa KM, Muth MK. Effects of front-of-package and shelf nutrition labeling systems on consumers. Nutr Rev 2013 Jan;71(1):1-14. [CrossRef] [Medline]
- Stanford DD. Bloomberg Business Week. 2012 Mar 14. Anti-obesity soda tax fails as lobbyists spend millions URL: http://www.bloomberg.com/news/articles/2012-03-13/anti-obesity-soda-tax-fails-as-lobbyists-spend-millions-retail [WebCite Cache]
- Nichols S, Haldane C, Wilson JR. Measurement of presence and its consequences in virtual environments. International Journal of Human-Computer Studies 2000 Mar;52(3):471-491. [CrossRef]
- Waterlander WE, Scarpa M, Lentz D, Steenhuis IHM. The virtual supermarket: an innovative research tool to study consumer food purchasing behaviour. BMC Public Health 2011;11:589 [FREE Full text] [CrossRef] [Medline]
- Nederkoorn C, Guerrieri R, Havermans RC, Roefs A, Jansen A. The interactive effect of hunger and impulsivity on food intake and purchase in a virtual supermarket. Int J Obes (Lond) 2009 Aug;33(8):905-912. [CrossRef] [Medline]
- Josman N, Kizony R, Hof E, Goldenberg K, Weiss PL, Klinger E. Using the virtual action planning-supermarket for evaluating executive functions in people with stroke. J Stroke Cerebrovasc Dis 2014;23(5):879-887. [CrossRef] [Medline]
- Van Kleef E. Food Intake Control. 23 Mar 2012. Have a look at the new virtual supermarket of Wageningen University URL: http://foodintakecontrol.blogspot.co.nz/2012/03/have-look-at-new-virtual-supermarket-of.html [accessed 2014-08-11] [WebCite Cache]
- Witmer BG, Jerome CJ, Singer MJ. The factor structure of the presence questionnaire. Presence: Teleoperators and Virtual Environments 2005 Jun;14(3):298-312. [CrossRef]
- Witmer BG, Singer MJ. Measuring presence in virtual environments: a presence questionnaire. Presence 1998 Jun;7(3):225-240. [CrossRef]
- Waterlander WE, Ni Mhurchu C, Steenhuis IHM. Effects of a price increase on purchases of sugar sweetened beverages. Results from a randomized controlled trial. Appetite 2014 Jul;78:32-39. [CrossRef] [Medline]
- Waterlander WE, Steenhuis IHM, de Boer MR, Schuit AJ, Seidell JC. Introducing taxes, subsidies or both: the effects of various food pricing strategies in a web-based supermarket randomized trial. Prev Med 2012 May;54(5):323-330 [FREE Full text] [CrossRef] [Medline]
- Waterlander WE, Ni Mhurchu C, Steenhuis IH. The use of virtual reality in studying complex interventions in our every-day food environment. In: Tang X, editor. Virtual Reality Human Computer Interaction. Rijeka, Croatia: InTech; 2012:229-260.
- Bioinformatics. The Virtual Supermarket: an innovative research tool to study food purchasing behaviour URL: http://www.bioinformatics.org/supermarket/index.html [accessed 2015-04-16] [WebCite Cache]
- Woodward E, Eyles H, Ni Mhurchu C. Key opportunities for sodium reduction in New Zealand processed foods. Aust N Z J Public Health 2012 Feb;36(1):84-89. [CrossRef] [Medline]
- Retail World Annual Report. Rydalmere, NSW: Retail Media; 2010.
- Aguirregabiria V. Sales promotions in supermarkets: Estimating their effects on profits and consumer welfare. Working paper. Boston: Boston University; 2002. URL: http://economics.yale.edu/sites/default/files/files/Workshops-Seminars/Industrial-Organization/aguirre-030327.pdf [accessed 2014-08-10] [WebCite Cache]
- Glanz K, Bader MDM, Iyer S. Retail grocery store marketing strategies and obesity: an integrative review. Am J Prev Med 2012 May;42(5):503-512. [CrossRef] [Medline]
- Nestle M. Conclusion: The politics of food choice. In: Goldstein E, editor. Food Politics: How the Food Industry Influences Nutrition and Health. Berkeley: University of California Press; 2007:358-374.
- Thornton LE, Cameron AJ, McNaughton SA, Worsley A, Crawford DA. The availability of snack food displays that may trigger impulse purchases in Melbourne supermarkets. BMC Public Health 2012;12:194 [FREE Full text] [CrossRef] [Medline]
- Thornton LE, Cameron AJ, McNaughton SA, Waterlander WE, Sodergren M, Svastisalee C, et al. Does the availability of snack foods in supermarkets vary internationally? Int J Behav Nutr Phys Act 2013;10:56 [FREE Full text] [CrossRef] [Medline]
- Grewe P, Lahr D, Kohsik A, Dyck E, Markowitsch HJ, Bien CG, et al. Real-life memory and spatial navigation in patients with focal epilepsy: ecological validity of a virtual reality supermarket task. Epilepsy Behav 2014 Feb;31:57-66. [CrossRef] [Medline]
- Sharpe K, Staelin R, Huber J. Using extremeness aversion to fight obesity: policy implications of context dependent demand. J Consum Res 2008 Oct;35(3):406-422. [CrossRef]
- Seymour NE, Gallagher AG, Roman SA, O'Brien MK, Bansal VK, Andersen DK, et al. Virtual reality training improves operating room performance: results of a randomized, double-blinded study. Ann Surg 2002 Oct;236(4):458-63; discussion 463. [CrossRef] [Medline]
- Riva G. Virtual reality in psychotherapy: review. Cyberpsychol Behav 2005 Jun;8(3):220-30; discussion 231. [CrossRef] [Medline]
- Cesa GL, Manzoni GM, Bacchetta M, Castelnuovo G, Conti S, Gaggioli A, et al. Virtual reality for enhancing the cognitive behavioral treatment of obesity with binge eating disorder: randomized controlled study with one-year follow-up. J Med Internet Res 2013;15(6):e113 [FREE Full text] [CrossRef] [Medline]
- Wiederhold BK, Gevirtz R, Wiederhold MD. Fear of flying: a case report using virtual reality therapy with physiological monitoring. Cyberpsychology & Behavior 1998 Jan;1(2):97-103. [CrossRef]
- Nash EB, Edwards GW, Thompson JA, Barfield W. A review of presence and performance in virtual environments. International Journal of Human-Computer Interaction 2000 May;12(1):1-41. [CrossRef]
- Ludwig DS, Nestle M. Can the food industry play a constructive role in the obesity epidemic? JAMA 2008 Oct 15;300(15):1808-1811. [CrossRef] [Medline]
|RCT: randomized controlled trials|
|SMS: short message service|
|VR: virtual reality|
Edited by G Eysenbach; submitted 11.08.14; peer-reviewed by P Bordnick, S Boucher; comments to author 23.10.14; revised version received 25.11.14; accepted 13.12.14; published 28.04.15
©Wilma Elzeline Waterlander, Yannan Jiang, Ingrid Hendrika Margaretha Steenhuis, Cliona Ni Mhurchu. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 28.04.2015.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.