Supportive environments are essential for people to make healthier food choices and there is a growing call to implement structural interventions such as fiscal policies and front-of-pack labeling to create a healthier food environment [1-3]. Supermarkets form a key setting for health-promoting interventions [4,5] because they have a central position within the food chain [6,7] and they are where people buy most of their food (87% of households in New Zealand buy foods from supermarkets weekly or more often [8]). Although structural interventions in the retail setting have clear potential, high-quality evidence on their effectiveness is limited [9,10]. Evidence from randomized controlled trials (RCTs) is particularly lacking. Although examples of high-quality supermarket trials exist [10-12], experimental studies have mostly been conducted in settings with a limited number of food choices and most lacked objective outcome measures [13].

The main reason for the dearth of high-quality supermarket trials is that they are complex and costly to conduct. This is especially true for RCTs (eg, it would be problematic to raise prices on soft drinks [simulating a tax] in some supermarkets but not in others). Furthermore, opposition and extensive lobbying from the food industry against several proposed intervention strategies is another important reason for the lack of experimental studies [14].

In order to find a solution to the complexities of undertaking supermarket intervention research, we developed a research tool that can be used to study the effects of interventions in a virtual reality setting: the Virtual Supermarket (see Figure 1). Virtual environments are computer-generated models, which participants can experience and interact with intuitively in real time [15]. Gaming technology is used to simulate a real supermarket shopping experience where study participants purchase virtual food items. Photographs of real foods are used to compose virtual food products and prices are displayed on virtual shelf labels. Currently, 2 versions of this Virtual Supermarket exist: the original Dutch version [16] and a New Zealand version (developed using a similar methodology) that is the subject of this study. Other examples of virtual supermarkets can be found in the scientific literature; however, to our best knowledge, these have different functionalities [17], are developed for other purposes (eg, rehabilitation) [18], or have not published study results yet [19].

To obtain valid study results, it is important that the Virtual Supermarket simulates a real supermarket as closely as possible. Here, one of the most important traits is the level of presence people experience [15]. Presence is defined as “a psychological state of ‘being there’ mediated by an environment that engages our senses, captures our attention, and fosters our active involvement” [20] and reflects “the subjective experience of being in one place or environment, even if one is physically located in another” [21]. Factors that are hypothesized to contribute to a sense of presence are (1) involvement, (2) sensory fidelity, (3) adaptation/immersion, and (4) interface quality factors. Second, a key question is whether peoples’ virtual behavior reflects their real-life behavior accurately. Studies using the Dutch Virtual Supermarket have already shown some promising results [22,23]. However, to date, validation of the software has been solely based on self-report (questionnaire) data. Therefore, the aim of this study was to validate the NZ Virtual Supermarket by comparing virtual and real-life food purchasing behavior using objective outcome measures and also to evaluate the level of presence that people experience. Results will inform further development of the software and guide the development of future experiments using the Virtual Supermarket.

This study shows that consumer purchasing patterns in the Virtual Supermarket were comparable to those in real supermarkets and confirms that the Virtual Supermarket is a valid tool to measure food purchasing behavior in a supermarket setting. Furthermore, a large majority of participants experienced a medium-to-high sense of presence in the Virtual Supermarket. The categories “fruits and vegetables” and “dairy” showed the strongest difference between virtual and real purchases. Results will be useful to guide further improvement of the software.

This validation study examined differences in purchasing patterns between virtual and real shopping occasions across a total of 18 different food categories. Results demonstrate that the 4 food groups with the highest relative expenditures were the same for the virtual and real shopping occasions (eg, fresh fruit and vegetables, bread and bakery, dairy, and meat and fish). Likewise, the 4 food groups with the lowest proportional expenditures were the same (eg, other miscellaneous, deli and chilled foods, baking and cooking goods, and baby care products). These results indicate that the overall purchasing patterns were similar in the virtual and real supermarket and that proportional purchases within food groups were comparable.

One key limitation of the Virtual Supermarket is that participants did not use real money and, therefore, could spend more than they would in real life. For this reason, participants were asked to set a realistic shopping budget and the software did not allow them to overspend. Results showed no patterns of overspending in the Virtual Supermarket compared to the real supermarket; for some food categories, people did spend more in the Virtual Supermarket (dairy, bread and bakery, and eggs) but for others they spent less (fruit and vegetables, and snack foods). Unexpectedly, total absolute expenditures were substantially lower in the Virtual Supermarket compared to the real supermarket. On average, participants spent approximately 70% of their shopping budget in the Virtual Supermarket (which is reasonable); however, in approximately 10% of the shopping occasions, participants spent less than 30% of their budget. One plausible explanation for the relatively low expenditures is that the Virtual Supermarket holds no sales promotions. A 2002 study by Aguirregabiria [28] revealed that typical sales promotions (which are approximately a 20% discount) increase weekly sales rates by 500%. In addition, choices are often influenced by other in-store promotions, such as end-of-aisle and merchandising displays [29], none of which are available in our Virtual Supermarket.

We could include promotions in future versions of the Virtual Supermarket; however, when designing RCTs it is important to realize that these features might influence the outcome of interest. One key advantage of the Virtual Supermarket compared to real supermarkets is that it enables creation of a “neutral” shopping environment with completely controllable experimental and control conditions enabling effective testing of the effects of changing only one element (eg, price) without interference of other effects (eg, sales signs). Also, it can be used nationwide because it does not require participants to come to a specific research location. Depending on the specific study aims, it is important to consider the balance between making the Virtual Supermarket as realistic as possible (external validity) and designing it to deliver a valid study outcome (internal validity). A major advantage of the Virtual Supermarket is that it is modifiable and researchers can design the Virtual Supermarket in accordance with specific study needs.

In addition to overall shopping patterns, this study compared proportional purchases within 18 food categories. Results demonstrated that 6 of the food categories differed significantly in proportion of total expenditures between the virtual and real shops. These categories were bread and bakery; dairy; fruit and vegetables, fresh; fruit and vegetables, other; snack foods; and eggs. Although this finding is important, it needs to be interpreted with some caution. Most importantly, we made multiple comparisons between groups, which increased the likeliness of finding significant results. A more conservative level of significance of P=.01 might be appropriate, in which case only 3 categories (dairy, processed fruit and vegetables, and snack foods) would have shown relevant significant differences. Furthermore, our focus was on proportional purchases, meaning that if one food category showed clear differences, other food categories would be affected as well (eg, if participants spent 17% on meat in the Virtual Supermarket there was only 83% left for other categories). The reason why we focused on relative purchases and not absolute purchases is that the prices in the Virtual Supermarket and the real supermarket were expected to be different due to promotional and seasonality effects in the real supermarket. Finally, it is important to consider that our findings are quite conservative. Our conclusions are based on comparing shopping behavior in 2 different environments. However, it also can be expected that 2 shops in the same environment can be slightly different (eg, people do not buy exactly the same each time). Our analysis did not account for this “natural” variation in shopping behavior, which further strengthens our finding that the differences between the Virtual Supermarket and the real supermarket were relatively minor.

Keeping this in mind, food categories that showed the most important differences between the virtual and real shopping occasions were fresh fruit and vegetables, dairy, and snack foods. For fresh fruit and vegetables, participants spent significantly more in real life compared to the virtual shops. One important explanation for this finding is that many participants misunderstood the interface in the Virtual Supermarket for purchasing loose items. After clicking on a fruit or vegetable product, participants were asked to enter the number of grams they wanted to buy (eg, 100 grams of apples). However, we observed that a large number of participants (n=29) misinterpreted this question and entered the number of items they wanted to buy (eg, 2 apples). The software registered this automatically as grams, meaning that the till receipt would show 2 grams of apples. This error caused a significant underrepresentation of fruit and vegetable expenditures in the Virtual Supermarket and if these participants were excluded from analysis the proportional expenditures on fruit and vegetables increased from mean of approximately 14% (SD 14) to a mean of approximately 15% (SD 15). This effect might also partly explain the observed underspending in the Virtual Supermarket as mentioned previously. Therefore, improving the interface for loose items will be a key priority when updating the Virtual Supermarket. Looking at snack foods, the relative underspending within this category in the Virtual Supermarket could be due to a number of reasons. First, snack foods are normally heavily promoted which leads to increased sales [29-31]. This effect was not captured in the Virtual Supermarket. Also, supermarkets tend to provide more shelf space for snacks foods, which is expected to increase sales [32]. The Virtual Supermarket used a representative product selection based on the number of different types of products/brands available for sale in each food category in NZ (eg, different types of soft drinks), not accounting for the exact number of individual products that is normally displayed on the shelves (eg, supermarkets often display a large number of the same Coca-Cola bottles covering a whole aisle; the Virtual Supermarket did not account for this). To improve the resemblance with real supermarkets, it might be important to measure shelf space for all products in real supermarkets and design the Virtual Supermarket accordingly. However, there is a large variation in design between different supermarkets making it hard to determine the best representation; also, this requires cooperation from supermarkets which might be difficult to achieve.

Another limitation of this study was the relatively large number of participants that dropped out due to software problems. In particular, a large number of participants reported that the software could not be run on their (mostly older) computers. Since completion of this validation study, we have found a solution to this by improving the way the software processes the individual product images. Therefore, for future studies, the NZ Virtual Supermarket will be more compatible with older computers and laptops. Nevertheless, the high dropout could bias our results and participants that completed the study might have been better using the software than an average user. Another problem was that we were not able to recruit an ethnically diverse sample and, therefore, are not able to draw conclusions on the acceptability and usability of the Virtual Supermarket among Māori (indigenous) and Pacific people. To ensure diversity in future studies, it would be important to use targeted recruitment strategies for these populations. Nevertheless, our sample was relatively diverse with regard to other characteristics, including education level, age, work status, and income, and there were no signs of differential dropout.

To our knowledge, this is one of the first studies making a direct comparison between virtual and real shopping behavior. One other example is a 2004 study by Grewe et al [33] who validated a virtual reality (VR) shopping task to assess real-life memory and spatial navigation in patients with epilepsy. The authors found that the VR tool was valid and generalizable compared to a real-life shopping task. Furthermore, Sharpe et al [34] validated fast food and soft drink purchases made in an online simulated road trip against actual purchasing and consumption of a lunch meal from McDonalds 1 week later. This study found strong evidence that virtual and real purchases were similar and that participants were consistent in the type of foods they chose in the virtual task [34]. In other health-related domains, there is more evidence for the realism and performance of virtual environments. For example, VR is frequently used to improve surgical technical skills [35], as a therapeutic instrument in clinical psychology [36,37], and in the treatment of several phobias [38]. Also, in the field of computer sciences, there is a wealth of research into human computer interaction [39]. Linking this evidence to the Virtual Supermarket, there is good reason to assume that people experience a high presence because of the high realism (eg, it requires very little imagination to recognize the software as a supermarket), the realistic movement (pushing a shopping trolley, 3D features), and the fact that it is easy to locate oneself due to clear landmarks (eg, aisles) [39]. One way to further improve presence is through the use of avatars. People tend to feel a higher feeling of presence when there is a virtual representation of oneself in the virtual world and when other users recognize them (other avatars) [39]. It would be interesting to explore the use of avatars further and to test the impact of social interactions in the Virtual Supermarket.

Another way to improve the feeling of presence is by improving software graphics and functionality. With improving technology, virtual environments are becoming increasingly realistic and we aim to improve the Virtual Supermarket alongside improving technology. Although we realize that virtual worlds will never be the same as real life, they do provide a clear opportunity to conduct research that would otherwise be nearly impossible. In addition, the Virtual Supermarket offers the opportunity to provide much needed independent rigorous experimental evidence to inform (controversial) food policy [40].

This study shows that the Virtual Supermarket is a valid and useful tool to study food-purchasing behavior. The results of this validation will provide valuable guidance regarding further improvements to the software and to the optimization of the internal and external validity of this innovative methodology.