This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.
Self-focused augmented reality (AR) technologies are growing in popularity and present an opportunity to address health communication and behavior change challenges.
We aimed to examine the impact of self-focused AR and vicarious reinforcement on psychological predictors of behavior change during the COVID-19 pandemic. In addition, our study included measures of fear and message minimization to assess potential adverse reactions to the design interventions.
A between-subjects web-based experiment was conducted to compare the health perceptions of participants in self-focused AR and vicarious reinforcement design conditions to those in a control condition. Participants were randomly assigned to the control group or to an intervention condition (ie, self-focused AR, reinforcement, self-focus AR × reinforcement, and avatar).
A total of 335 participants were included in the analysis. We found that participants who experienced self-focused AR and vicarious reinforcement scored higher in perceived threat severity (
Augmenting one’s reflection with vicarious reinforcement may be an effective strategy for health communication designers. While our study’s results did not show adverse effects in regard to fear and message minimization, utilization of self-focused AR as a health communication strategy should be done with care due to the possible adverse effects of heightened levels of fear.
With self-focused augmented reality (AR) usage increasing in recent years [
In response to the recent increase in self-focused AR usage [
Research in psychology suggests that heightening self-focused attention (manipulated by using a mirror or video camera) has implications for perception, affective experiences (emotions, feelings, and moods), and behavior [
Research prototypes have explored self-focus [
We present findings from an online experiment on the impact of combining self-focused AR with vicarious reinforcement, visualizing the cause and effect of risk-mitigating behavior layered onto one’s reflection. Our study took place during the COVID-19 pandemic, focusing on hand hygiene behavior as an effective measure against pathogen transmission [
Various health behavior change models [
Research suggests that self-focused attention can result in action consistency with behavioral standards [
Research suggests that fear may play a large role in health behavior, especially during public health emergencies such as the current pandemic. Harper et al [
It is important to consider the potential negative impact of heightening fear and risk perceptions. While Harper et al [
Our study investigated the impact of vicarious reinforcement outcome expectancy when combined with self-focused AR. Research suggests that outcome expectancy mediates the impact of self-focused attention on behavior. For someone who has been made aware of a discrepancy between “actual state” and “desired state,” if they don’t believe a suggested behavior change will result in the “desire state,” they are more likely to change the “desired state” [
While studies combining self-focused attention and vicarious reinforcement have yet to see much direct utilization in human-computer interaction research, a few studies on health smart mirrors [
While applications of smart mirrors for health care are limited, exploratory research prototypes have shown their potential to detect emotional states, monitor physiological parameters, and encourage behavior change. The Wize Mirror [
A recent study by Jung et al [
One notable finding from Jung et al [
Yee and Bailenson [
Our study expands on existing research by investigating how health behavior self-focused AR may impact specific predictors of behavioral intentions and what negative implications may exist in regard to fear control responses.
In our research, we examined the impact of self-focused AR and vicarious reinforcement on perception and emotion as it relates to hand washing health beliefs and behavioral intentions. Below, we present our hypotheses:
H1: The combination of self-focused AR and vicarious reinforcement will result in higher levels of perceived positive outcome expectancy, perceived threat severity and susceptibility, fear, and message minimization when compared to a control.
H2: Using avatar representations in self-focused AR with vicarious reinforcement will result in higher levels of threat severity, susceptibility, fear, and message minimization compared to a control.
To study the effects of self-focused AR on behavioral intention and perception, we conducted an online experiment. Participants interacted (
A participant in one of the self-focused augmented reality design groups viewing a reflection of themselves using a video feed from their camera.
A screenshot from the web application.
An example of the progression of vectors used in the handwashing animation.
We conducted a between-subjects experiment where users interacted with a web application and then responded to an online questionnaire. The study followed a posttest-only control group design to avoid a testing threat to internal validity. The design of the web application differed depending on the intervention condition each participant was randomly allocated to. All five conditions displayed the same information about COVID-19, including how it is spread and preventative measures as described by the US Centers for Disease Control and Prevention (CDC) [
In the control condition (
Control condition.
In the reinforcement condition (
Reinforcement condition.
The self-focus condition (
Self-focused augmented reality condition.
The self-focus AR × reinforcement condition (
Self-focus augmented reality × reinforcement condition.
In the avatar condition (
Avatar condition.
Participants were recruited via Prolific (a crowdsourcing platform [
The experiment took place between August 6-21, 2020. In all five conditions, after receiving consent, we described the experiment as a study on health information presentation and provided instructions to review the information given carefully. In the three self-focused-AR–based conditions, we displayed information on how to set up the web camera for the study.
All conditions provided information about COVID-19. Details focused on how the virus is spread and preventative measures as described by the CDC [
The questionnaire included questions to check whether the participant is paying attention. Three multiple-choice questions asked the user about the information displayed in the study (eg, What is a recommended preventative measure to reduce the spread of the coronavirus?). To validate that self-focused AR interventions were delivered correctly using the camera, participants in these conditions were informed, prior to the study, that screenshots would be collected randomly throughout the animation. The screenshots were reviewed to ensure that participant’s reflections were displayed to them and that those in the self-focus AR × reinforcement condition had their hands within the view as instructed. Only those who followed the instructions, verified by screenshots, were included in the final data set.
We collected measures of self-reported health beliefs, behavioral intention, and self-reported perceptions of COVID-19 (
Although adapted from previous research, cross loading was a concern due to the rewording of items and the difference in factors present compared to the adapted questionnaires. For example, we added items to measure opinions about perceived threat severity and susceptibility of family and friends. To examine the validity and reliability of our measures, we conducted exploratory and confirmatory factor analysis using a split-sample approach, with one half to develop a model and the other half to validate. Factors loaded as expected (
Data on demographics and the COVID-19 risk of a severe illness of loved ones were collected. Participants were asked to report their age, gender, and the state in which they currently reside. In addition, they were asked to report if they have a family member or friend who is at high risk of severe illness if they are infected with COVID-19. It was noted that one is considered high risk if they are ≥65 years and/or have underlying medical conditions.
Questionnaire items.
Variable and code | Questionnaire item | |
|
||
|
inte | I intend to wash my hands, as instructed in this study, |
|
||
|
expe1 | I believe proper handwashing, as instructed in this study, will help make me |
|
expe2 | I believe proper handwashing, as instructed in this study, will help |
|
||
|
fear1 | The emotion that I am feeling about the coronavirus (COVID-19) pandemic is: …Frightened |
|
fear2 | …Scared |
|
fear3 | …Anxious |
|
||
|
reac1 | To what extent do you feel that |
|
reac2 | …Misleading |
|
reac3 | …Distorted |
|
||
|
seve1 | I believe that the coronavirus disease (COVID-19) is a serious threat to my personal health. |
|
seve2 | I believe that the coronavirus disease (COVID-19) is a serious threat to my family members (immediate or extended). |
|
seve3 | I believe that the coronavirus disease (COVID-19) is a serious threat to my friends. |
|
seve4 | I believe that the coronavirus disease (COVID-19) is a serious threat to the general public. |
|
||
|
susc1 | I am at risk of catching the coronavirus disease (COVID-19). |
|
susc2 | My family (immediate or extended) members are at risk of catching the coronavirus disease (COVID-19). |
|
susc3 | My friends are at risk of catching the coronavirus disease (COVID-19). |
Analysis of our data using histograms and the Shapiro-Wilk test showed that the data were not normally distributed. Shapiro-Wilk
A total of 502 individuals participated in the study. Of this, 335 participants met the attention and screenshot verification checks (see
Although our study focused on predictors of behavioral intentions, we began with results pertaining to intention (
Responses for behavioral intention. Responses are strongly skewed toward higher levels of agreement for all conditions, indicating a ceiling effect. AR: augmented reality.
Hypothesis 1 proposed that a combination of vicarious reinforcement and self-focused AR would result in higher levels of fear, perceived threat severity, susceptibility, outcome expectancy, and message minimization compared to the control group. Compared with participants in the control (
For perceived threat severity, median scores for the self-focus AR × reinforcement and control groups were 6.25 and 6.00, respectively (
Responses for outcome expectancy. Responses are strongly skewed toward higher levels of agreement for all conditions, indicating a ceiling effect. AR: augmented reality.
Significantly higher levels of perceived threat severity among participants in the self-focus augmented reality (AR) × reinforcement condition compared to the control condition. No significant differences were found between the control group and the other conditions.
Significantly higher levels of perceived threat susceptibility among participants in the self-focus augmented reality (AR) × reinforcement condition compared to the control condition. No significant differences were found between the control group and the other conditions.
Additional analysis revealed that the use of self-focused AR and vicarious reinforcement individually did not impact measured predictors of intention, except for in the case of fear. When compared with participants in the control condition, those in the reinforcement condition (without self-focused AR) did not have a significant difference in medians for outcome expectancy (
A mediation model was used to test whether self-focus AR × reinforcement affects behavioral intention through perceived threat susceptibility and perceived threat severity (
Although fear was not significantly different from the control, we investigated whether fear or message minimization impacted intentions for those in the self-focus AR × reinforcement condition. First, a mediation model was used to test whether self-focus AR × reinforcement affects perceived fear through perceived threat susceptibility and perceived threat severity (
Next, a mediation model was used to test whether the self-focus AR × reinforcement condition affects behavioral intention through fear. A significant indirect effect of the self-focus AR × reinforcement condition on intention was found with fear as the mediator (b=.07, 95% CI 0.01-0.16, SE 0.04). An additional model tested whether the self-focus AR × reinforcement condition affects message minimization through fear, threat severity, or threat severity. A significant negative indirect effect of the self-focus AR × reinforcement condition on message minimization was found with severity as the mediator (b=–.07, 95% CI –0.16 to –0.008, SE 0.04. A negative serial mediation effect with susceptibility and severity was also found (b=–.07, 95% CI –0.16 to –0.008, SE 0.04).
The self-focus augmented reality (AR) × reinforcement, susceptibility, severity, and intention mediation model. The self-focus AR × reinforcement condition resulted in an indirect effect on intention through threat susceptibility and threat severity. This condition also had an indirect effect on perceived threat severity through perceived threat susceptibility.
The self-focus augmented reality (AR) × reinforcement, susceptibility, severity, and fear mediation model. The self-focus AR × reinforcement condition resulted in an indirect effect on fear through threat susceptibility and threat severity.
Hypothesis 2 proposed that combining vicarious reinforcement and self-focused AR while using an avatar would result in higher levels of positive outcome expectancies, fear, perceived threat severity, perceived threat susceptibility, and message minimization when compared to a control. Compared to the control, those in the avatar condition did not have significantly different levels of outcome expectancy (
At the end of the questionnaire, we asked participants to enter optional free-form text about the study. Themes among the responses included impact on knowledge, risk perception formation, and challenges with the self-focus AR × reinforcement condition. Several participants in the reinforcement condition provided generalized statements that the study was educational and helpful. The following comments from the self-focused AR group provided more details.
I thought the handwashing animation together with the illustration of where dirt is cleaned from the hands was very informative.
Some participants learned a new technique or strategy:
I learned some new handwashing techniques! (Particularly, locking your hands together by curling your fingers into each other to get the backs of the fingers).
I hadn’t thought about separately lathering and washing my thumbs.
Comments on personal concerns about risk indicate that future work measuring these variables may consider the time spent around others vs alone and personal risk:
I answered questions knowing that my husband’s and my job allow us to work from home, which decreases our risk significantly, and that most of my family lives in a rural area, also less susceptible to infection.
I know that I take it way more seriously due to the cancer treatment drug I take than most of my friends and peers because if I get it, I am not strong enough to fight it off. I think that factors in way more than friends and family risk, at least for me personally.
Lastly, a few participants in the self-focus AR × reinforcement group expressed confusion about the design. P7, for example, expressed difficulty in positioning themselves on the screen.
It was an interesting survey, but the instruction given for the hand part was kind of hard to complete because the outline of the hands and the picture did not match. However, I tried my best to make it work.
Found the movements in the video hard to follow along with, but I tried my best!
The handwashing directions confused me. At first, I didn’t understand that I wasn’t supposed to mimic the exact instructions.
The responses of P8 and P9 suggest that participants may have practiced along with the video animation. Practicing was not a requirement of participation but appeared to be a trend among those in this condition.
This study explored the impact of self-focus and vicarious reinforcement design interventions on psychological predictors of behavior change during the COVID-19 pandemic. Our results showed that combining self-focused AR with vicarious reinforcement increases perceived threat severity and threat susceptibility and could potentially impact behavioral intentions.
Our results did not show any direct effects on behavioral intention (
Our findings also revealed interesting relationships between the self-focus AR × reinforcement condition, perceived threat severity, threat susceptibility, fear, and intention. Mediation models showed the self-focus AR × reinforcement condition to positively affect intention and fear through increased perceived threat susceptibility and threat severity. We found self-focus AR × reinforcement to increase perceived threat severity through increased perceived susceptibility. Lastly, we found self-focus AR × reinforcement to indirectly affect intention with fear as the mediator. These results suggest that design strategies that layer a health threat directly on an individual’s reflection may increase one’s perceived threat susceptibility, threat, severity, fear, and indirectly behavioral intention. While such strategies might help meet behavior change design objectives, it is essential to note the potential consequences of designs that increase fear, especially in the context of a public health emergency.
Based on the EPPM, Li [
Self-focused AR and vicarious reinforcement embedded as features independently (versus combined) did not show a significant result on any of the tested predictors of behavior change except fear. We provide a few possible explanations for these results. First, regarding mirror self-focus, threat severity, and susceptibility, there may have been too large of a time gap between when participants reviewed the health information provided and when they looked at their self-reflection. In the conditions combining the features, there was content on the screen, reminding participants of the threat at hand. Future work may account for this difference by providing text-based information over one’s reflection. Second, regarding vicarious reinforcement and outcome expectancy,
It is important to note that this research took place during a long-term global public health emergency with restrictions on lifestyles that can take a while to get adjusted to. Health perceptions related to current circumstances are subject to change throughout the lifecycle of a pandemic. Our findings warrant replication studies that consider changes in severity, government mandates, social perceptions, and the availability and range of tools for risk mitigation (vaccines/medication, personal protective equipment, etc).
The data used in this study are self-reported and susceptible to response biases, specifically social desirability bias. Due to the severity of the pandemic, government mandates, news coverage, and social discussions may have increased the pressure to respond in ways that align with social norms. Future work should aim to use methods to decrease the impact of this limitation.
It is important to note that focusing on individual constructs may create an ineffective design system if the construct only works in combination with other constructs [
While our study indicates that combining self-focused AR with vicarious reinforcement may affect health behavior change by influencing threat severity and susceptibility, we lack a data-driven explanation of why. Future work may benefit from the inclusion of quantitative measures for self-focused attention to compare with severity and susceptibility scores.
Future work may lend itself to developing experimental methods to explore the extent to which self-monitoring, reflective thinking, self-evaluation, and emotion management naturally occur (or do not occur) when using self-focused AR. Conducting these experiments will provide deeper insights into how self-focused AR impacts the psychological mechanism related to behavior change and possibly inspire experiments on how the combination of self-focused attention and other design features could enhance this effect.
According to user feedback, future work should also consider accounting for time spent around others vs alone and personal health when measuring perceived threat severity. Responses also indicate that when measuring perceptions about new messaging, participants should be instructed to respond based on their preferred news source to limit confusion related to the different opinions they hold for individual new sources.
Two comments from the user feedback indicated that participants might have been actually practicing handwashing movements while viewing the animation. During the screenshot verification process, it was noted whether a participant was observed practicing along with the handwashing video. Practicing was not a requirement of participation and was not mentioned in any instructions provided to them. A total of 32 out of 63 participants were observed practicing along with the video in the self-focus AR × reinforcement condition. The self-focused AR and avatar conditions had 7 and 6 individuals observed practicing, respectively. These results cannot be used to make any claims due to the study’s technical setup. Those in the self-focus AR × reinforcement group were instructed to have their hands in view of the camera. Those in other conditions may have practiced off-camera. However, as practicing may affect behavior, this is another potential area for future research.
As self-focused AR technologies grow in popularity, it is important to understand how such experiences could impact perceptions, emotions, and behavioral intentions. Previous research [
We found that displaying germs disappearing directly from the user’s self-reflection during a handwashing animation will result in higher scores for perceived threat severity and susceptibility when compared to the control or conditions that implemented self-reflection and a display of germs disappearing separately. Increased perceived severity and susceptibility were found to increase behavioral intention. These findings indicate that combining self-focused AR with vicarious reinforcement may be an effective strategy for health communication designers. However, we also voice concern about the possible adverse effects of heightening levels of fear as a design strategy. While our study did not show concerning results, prior research indicates that heightening fear as a health communication strategy can lead to defensive reactions (versus changing behavior) [
Factor loadings.
augmented reality
Centers for Disease Control and Prevention
Extended Parallel Process Model
World Health Organization
We thank our Wellesley College HCI Lab colleagues, family, and friends for participating in the pilot testing for the study. We thank Angela Qian and Karl Ivan M San Luis for their feedback on the study, as well as Shaun Digan for proofreading. This material is based on work supported by the National Science Foundation (1814628). Any opinions, findings, and conclusions or recommendations expressed in this paper are those of the authors and do not necessarily reflect those of the National Science Foundation.
None declared.