Background: In February 2020, the Chinese government imposed a complete lockdown of Wuhan and other cities in Hubei Province to contain a spike of COVID-19 cases. Although such measures are effective in preventing the spread of the virus, medical professionals strongly voiced a caveat concerning the pandemic emotional burnout at the individual level. Although the lockdown limited individuals’ interpersonal communication with people in their social networks, it is common that individuals turn to social media to seek and share health information, exchange social support, and express pandemic-generated feelings.
Objective: Based on a holistic and multilevel perspective, this study examines how pandemic-related emotional exhaustion enacts intrapersonal, interpersonal, and hyperpersonal emotional regulation strategies, and then evaluates the effectiveness of these strategies, with a particular interest in understanding the role of hyperpersonal-level regulation or social media–based regulation.
Methods: Using an online panel, this study sampled 538 Chinese internet users from Hubei Province, the epicenter of the COVID-19 outbreak in China. Survey data collection lasted for 12 days from February 7-18, 2020, two weeks after Hubei Province was placed under quarantine. The sample had an average age of 35 (SD 10.65, range 18-78) years, and a majority were married (n=369, 68.6%).
Results: Using structural equation modeling, this study found that intrapersonal-level (B=0.22; β=.24; P<.001) and interpersonal-level (B=0.35; β=.49; P<.001) emotional regulation strategies were positively associated with individuals’ outcome reappraisal. In contrast with intrapersonal and interpersonal regulations, hyperpersonal (social media–based) regulation strategies, such as disclosing and retweeting negative emotions, were negatively related to the outcome reappraisal (B=–1.00; β=–.80; P<.001).
Conclusions: Consistent with previous literature, intrapersonal-level regulation (eg, cognitive reappraisal, mindfulness, and self-kindness) and interpersonal-level supportive interaction may generate a buffering effect on emotional exhaustion and promote individuals’ reappraisal toward the stressful situation. However, hyperpersonal-level regulation may exacerbate the experienced negative emotions and impede reappraisal of the pandemic situation. It is speculated that retweeting content that contains pandemic-related stress and anxiety may cause a digital emotion contagion. Individuals who share other people’s negative emotional expressions on social media are likely to be affected by the negative affect contagion. More importantly, the possible benefits of intrapersonal and interpersonal emotion regulations may be counteracted by social media or hyperpersonal regulation. This suggests the necessity to conduct social media–based health communication interventions to mitigate the social media–wide negative affect contagion if lockdown policies related to highly infectious diseases are initiated.
Since the index case of COVID-19 was first documented in Wuhan on December 1, 2019 , the COVID-19 pandemic has swept across the world. On January 23, 2020, the Chinese government imposed a complete lockdown of Wuhan and other cities in Hubei Province, the epicenter of the COVID-19 outbreak in China, to contain the spike in new cases. In an effort to flatten the curve, this measure stipulated that only one person from each household was permitted to go outside the home for provisions once every 2 days, except for medical reasons or employment at food stores, pharmacies, or hospitals. Following the order of complete social distancing, local residents in Hubei Province had to drastically modify their lifestyles. This was not only in Hubei, and this can now be observed in other parts of the world such as London. Although such measures are effective in containing the spread of the virus at the societal level, medical professionals strongly voiced a caveat concerning the pandemic emotional burnout at the individual level [ ]. The pandemic itself, as well as the drastic social distancing policies, generates unprecedented stressors including perceived severe threats to personal safety, intense fears, strong feelings of being out of control, exhaustion, and loneliness, thus exerting compounding effects on individuals mentally, emotionally, and physically [ , ].
Given the affective cost of coping with the pandemic, health authorities worldwide, including the US Centers for Disease Control and Prevention, have called for more attention to daily emotion regulation among individuals . According to the literature on social psychology and interpersonal communication, traditional emotion regulation strategies include cognitive reappraisal, self-kindness regulation, and interaction with one’s support network. In addition to these traditional adjustment strategies, social media has become an important channel for disclosing and regulating negative emotions [ , ]. During the lockdown, individuals in isolation turn to social media to seek and share health information, exchange social support, and express pandemic-generated feelings [ ]. Although previous studies suggest that social media use can promote psychological well-being through satisfying individual’s need for belonging, receiving informational and emotional support, and reducing stress and loneliness [ ], empirical evidence equally reveals that, during the COVID-19 pandemic, social media use exacerbates the anxiety, stress, and depression symptoms people have already experienced [ , ]. The effectiveness of social media–based emotion regulation, along with its relationship with the traditional regulation strategies, calls for a careful and systematic examination.
Additionally, existing COVID-19–related studies on the association between social media use and mental health focus on social media use alone without considering other possible methods of emotion regulation [, ]. However, the extant literature on emotion regulation and well-being suggests that, although regulation effectiveness differs across each strategy, there is no single best and cure-all solution [ ]. Instead, researchers suggest that a combination of several strategies and a certain level of flexibility in choosing regulation strategies predict more desirable outcomes [ ]. Thus, researchers have called for adopting a multilevel approach to understanding the emotion regulation process within and across levels, along with the dynamics between regulation strategies across levels [ ]. Based on a holistic and multilevel perspective, this study first examined how pandemic-related emotional exhaustion enacts regulation behaviors at each level and then evaluated the effectiveness of these regulation strategies, with a particular interest in depicting the interconnectedness among intrapersonal, interpersonal, and hyperpersonal regulation processes. Therefore, this study can present a more comprehensive picture of emotional regulation strategies during the COVID-19 lockdown than other studies.
This study also has potential practical implications for psychological intervention during lockdowns. The research is based on a survey of respondents from China’s Hubei Province, which was collected in early February 2020. During that period, people including experts knew little about the new coronavirus, and Hubei Province implemented perhaps the most stringent but effective COVID-19 lockdown measures in the world. An analysis of an extreme case of scientific uncertainty and emotional exhaustion can offer valuable practical experiences for professionals to design their psychological intervention schemes for similar situations in the future.
Emotional Exhaustion in the COVID-19 Pandemic
Emotional exhaustion has been highlighted as one of the most prominent stress reactions signifying a state of feeling emotionally drained and a depletion of affective resources [- ]. Exhaustion concerns have been examined in medical, educational, and organizational contexts, where individuals cope with stress and uncertainty on a routine basis [ - ]. The conceptualization of emotional exhaustion addresses its nature of being intensely affective and energy depleting, and therefore directly influences individuals’ appraisal and coping with the stressors.
Epidemic studies suggest that infectious diseases characterized by long duration, extra complexity, ambiguity, and excessive demands on coping resources could result in persistent psychopathological consequences such as acute distress, anxiety, and emotional exhaustion . The results from two cross-sectional studies conducted in China and Italy show that nurses and medical professionals participating in COVID-19 emergency-related work experienced moderate to severe emotional exhaustion [ , ]. In addition to medical professionals, the public is also impacted by acute prevalent stressors during the outbreaks of infectious diseases. For example, Chinese college students reported significant emotional distress and psychological burnout in response to the severe acute respiratory syndrome (SARS) epidemic in 2003 [ , ], and similar emotional distress (eg, anxiety, stress, fear, and depression symptoms) was observed in various countries during the recent COVID-19 pandemic [ , , ].
When experiencing emotional exhaustion, individuals initiate regulation processes to manage stressor-generated feelings and cope with the situation . Empirical evidence in the past two decades suggests using various regulation strategies such as mindfulness, cognitive restricting, avoidance, and support seeking to facilitate adaptive stress appraisal and predict decreased emotional exhaustion [ , ]. Moreover, the results demonstrate that emotion regulation strategies serve as a buffer against the negative impacts of anxiety, fear, or stress and promote better psychological adjustment to the situation [ ].
Reappraisal of the Pandemic Situation Through Multilevel Emotion Regulation
The goal to influence one’s emotional trajectory activates the emotion regulation process . According to the transactional theory of stress and coping, individuals constantly appraise external stimuli within their environment, among which those appraised as threatening, harmful, or challenging generate negative emotions, further initiating the regulation process [ ]. In the regulation processes, individuals enact various strategies, deliberately or unconsciously, to change the current affective state.
According to the transactional theory, the regulation process involves “constantly changing cognitive and behavioral efforts to manage external and/or internal demands that are appraised as taxing or exceeding the resources of a person” . Individuals can either make efforts to manage the stressor or regulate resultant emotions directly. Therefore, regulation is conceptualized as a dynamic and adaptive process that restores the equilibrium between individuals’ appraisal of the environment and one’s coping resources. The transactional theory suggests assessing the effectiveness of the regulation process through cognitive reappraisal, including a re-evaluation of the situation’s nature (eg, from stressful to benign) and individuals’ coping ability (eg, from inadequate to adequate) toward it [ ].
Intrapersonal-Level Emotional Regulation
Research on emotion regulation has identified various regulation strategies that vary in terms of their primary impacts on the emotion generative process. Empirical evidence suggests that individuals differ systematically in their use of regulation strategies and the reported effectiveness . In the process model of emotion regulation, Gross [ ] proposes to use cognitive reappraisal as a common emotion regulation strategy. Cognitive reappraisal is an antecedent-focused strategy that occurs before the emotion responses have been entirely generated. By reconstructing the emotion-eliciting situation differently, reappraisal alters its emotional impact and the subsequent emotional trajectory. Empirical evidence has demonstrated the effectiveness of reappraisal on downregulating negative emotions, improving environmental mastery, and promoting psychological well-being [ , ].
In recent decades, mindfulness has received more attention in cognitive and behavioral therapy, and has been recommended as an alternative to reappraisal regulation. Mindfulness regulation requires purposefully and nonjudgmentally focusing on the present moment and therefore promotes awareness of one’s current state and an openness to accept it [, ]. The results from a daily diary study reveal that mindfulness regulation predicts emotional well-being, including a lower-level negative affect and a higher-level positive affect [ ].
In addition to mindfulness regulation, researchers in positive psychology also suggest the use of self-kindness as an alternative to regulate negative affect. Self-kindness regulation entails understanding, gentleness, and love toward oneself in times of struggle; it not only is teachable through weeklong interventions but also demonstrates the buffering effects on stress and self-criticism [, ].
Reappraisal, mindfulness, and self-kindness, although occurring at different stages during the emotion-generative process and each with a unique locus of focus, happen all at the intrapersonal level within an individual’s cognitive processing. These three regulation strategies are selected to reflect the characteristics of some commonly adopted intrapersonal-level regulations, rather than to provide a comprehensive review of emotion regulation. By involving these three strategies, this study first re-evaluates the extent that intrapersonal-level emotion regulation is associated with emotional exhaustion and outcome reappraisal in the context of the COVID-19 pandemic. In particular, we formed the following hypotheses (H):
- H1a: Pandemic-related emotional exhaustion will be positively associated with intrapersonal-level emotion regulation.
- H2a: Intrapersonal-level regulation will be positively associated with outcome reappraisal of the pandemic situation.
Interpersonal-Level Emotion Regulation and Support Seeking
Support seeking is commonly viewed as an active regulation behavior to cope with stressful situations and emotional distress . It differs from the intrapersonal-level regulation process in its fundamentally social nature. As the “first act” in the process of supportive communication, support seeking is defined as “an intentional communicative activity to elicit supportive actions from others” [ ]. When facing health-related stress and anxiety, individuals turn to their friends and family for emotional support to regain coping efficacy toward the situation. A cross-sectional study demonstrated that individuals who engaged in active support-seeking behaviors reported less depressive and anxiety symptoms when coping with SARS-related stressors [ ].
The way people seek support from others varies in terms of the directness of one’s communication. A direct seeking behavior involves explicitly asking for assistance and clearly discussing the problem or distress . In contrast, indirect seeking is often less informative about one’s problem and support needs, and often expresses one’s desire through nonverbal cues such as sighing, fidgeting, and avoiding eye contact [ , ]. The choice of direct and indirect seeking strategies is influenced by various factors including help seekers’ characteristics (eg, gender or social skills), appraisal of the current problem (eg, perceived stigma, attribution of the cause, or ego-relevance), and the relational nature with the support provider [ ]. Although differing in their effectiveness, both support-seeking strategies are commonly adopted in the interpersonal-level regulation process and have demonstrated effects on improving emotional distress [ ]. Thus, we hypothesize the following:
- H1b: Pandemic-related emotional exhaustion will be positively associated with interpersonal-level emotion regulation.
- H2b: Interpersonal-level emotion regulation will be positively associated with outcome reappraisal of the pandemic situation.
Hyperpersonal-Level Emotion Regulation
The prevalence of social technologies has expanded and transformed how individuals share and regulate emotional experiences. Recent studies have witnessed an increase in social media use and pandemic-related online expression, particularly aiming to manage the stress, anxiety, and emotional burnout toward COVID-19 [, ]. During lockdown and social distancing, individuals express emotional experiences and release pent-up feelings to their social network and beyond through posting on social media [ ]. Empirical evidence has shown that individuals who experience a higher intensity of stress provide more frequent and intimate self-disclosure on social media [ ]. Therefore, research on emotion regulation proposes adopting online self-disclosure as an alternative regulation strategy and expects it to increase sharers’ ability to comprehend and further vent the negative affect [ ].
However, the social context of the computer-mediated communication environment fundamentally changes the nature and the psychological outcomes of emotional disclosure. Unlike writing down feelings in a private diary, self-disclosure on social media always involves the presence of the imaginary audience who may or may not participate in concrete conversations. Therefore, social media constructs a hyperpersonal communication environment where social context cues such as the appearance and facial expressions of the conversational partner are constantly unavailable, and individuals perceive the possibility of others “reading” and “interacting” with their posts through an overattribution of “the abstract others” being present in the mediated environment .
Under the observation of the imaginary audience, sharing negative emotions, either through posting one’s feelings or retweeting emotional content that reflects one’s current affective state, may be regarded as inappropriate and could result in psychological maladjustment. The feeling of “inappropriateness” to negative disclosure feelings comes from both the “positivity norm” of internet culture and individuals’ self-presentation desires [, ]. In particular, negative sharing receives less feedback, reduces perceived connectedness in the online community, and generates concerns about the social cost of a negative self-presentation. In line with the theoretical proposition, the results from a daily diary study reveal that disclosing negative emotions on social media increases negative affect, rather than neutralizing it [ ].
In addition to posting feelings online, social media provides an alternative way to disclose one’s affective state, through retweeting emotional content. Retweeting emotional content, as another type of emotional disclosure, differs from the posting behavior by creating a connection between two individuals, the one who writes the original tweet and the one who shares it. Through retweeting, emotional experiences can be transmitted from one person to another and become contagious. Given that the COVID-19 outbreak has generated impacts on a large scale, individuals are more susceptible to take on other people’s stress, anxiety, and emotional burnout on social media . Considering the effects of social media self-disclosure and emotion contagion, we further hypothesize that hyperpersonal-level emotion regulation, such as sharing feelings or retweeting emotional content on social media, may generate maladaptive effects.
- H1c: Pandemic-related emotional exhaustion will be positively associated with hyperpersonal-level emotion regulation.
- H2c: Hyperpersonal-level emotion regulation will be negatively associated with outcome reappraisal of the pandemic situation.
A Multilevel Approach to Emotion Regulation
The process of emotion regulation involves using internal, relational, and societal resources to manage individuals’ affective state and, thereby, is fundamentally a multilevel construct. Individuals dynamically adjust regulation strategies across contexts and use emotion regulation within and across levels to maximally succeed in pursuing their own idiosyncratic goals. A growing understanding suggests matching regulation strategies to environmental circumstances and evaluating the effectiveness of emotion regulation holistically, rather than focusing on individual strategies .
In this study, we adopt a multilevel approach to depict the interconnectedness between regulation processes at intrapersonal, interpersonal, and hyperpersonal levels . By incorporating a multilevel focus, we examine the system feature of emotion regulation and thereby reveal the mechanism of regulation effectiveness from a broader sense (see for the conceptual model). For example, when individuals engage in emotion regulation at one level, will they simultaneously use regulation strategies at other levels? Furthermore, will regulation processes at different levels generate cumulative effects or counteract each other on coping effectiveness? Thus, we came up with a research question (RQ) about the relationship between regulation processes at three levels.
- RQ: How are emotion regulation processes at intrapersonal, interpersonal, and hyperpersonal levels correlated with each other?
Sample and Procedure
An online survey was conducted to examine individuals’ emotional state, regulation strategies, and outcome reappraisal and coping during the COVID-19 outbreak. Data collection lasted for 12 days from February 7 to 18, 2020, two weeks after Hubei Province was placed under quarantine. During the survey period, participants were undergoing quarantine based on government policy and received a link to the online questionnaire.
The survey was first designed in English and then translated into Chinese by two bilingual communication researchers. The final version was further reviewed by the third bilingual researcher with expertise in public health to ensure the accuracy of the translation. Sociocultural background and linguistic nuances were recognized and reflected during the translation. Additionally, we checked the validity of the measurements originally developed in Western contexts. This study measured five latent variables including emotional exhaustion, intrapersonal-level emotion regulation, interpersonal-level emotion regulation, hyperpersonal-level emotion regulation, and outcome reappraisal. All these measurement scales were successfully applied to Chinese respondents in previous studies that demonstrated their validity in the Chinese context [- ].
Participants (N=538; male: n=273; female: n=265) from China’s Hubei Province, where the COVID-19 pandemic first broke out, were recruited from an online survey service in China. After the outbreak in Hubei, the Chinese government implemented strict lockdown and social distancing policies, and most social interactions of people in Hubei took place on mediated channels, which gave the researchers a unique context to understand emotional regulation strategies. The study used quota sampling to draw participants, which is common for online panel research . The quota sampling scheme used age group and gender as the criteria to design subgroups (there were roughly equal numbers of respondents in the six subgroups). The age groups included 18-29 years, 30-39 years, and 40 years or older. The final sample had an average age of 35 (SD 10.65; range 18-78) years and with a majority being married (n=369, 68.6%). The respondents reported very frequent use of social media during the lockdown: for WeChat and microblogs, the average score was 6.01 (SD 1.17) and, for online forums and WeChat Groups, the average scores was 5.47 (SD 1.40; both on a scale of 1-7).
Compared to Chinese census data , the sample recruited from the online panel was younger and had slightly more females, and the sample were exclusively internet users. Since the major goal of the study is to examine how emotional regulation through social media is associated with reappraisal, the online panel data can best satisfy our research agenda. The age and gender quota in the final sample is specified to test the proposed conceptual model. Considering that people younger than 35 years account for 82%-83% of China’s internet users [ ], we oversampled this age group to reflect the majority of internet users in China.
All measures were on a 7-point scale ranging from 1 (not at all) to 7 (very much) unless otherwise indicated.
Emotional exhaustion was measured with six items adopted from the emotional exhaustion subscale in the Maslach Burnout Inventory–General Survey developed by Schaufeli and colleagues . The Maslach Burnout Inventory is widely used in research to measure emotional exhaustion in various situations and has been validated in studies examining COVID-19–related burnout [ ]. We removed three working-related items and kept the remaining six items to measure general emotional exhaustion toward COVID-19, since most people were not back at work when the data was collected. In particular, participants rated to what extent they feel emotionally drained and intense fear and anxiety during the lockdown (mean 20.25, SD 8.64; Cronbach α=.89).
Intrapersonal-Level Emotion Regulation
Three regulation strategies including cognitive reappraisal, self-kindness, and mindfulness were reported to assess intrapersonal-level emotion regulation. Considering that participants were experiencing pandemic-generated stress, we kept the questionnaire brief to increase the completion rate and reduce the response burden. Therefore, when measuring emotion regulation strategies, we only selected items with the highest factor loadings and that were suitable for the research context . Given the one-dimensional nature of each regulation strategy, a short scale (even a single-item one) could demonstrate good validity against equivalent full-scale versions, if not be even preferable [ , ]. In this study, the results of the Cronbach alpha and Spearman–Brown formula, the latter of which was adopted for two-item measurements [ ], demonstrated good reliabilities. Cognitive reappraisal regulation was measured with five items from the Emotion Regulation Questionnaire by Gross and John [ ] asking the level that participants “change the way they think about the pandemic situation when they want to feel less negative emotion” (mean 25.94, SD 5.04; Cronbach α=.76). Self-kindness regulation was measured with five items based on the self-kindness subdimension in the Self-Compassion Scale by Neff [ ]: “I give myself the caring and tenderness I need during the COVID-19 pandemic” (mean 24.86, SD 5.66; Cronbach α=.81). Mindfulness regulation was measured with three items adopted from the mindfulness subdimension in the Self-Compassion Scale by Neff [ ]: “When I am feeling down during the pandemic, I try to approach my feelings with openness” (mean 15.54, SD 3.34; Cronbach α=.68).
Interpersonal-Level Emotion Regulation
Two types of support-seeking behaviors (direct and indirect seeking) were assessed based on measurements developed by Derlega and colleagues  to evaluate interpersonal-level emotion regulation. Direct support seeking was measured with four items (eg, “I tell people the exact emotions I am experiencing because of the pandemic during the supportive interaction” and “I ask people for help when feeling negative about the pandemic situation during supportive interactions”; mean 13.13, SD 6.42; Cronbach α=.89). Indirect support seeking was measured with four items (eg, “I sign a lot when talking about the pandemic situation during the supportive interaction”; mean 10.05, SD 5.66; Cronbach α=.85). In addition, emotional expressivity in support-seeking conversation was measured with two items from the Emotional Expressivity Scale developed by Kring and colleagues [ ] (eg, “I express my feelings about the pandemic in my conversation”; mean 7.82, SD 3.18; rSpearman-Brown=0.77).
Hyperpersonal-Level Emotion Regulation
Regulation strategies on the hyperpersonal-level focus on two emotional expressive behaviors online, posting and retweeting. Emotional expressive posting was measured with two items from the Emotional Expressivity Scale developed by Kring and colleagues  (eg, “I express my feelings about the pandemic in my online post”; mean 7.24, SD 3.27; rSpearman-Brown=0.79). Emotional expressive retweeting was measured by asking respondents to what extent their retweeting content conveys “fear,” “anxiety,” and “stress” about the pandemic situation (mean 8.22, SD 4.53; Cronbach α=.85).
Two types of outcome reappraisal were examined as dependent variables with measures from Holmstrom and Kim . Emotion-focused reappraisal was measured with four items asking to what extent respondents, for example, “feel less stressful towards the pandemic situation.” Problem-focused reappraisal was measured with five items such as “I know more about how to cope with the pandemic situation.” An additive index of nine items was created (mean 47.88, SD 9.31; Cronbach α=.89).
Structural equation modeling was used to test the relationship between all components in the proposed model simultaneously while accounting for measurement error. We first ran a confirmatory factor analysis to assess the construct validity of the measurement model and then created a path model using the Lavaan package in R (R Foundation for Statistical Computing). We constructed and examined a five-factor model under maximum likelihood estimation, as the multivariate normality assumption was not violated. A correlation matrix of latent variables (composite indexes were computed) is shown in.
|1. Outcome reappraisal|
|2. Emotional exhaustion|
|6. Direct seeking|
|7. Indirect seeking|
|8. Expressive talk|
|9. Expressive posting|
|10. Expressive retweeting|
In building the measurement model, the antecedent emotional exhaustion and outcome reappraisal were first-order latent variables. For the three levels of regulation strategies, three second-order latent variables were assessed. The second-order intrapersonal-level emotional regulation included cognitive reappraisal, self-kindness, and mindfulness; interpersonal-level regulation consisted of direct and indirect support seeking and emotional expressivity in support seeking conversation; and hyperpersonal-level regulation encompassed emotional expressive posting and emotional expressive retweeting. The measurement portion of the model demonstrated a good fit (χ2841/df=2.50; root mean square error of approximation [RMSEA]=0.053; standardized root mean square residual [SRMR]=0.067). A comparative fit index (CFI) was not computed to assess the model fit in this study since, according to statistician Kenny , a CFI is not informative when the RMSEA of the null model is less than 0.158 (in this case, it was 0.155).
Path Model and Hypothesis Testing
Further, we turned to the path model to examine the relationships among emotional exhaustion, three levels of emotion regulation, and outcome reappraisal. In the path model (), emotional exhaustion was included as the exogenous variable and was left in the model to be associated with three levels of regulation strategies. Intrapersonal, interpersonal, and hyperpersonal emotion regulation were defined as three second-order latent factors in the model. Each of the three was allowed to be related to outcome reappraisal. To explore the relationships between three levels of the regulation process, we correlated the three regulation strategies (second-order latent variables in the middle of ). The hypothesized model demonstrated good fit (χ2842/df=2.68; RMSEA=0.056; SRMR=0.068). displays the path estimates that are statistically significant at the P level of .05. The following paragraphs show the results of the hypothesis testing.
H1a hypothesized a positive association between emotional exhaustion and intrapersonal-level emotion regulation. According to the results of the test, H1a was supported (B=0.11; β=.13; P=.01). H2a predicted that intrapersonal-level regulation is positively related to outcome reappraisal. The results suggest that H2a was supported (B=0.22; β=.24; P<.001).
H1b hypothesized that emotional exhaustion is positively associated with interpersonal-level emotion regulation. H1b was supported (B=0.32; β=.31; P<.001). H2b predicted that interpersonal-level emotion regulation has a positive association with outcome reappraisal. The results of the path model suggest that H2b was supported (B=0.35; β=.49; P<.001).
H1c predicted that emotional exhaustion is positively related to hyperpersonal-level emotion regulation, which was supported (B=0.42; β=.70; P<.001). H2c hypothesized a negative relationship between hyperpersonal-level emotion regulation and outcome reappraisal. The results were in line with our prediction (B=–1.00; β=–.80; P<.001).
The only research question in this study explores the relationships among the three levels of the regulation process. The results revealed that interpersonal-level emotional regulation is positively associated with intrapersonal-level regulation (B=0.23; β=.19; P=.001) and hyperpersonal-level regulation (B=0.57; β=.89; P<.001). In contrast, there exists no correlation between intrapersonal-level regulation and hyperpersonal-level regulation (B=0.003; β=.006; P=.94).
The COVID-19 pandemic has incurred unprecedented stressors, including the absence of vaccines, loneliness during social distancing, and concerns about the well-being of oneself and loved ones . Given the evidence of affective cost among individuals across different cultures [ ], it is crucial to understand and provide health interventions to individuals’ emotion regulation processes [ , ]. This study shows that residents in Hubei Province experienced a moderate level of emotional exhaustion during the COVID-19 outbreak and the ensuing unprecedented lockdown. The results find that individuals actively adopted cross-level regulation strategies to cope with the pandemic-generated burnout.
Understanding and Intervening Emotion Regulation Systematically
As the epicenter of the COVID-19 outbreak, Hubei Province implemented stringent social distancing measures and mobility restrictions, such as residents-only entry policies in residential communities, compulsory wearing of face masks, and closing nonessential community services. To Hubei residents that had a lack of understanding toward the pathogenesis of the virus and the medical treatment and the vaccines that were under testing, the uncertainty and anxiety toward the pandemic accumulated to a high level. Under the unique circumstance, this study examined regulation processes among Hubei residents during the lockdown at intrapersonal, interpersonal, and hyperpersonal levels. Consistent with the previous studies [, - ], the results showed that intrapersonal-level regulation (eg, cognitive reappraisal, mindfulness, and self-kindness) and interpersonal-level supportive interaction seem to generate a buffering effect on emotional exhaustion and promote individuals’ reappraisal toward the stressful situation. However, according to the results of the study, hyperpersonal-level regulation exacerbates the experienced negative emotions and impedes reappraisal of the pandemic situation.
Based on a multilevel approach recommended by Cook and colleagues , this study examined three levels of emotion regulation processes simultaneously and depicted the interconnectedness between levels of regulation behaviors. The results reveal that the adoption of interpersonal-level regulation is commonly combined with intrapersonal or hyperpersonal regulation strategies. Individuals who either regulate one’s negative emotions online or internally also turn to friends and family for emotional support to cope with pandemic-generated affect. The complementary nature between interpersonal-level regulation and the other two levels points out the unique contribution of social interactions and the necessity of a certain degree of sociality in the regulation process.
In addition, the positive correlation between intra- and interpersonal level regulation behaviors raises concerns about the disparity regarding regulation resources and ability among individuals. In particular, individuals who are better at internal regulation (eg, cognitive reappraisal, self-kindness, and mindfulness) also engage more frequently in supportive interactions with one’s network, leading to a potential gap in emotional well-being between “skillful” regulator with rich support resources and the others who are bad at managing negative emotions and have fewer support resources. Thus, to eliminate the gap, community-wide interventions could prioritize those with fewer network resources and worse regulation ability to promote their emotional well-being.
The Dark Side of Social Media–Based Emotion Regulation
In contrast with intrapersonal and interpersonal emotional regulations, social media–based regulation strategies, such as disclosing and retweeting negative emotions, generate maladaptive effects. The results in this study reveal that individuals who frequently disclose pandemic-related feelings and retweet COVID-19–related negative emotions on social media reported less reappraisal of the stressful situation. The possible maladaptive effects of social media use during the pandemic is consistent with several recent studies in the context of the current COVID-19 outbreak [, , ]. Based on the existing literature, this study analyzes social media in a more refined manner. Instead of measuring social media use as a whole, this study enhances our understanding by examining two specific active social media participation behaviors, namely, posting and retweeting. Through posting and retweeting emotional content, individuals use social media as a channel to disclose and regulate their negative emotions. However, the data of this study suggests that disclosing and sharing pandemic-related anxiety and fear on social media cannot help with relieving stress but may lead to a large-scale contagion of emotional burnout.
The possible counter-effects of hyperpersonal regulation may be due to the fact that the pandemic raises societal-level emotional burnout, during which many fellow social media users equally experience excessive demands on coping both emotionally and instrumentally . This may limit their mental resources to provide supportive feedback to negative affective sharing on social media [ ]. Therefore, disclosing negative feelings may cause others’ fear and anxiety, which further results in self-censorship of one’s disclosure behavior as “inappropriate” [ ]. In addition, retweeting content that contains pandemic-related stress and anxiety may cause a digital emotion contagion [ ]. Individuals who share other people’s negative emotional expressions on social media are likely to be affected by the negative affect contagion.
The results of the study sound stern alarms. This study finds that the size of the coefficients for emotional exhaustion’s association with intrapersonal regulation is the least, that for interpersonal regulation is mid-sized, and that for hyperpersonal regulation is the largest. In other words, in the era of social media, it is relatively hard for intrapersonal regulation to be activated, while emotional exhaustion can most easily kickoff hyperpersonal and interpersonal regulation. However, it should be noted that the outcomes of the latter two regulation strategies are completely opposite, and the size of the coefficient of hyperpersonal regulation is almost twice that of interpersonal regulation. This means that social media use (B=–1.00; β=–.80; P<.001) may counteract the benefits obtained through interpersonal (B=0.35; β=.49; P<.001) and intrapersonal (B=0.22; β=.24; P<.001) regulations. Without proper external interventions, individuals are unlikely to relieve their stress through easily obtained tools on their own during the lockdown in a pandemic.
The results in this study bear practical implications. The survey data was collected right after the outbreak of COVID-19 in Hubei Province, therefore providing a precious opportunity to examine individual’s mental well-being and emotion regulation behaviors during the initial stage of the public health risk. According to the crisis and emergency risk communication model, the initial stage of a public emergency is commonly characterized by a high level of uncertainty, a need for reducing stress, and a desire for reassuring self-efficacy . Empirical evidence demonstrates that the high uncertainty and self-relevance nature of the pandemic’s initial stage generates moderate to high level emotional exhaustion among individuals. Further, the ability to actively and effectively regulate pandemic-generated emotional exhaustion varies significantly among individuals, and thus calls for public health communication interventions.
The social media–based emotion regulation process generates more impact on outcome reappraisal, compared to intrapersonal and interpersonal regulation. Considering the maladaptive effects of social media–based regulation, it is necessary to conduct effective health communication intervention in the emotional contagion on social media platforms if lockdown policies are to be initiated during public health crises. In the first step, public health agencies may consider using natural language processing techniques to conduct sentiment analysis of social media posts by individuals to closely monitor the stress level and anxiety tendency, and pinpoint the user segments with such symptoms. Further, these agencies may consider collaborating with social media platforms and using their content recommendation systems to share supportive messages with the target user segments to alleviate their stress and anxiety at the individual level, thus impeding the large-scale contagion of negative affect on social media.
Interpersonal-level regulation is most effective for managing pandemic-generated emotional exhaustion. Considering the benefits of social interactions on coping with pandemic-generated stress (interpersonal-level regulation), public health agencies may consider adopting emerging media technologies such as social robots and augmented and virtual reality to create opportunities for community residents to connect and talk with each other, which social distancing and lockdown policies have made difficult . These new communication technologies could possibly create a mediated environment to fulfill individuals’ needs for social interaction and provide an effective channel for community-level intervention.
In addition, intrapersonal-level regulation, though demonstrated as effective and beneficial, can hardly be activated when experiencing emotional exhaustion. Public health interventions should consider focusing on improving awareness of individuals’ emotional state and providing guidance on conducting intrapersonal-level regulations. For example, health interventions could use mobile phone–based apps to help individuals practice mindfulness meditation and cognitive reappraisal. Public service advertisements could convey self-kindness messages to the public through broadcasting. Community-level regulation could hold online hangouts to help individuals engage in emotion regulation together virtually.
More importantly, this study implies the limitations of one-to-one psychological counseling. The results suggest that individuals in public health crises attempted various methods to regulate their negative affect. This means that, by incorporating the structural features of emotion regulation, health interventions may adopt a flexible combination of cross-level regulation strategies, based on the contexts in which individuals are embedded.
Limitations and Future Research
The findings from this study should be considered alongside its limitations. A perennial problem with surveys is the bias of retrospective self-reports. Although emotion regulation is commonly defined as a goal-generated activity, some regulation processes may also happen subtly and unconsciously. In addition, the accuracy of recognizing, recalling, and reporting regulation processes varies among individuals based on their emotional intelligence level. Future studies could adopt a daily diary methodology to eliminate potential bias.
Another drawback of the study is the cross-sectional design, and readers should be cautioned about the causality. Future studies could adopt a longitudinal design and test the intervention effectiveness on a 2-week basis. In addition, this study examined individual’s emotion regulation behaviors at the initial stage of the COVID-19 outbreak. The initial stage of public health emergencies possesses a unique nature of uncertainty and a need for reassurance. Future studies could extend this study by focusing on other stages and making a systematic comparison.
By adopting a multilevel approach, this study examined emotion regulation processes that take place at different levels. The multilevel approach provides a systematic depiction of regulation processes while allowing for the flexibility of choosing and combining regulation strategies, therefore suiting the theoretical propositions of emotion regulation more appropriately. Future studies should consider adopting the multilevel approach when examining various regulation behaviors.
This study was supported by the Ministry of Education Key Project of Key Research Institute of Humanities and Social Sciences (No. 16JJD860008).
Conflicts of Interest
- Huang C, Wang Y, Li X, Ren L, Zhao J, Hu Y, et al. Clinical features of patients infected with 2019 novel coronavirus in Wuhan, China. Lancet 2020 Feb 15;395(10223):497-506 [FREE Full text] [CrossRef] [Medline]
- Queen D, Harding K. Societal pandemic burnout: a COVID legacy. Int Wound J 2020 Aug;17(4):873-874 [FREE Full text] [CrossRef] [Medline]
- American Psychiatric Association. Diagnostic and statistical manual of mental disorders (DSM-5®). Washington, DC: American Psychiatric Publishing; 2013.
- Restauri N, Sheridan AD. Burnout and posttraumatic stress disorder in the coronavirus disease 2019 (COVID-19) pandemic: intersection, impact, and interventions. J Am Coll Radiol 2020 Jul;17(7):921-926 [FREE Full text] [CrossRef] [Medline]
- Healthcare personnel and first responders: how to cope with stress and build resilience during the COVID-19 pandemic. Centers for Disease Control and Prevention. 2020. URL: https://www.cdc.gov/coronavirus/2019-ncov/hcp/mental-health-healthcare.html [accessed 2021-01-01]
- Choi M, Toma CL. Social sharing through interpersonal media: patterns and effects on emotional well-being. Comput Hum Behav 2014 Jul;36:530-541. [CrossRef]
- Gross JJ. Emotion regulation: current status and future prospects. Psychological Inquiry 2015 Mar 09;26(1):1-26. [CrossRef]
- Zhong B, Huang Y, Liu Q. Mental health toll from the coronavirus: social media usage reveals Wuhan residents' depression and secondary trauma in the COVID-19 outbreak. Comput Human Behav 2021 Jan;114:106524 [FREE Full text] [CrossRef] [Medline]
- Zhan L, Sun Y, Wang N, Zhang X. Understanding the influence of social media on people’s life satisfaction through two competing explanatory mechanisms. Aslib J Inf Manage 2016;68(3):347-361. [CrossRef]
- Prikhidko A, Long H, Wheaton MG. The effect of concerns about COVID-19 on anxiety, stress, parental burnout, and emotion regulation: the role of susceptibility to digital emotion contagion. Front Public Health 2020;8:567250. [CrossRef] [Medline]
- Cook JE, Purdie-Vaughns V, Meyer IH, Busch JTA. Intervening within and across levels: a multilevel approach to stigma and public health. Soc Sci Med 2014 Feb;103:101-109. [CrossRef] [Medline]
- Figley C. Compassion Fatigue: Coping With Secondary Traumatic Stress Disorder in Those Who Treat the Traumatized. London: Brunner-Routledge; 1995.
- Halbesleben JRB, Bowler WM. Emotional exhaustion and job performance: the mediating role of motivation. J Appl Psychol 2007 Jan;92(1):93-106. [CrossRef] [Medline]
- Maslach C. Burnout: The Cost of Caring. Englewood Cliffs, NJ: Prentice Hall; 1982.
- Hülsheger UR, Alberts HJEM, Feinholdt A, Lang JWB. Benefits of mindfulness at work: the role of mindfulness in emotion regulation, emotional exhaustion, and job satisfaction. J Appl Psychol 2013 Mar;98(2):310-325. [CrossRef] [Medline]
- Kim B, Lee M, Kim K, Choi H, Lee SM. Longitudinal analysis of academic burnout in Korean middle school students. Stress Health 2015 Oct;31(4):281-289. [CrossRef] [Medline]
- Marjanovic Z, Greenglass ER, Coffey S. The relevance of psychosocial variables and working conditions in predicting nurses' coping strategies during the SARS crisis: an online questionnaire survey. Int J Nurs Stud 2007 Aug;44(6):991-998 [FREE Full text] [CrossRef] [Medline]
- Tam CWC, Pang EPF, Lam LCW, Chiu HFK. Severe acute respiratory syndrome (SARS) in Hong Kong in 2003: stress and psychological impact among frontline healthcare workers. Psychol Med 2004 Oct;34(7):1197-1204. [CrossRef] [Medline]
- Giusti EM, Pedroli E, D'Aniello GE, Stramba Badiale C, Pietrabissa G, Manna C, et al. The psychological impact of the COVID-19 outbreak on health professionals: a cross-sectional study. Front Psychol 2020;11:1684. [CrossRef] [Medline]
- Wan Z, Lian M, Ma H, Cai Z, Xianyu Y. Factors associated with burnout among Chinese nurses during COVID-19 epidemic: a cross-sectional study. BMC Nurs. Preprint posted online June 9, 2020. [FREE Full text] [CrossRef]
- Gan Y, Liu Y, Zhang Y. Flexible coping responses to severe acute respiratory syndrome-related and daily life stressful events. Asian J Soc Psychol 2004;7(1):55-66. [CrossRef]
- Main A, Zhou Q, Ma Y, Luecken LJ, Liu X. Relations of SARS-related stressors and coping to Chinese college students' psychological adjustment during the 2003 Beijing SARS epidemic. J Couns Psychol 2011 Jul;58(3):410-423. [CrossRef] [Medline]
- Eden AL, Johnson BK, Reinecke L, Grady SM. Media for coping during COVID-19 social distancing: stress, anxiety, and psychological well-being. Front Psychol 2020;11:577639. [CrossRef] [Medline]
- Ye Y, Long T, Liu C, Xu D. The effect of emotion on prosocial tendency: the moderating effect of epidemic severity under the outbreak of COVID-19. Front Psychol 2020;11:588701. [CrossRef] [Medline]
- Lazarus R, Folkman S. Stress, Appraisal, and Coping. New York: Springer; 1984.
- Gross JJ, Sheppes G, Urry HL. Cognition and Emotion Lecture at the 2010 SPSP Emotion Preconference. Cogn Emot 2011 Aug;25(5):765-781. [CrossRef] [Medline]
- Biggs A, Brough P, Drummond S. Lazarus and Folkman's psychological stress coping theory. In: Cooper CL, Quick JC, editors. The Handbook of Stress and Health. Oxford, UK: John Wiley & Sons; 2017:349-364.
- Gross JJ, John OP. Individual differences in two emotion regulation processes: implications for affect, relationships, and well-being. J Pers Soc Psychol 2003 Aug;85(2):348-362. [CrossRef] [Medline]
- Gross JJ. Emotion regulation in adulthood: timing is everything. Curr Dir Psychol Sci 2016 Jun 23;10(6):214-219. [CrossRef]
- Ray RD, McRae K, Ochsner KN, Gross JJ. Cognitive reappraisal of negative affect: converging evidence from EMG and self-report. Emotion 2010 Aug;10(4):587-592 [FREE Full text] [CrossRef] [Medline]
- Bishop SR. What do we really know about mindfulness-based stress reduction? Psychosom Med 2002;64(1):71-83. [CrossRef] [Medline]
- Kabat‐Zinn J. Mindfulness‐based interventions in context: Past, present, and future. Clinical Psychology: Science and Practice 2006 May 11;10(2):144-156. [CrossRef]
- Brockman R, Ciarrochi J, Parker P, Kashdan T. Emotion regulation strategies in daily life: mindfulness, cognitive reappraisal and emotion suppression. Cogn Behav Ther 2017 Mar;46(2):91-113. [CrossRef] [Medline]
- Neff KD, Kirkpatrick KL, Rude SS. Self-compassion and adaptive psychological functioning. J Res Pers 2007 Feb;41(1):139-154. [CrossRef]
- Neff K, Knox M. Self-compassion. In: Zeigler-Hill V, Shackelford TK, editors. Encyclopedia of Personality and Individual Differences. New York: Springer; 2020.
- Derlega V, Winstead B, Oldfield E, Barbee A. Close relationships and social support in coping with HIV: a test of sensitive interaction systems theory. AIDS Behav 2003 Jun;7(2):119-129. [CrossRef] [Medline]
- MacGeorge E, Feng B, Burlseson B. Supportive communication. In: Knapp ML, Daly JA, editors. The SAGE Handbook of Interpersonal Communication. London: Sage; 2011:317-354.
- Barbee AP, Cunningham MR. An experimental approach to social support communications: interactive coping in close relationships. Ann Int Commun Assoc 2016 May 18;18(1):381-413. [CrossRef]
- Goldsmith DJ. The communicative microdynamics of support. Ann Int Commun Assoc 2016 May 18;18(1):414-433. [CrossRef]
- Jiang L, Drolet A, Kim HS. Age and social support seeking: understanding the role of perceived social costs to others. Pers Soc Psychol Bull 2018 Jul;44(7):1104-1116. [CrossRef] [Medline]
- Zhang R. The stress-buffering effect of self-disclosure on Facebook: an examination of stressful life events, social support, and mental health among college students. Comput Hum Behav 2017 Oct;75:527-537. [CrossRef]
- Pennebaker JW. Writing about emotional experiences as a therapeutic process. Psychol Sci 2016 May 06;8(3):162-166. [CrossRef]
- Walther J, Van DHB, Ramirez A, Burgoon J, Peña J. Interpersonal and hyperpersonal dimensions of computer-mediated communication. In: Sundar SS, editor. The Handbook of the Psychology of Communication Technology. New York: Wiley & Sons; 2015:3-22.
- Metzler A, Scheithauer H. The long-term benefits of positive self-presentation via profile pictures, number of friends and the initiation of relationships on Facebook for adolescents' self-esteem and the initiation of offline relationships. Front Psychol 2017;8:1981. [CrossRef] [Medline]
- Vogel EA, Rose JP, Crane C. "Transformation Tuesday": temporal context and post valence influence the provision of social support on social media. J Soc Psychol 2018;158(4):446-459. [CrossRef] [Medline]
- Bonanno GA, Burton CL. Regulatory flexibility: an individual differences perspective on coping and emotion regulation. Perspect Psychol Sci 2013 Nov;8(6):591-612. [CrossRef] [Medline]
- Liu W, Chen L, Tu X. Chinese adaptation of Emotion Regulation Questionnaire for Children and Adolescents (ERQ-CCA): a psychometric evaluation in Chinese children. Int J Psychol 2017 Oct;52(5):398-405. [CrossRef] [Medline]
- Chen R, Sun C, Chen J, Jen H, Kang XL, Kao C, et al. A large-scale survey on trauma, burnout, and posttraumatic growth among nurses during the COVID-19 pandemic. Int J Ment Health Nurs 2021 Feb;30(1):102-116 [FREE Full text] [CrossRef] [Medline]
- Hou J, Wong SY, Lo HH, Mak WW, Ma HS. Validation of a Chinese version of the Five Facet Mindfulness Questionnaire in Hong Kong and development of a short form. Assessment 2014 Jun;21(3):363-371. [CrossRef] [Medline]
- Lin C, Wu Z, Detels R. Family support, quality of life and concurrent substance use among methadone maintenance therapy clients in China. Public Health 2011 May;125(5):269-274 [FREE Full text] [CrossRef] [Medline]
- Chan RCK, Shi C, Lui SSY, Ho KKY, Hung KSY, Lam JWS, et al. Validation of the Chinese version of the Clinical Assessment Interview for Negative Symptoms (CAINS): a preliminary report. Front Psychol 2015;6:7. [CrossRef] [Medline]
- Callegaro M, Baker R, Bethlehem J, Göritz A, Krosnick J, Lavrakas P. Online Panel Research: A Data Quality Perspective. London: John Wiley & Sons; 2014.
- Tabulation on the 2019 population census of the People's Republic of China: Table 2-1 2019. China Statistics Press. 2020 Dec 20. URL: http://www.stats.gov.cn/tjsj/ndsj/2019/indexeh.htm [accessed 2021-01-01]
- Fang X, Yen DC. Demographics and behavior of internet users in China. Technol Soc 2006 Aug;28(3):363-387. [CrossRef]
- Schaufeli W, Leiter M, Maslach C, Jackson S. The MBI-general survey. In: Maslach C, Jackson SE, Leiter MP, editors. Maslach Burnout Inventory Manual (3rd ed). Palo Alto, CA: Consulting Psychologists Press; 1996:19-26.
- Raykov T. Alpha if item deleted: a note on loss of criterion validity in scale development if maximizing coefficient alpha. Br J Math Stat Psychol 2008 Nov;61(Pt 2):275-285. [CrossRef] [Medline]
- Nagy M. Using a single-item approach to measure facet job satisfaction. J Occup Organizational Psychol 2002;75(1):77-86. [CrossRef]
- Robinson MA. Using multi-item psychometric scales for research and practice in human resource management. Hum Resource Manage 2017 Sep 22;57(3):739-750. [CrossRef]
- Eisinga R, Grotenhuis MT, Pelzer B. The reliability of a two-item scale: Pearson, Cronbach, or Spearman-Brown? Int J Public Health 2013 Aug;58(4):637-642. [CrossRef] [Medline]
- Neff KD. The development and validation of a scale to measure self-compassion. Self Identity 2003 Jul;2(3):223-250. [CrossRef]
- Kring AM, Smith DA, Neale JM. Individual differences in dispositional expressiveness: development and validation of the Emotional Expressivity Scale. J Pers Soc Psychol 1994 May;66(5):934-949. [CrossRef] [Medline]
- Holmstrom AJ, Kim S. The mediating role of cognitive reattribution and reappraisal in the esteem support process. Commun Res 2012 Jun 13;42(1):60-86. [CrossRef]
- Kenny DA. Measuring model fit. David A Kenny. 2020 Jun 5. URL: http://davidakenny.net/cm/fit.htm [accessed 2021-01-01]
- Wheaton MG, Prikhidko A, Messner GR. Is fear of COVID-19 contagious? The effects of emotion contagion and social media use on anxiety in response to the coronavirus pandemic. Front Psychol 2020;11:567379. [CrossRef] [Medline]
- Reynolds B, W Seeger M. Crisis and emergency risk communication as an integrative model. J Health Commun 2005;10(1):43-55. [CrossRef] [Medline]
- Ye J. Pediatric mental and behavioral health in the period of quarantine and social distancing with COVID-19. JMIR Pediatr Parent 2020 Jul 28;3(2):e19867 [FREE Full text] [CrossRef] [Medline]
|CFI: comparative fit index|
|MOE: Ministry of Education|
|RMSEA: root mean square error of approximation|
|RQ: research question|
|SARS: severe acute respiratory syndrome|
|SRMR: standardized root mean square residual|
Edited by G Eysenbach, C Basch; submitted 08.01.21; peer-reviewed by H Kim; comments to author 29.01.21; revised version received 03.02.21; accepted 01.03.21; published 16.03.21Copyright
©Ruosi Shao, Zhen Shi, Di Zhang. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 16.03.2021.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.