Published on in Vol 22, No 6 (2020): June

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/17253, first published .
Attitudes and Use of Information and Communication Technologies in Older Adults With Mild Cognitive Impairment or Early Stages of Dementia and Their Caregivers: Cross-Sectional Study

Attitudes and Use of Information and Communication Technologies in Older Adults With Mild Cognitive Impairment or Early Stages of Dementia and Their Caregivers: Cross-Sectional Study

Attitudes and Use of Information and Communication Technologies in Older Adults With Mild Cognitive Impairment or Early Stages of Dementia and Their Caregivers: Cross-Sectional Study

Original Paper

1Mental Health Department, Instituto de Investigación Biomédica de Málaga, University Regional Hospital of Malaga, Malaga, Spain

2Department of Health, Blekinge Institute of Technology, Karlskrona, Sweden

3Departamento de Fisioterapia, Instituto de Biomedicina de Málaga, Universidad de Málaga, Malaga, Spain

4Brain, Cognition and Behavior – Clinical Research, Consorci Sanitari de Terrassa, Barcelona, Spain

5Life Supporting Technologies Group, Universidad Politécnica, Madrid, Spain

6University Colleges Leuven-Limburg, Genk, Belgium

*these authors contributed equally

Corresponding Author:

Jose Guzman-Parra, PhD

Mental Health Department

Instituto de Investigación Biomédica de Málaga

University Regional Hospital of Malaga

Plaza Hospital Civil s/n Hospital Civil 1ª Planta Pabellón 4

Malaga

Spain

Phone: 34 951 29 03 07

Email: jgp00004@hotmail.com


Background: Information and communication technologies are promising tools to increase the quality of life of people with dementia or mild cognitive impairment and that of their caregivers. However, there are barriers to their use associated with sociodemographic factors and negative attitudes, as well as inadequate knowledge about technologies.

Objective: The aim of this study was to analyze technophilia (attitudes toward new technologies) and the use of smartphones and tablets along with associated factors in people with dementia/mild cognitive impairment and their caregivers.

Methods: Data from the first visit of the Support Monitoring and Reminder for Mild Dementia (SMART4MD) randomized multicenter clinical trial were used for this analysis. Data were obtained from two European countries, Spain and Sweden, and from three centers: Consorci Sanitari de Terrassa (Catalonia, Spain), Servicio Andaluz de Salud (Andalusia, Spain), and the Blekinge Institute of Technology (Sweden). Participants with a score between 20 and 28 in the Mini Mental State Examination, with memory problems (for more than 6 months), and who were over the age of 55 years were included in the study, along with their caregivers. The bivariate Chi square and Mann-Whitney tests, and multivariate linear and logistic regression models were used for statistical analysis.

Results: A total of 1086 dyads were included (N=2172). Overall, 299 (27.53%) of people with dementia/mild cognitive impairment had a diagnosis of dementia. In addition, 588 (54.14%) of people with dementia/mild cognitive impairment reported using a smartphone almost every day, and 106 (9.76%) used specific apps or software to support their memory. Among the caregivers, 839 (77.26%) used smartphones and tablets almost every day, and 181 (16.67%) used specific apps or software to support their memory. The people with dementia/mild cognitive impairment showed a lower level of technophilia in comparison to that of their caregivers after adjusting for confounders (B=0.074, P=.02) with differences in technology enthusiasm (B=0.360, P<.001), but not in technology anxiety (B=–0.042, P=.37). Technophilia was associated with lower age (B=–0.009, P=.004), male gender (B=–0.160, P<.001), higher education level (P=.01), living arrangement (living with children vs single; B=–2.538, P=.01), country of residence (Sweden vs Spain; B=0.256, P<.001), lower depression (B=–0.046, P<.001), and better health status (B=0.004, P<.001) in people with dementia/mild cognitive impairment. Among caregivers, technophilia was associated with comparable sociodemographic factors (except for living arrangement), along with a lower caregiver burden (B=–0.005, P=.04) and better quality of life (B=0.348, P<.001).

Conclusions: Technophilia was associated with a better quality of life and sociodemographic variables in people with dementia/mild cognitive impairment and caregivers, suggesting potential barriers for technological interventions. People with dementia/mild cognitive impairment frequently use smartphones and tablets, but the use of specific apps or software to support memory is limited. Interventions using these technologies are needed to overcome barriers in this population related to sociodemographic characteristics and the lack of enthusiasm for new technologies.

Trial Registration: ClinicalTrials.gov NCT03325699; https://clinicaltrials.gov/ct2/show/NCT03325699

J Med Internet Res 2020;22(6):e17253

doi:10.2196/17253

Keywords



The population in Europe is getting older, and consequently the number of people with dementia or mild cognitive impairment is increasing given the association of these conditions with age. It is estimated that the population with dementia will double by 2030 and triple by 2050, reaching more than 115 million individuals [1]. One of the core symptoms in mild cognitive impairment and in the early stages of dementia is memory impairment, which is a condition that is also associated with depression, sleep problems, and other behavioral symptoms [2,3]. Guaranteeing an optimal quality of life for people with these conditions remains an enormous challenge because there are no effective long-term pharmacological treatments in the majority of cases [4].

Information and communication technologies (ICTs), especially touchscreen technologies, are promising tools to increase the quality of life and cognitive function of people with dementia/mild cognitive impairment and their caregivers [5-9]. These technologies could be used to train cognitive functions, monitor health and movements, provide reminders to support memory, promote social support, improve communication with caregivers, and provide useful information about the condition. Smartphones and tablets have the advantage of not raising a stigma for the individuals that use them [10] because they are ubiquitous and used by the majority of the population, and they also represent a natural source of data for professionals and researchers [10-13]. Over 40,000 health-related apps exist but very few are specifically designed for people with dementia/mild cognitive impairment [14]. Previous studies have found barriers associated with the use of ICTs in older adults, including age and education level [15]. Likewise, barriers for the use of technology in people with dementia/mild cognitive impairment have been described, including negative attitudes toward ICTs, inaccurate perceptions of ICTs, and poor technology knowledge [16]. Although the attitudes and knowledge of ICTs and touchscreen technologies could determine the use of health apps, to the best of our knowledge, few large-sample studies  have investigated the attitude and use of these technologies in older adults, including people with dementia/mild cognitive impairment and their caregivers. Technophilia is one of the emerging concepts regarding attitudes toward technologies. One of the definitions of technophilia is the “attraction, enthusiasm of the human individual determined by the activities which involve the use of advanced technologies. It is expressed by easy adaptation to the social changes brought by technological innovations” [17].

The aim of this study was to analyze technophilia in people with dementia/mild cognitive impairment and their caregivers and to determine the sociodemographic and clinical factors associated with technophilia. Another aim was to analyze how this population uses smartphones, tablets, apps, and software to support their memory, and to identify factors associated with the use of apps to support memory. We tested the hypothesis that different sociodemographic and clinical factors in people with dementia/mild cognitive impairment and their caregivers are associated with technophilia and the use of specific apps to support memory in tablets or smartphones.


Study Design

In this cross-sectional study, data from the baseline assessment of the Support Monitoring and Reminder for Mild Dementia (SMART4MD) randomized multicenter clinical trial (ClinicalTrials.gov NCT03325699) were used. The objective of the trial was to create a digital platform (SMART4MD) for a tablet and to test if the platform had an impact on the quality of life of people with dementia/mild cognitive impairment and their caregivers. More detailed information on the trial is available in the published protocol [18].

Setting

This study was carried out in two European countries, Spain and Sweden, and at three centers: Consorci Sanitari de Terrassa (Catalonia, Spain), Servicio Andaluz de Salud (Andalusia, Spain), and the Blekinge Institute of Technology (Sweden).

Participants

A total of 1086 participant dyads (N=2172) were included in the study. The participant dyads comprised people with dementia/mild cognitive impairment and their informal caregivers. The participants were selected using a nonprobabilistic consecutive sampling method. The inclusion criteria were as follows: (1) score of 20 to 28 points on the Mini Mental State Examination (MMSE), (2) experience of memory problems over a substantial period of time (more than 6 months), (3) aged>55 years, (4) recipients of home care, (5) have an informal caregiver, (6) taking prescribed medication and in charge of it, and (7) no physical conditions that would reduce their ability to use a touchscreen app. The exclusion criteria were as follows: (1) terminal illness with less than 3 years of expected survival, (2) score above 11 on the Geriatric Depression Scale (GDS-15), or (3) another known significant cause of disease as an explanation for cognitive impairment such as substance abuse, bipolar disorder, schizophrenia, or developmental disorders.

Measures

Dependent Variables

The dependent variables were technophilia and use and familiarity with touchscreen devices. The TechPH questionnaire was used to assess technophilia [19]. This questionnaire includes 6 items assessed on a 5-point Likert scale from 1 (fully disagree) to 5 (fully agree), which was designed to specifically assess technophilia in the older population. The scale has two factors: technology enthusiasm and technology anxiety. The TechPH index is a score derived from the 6 items (the sum of items divided by 6) ranging from 1 to 5. More information on the TechPH index is included in the report on the validation study [19].

Use and familiarity with touchscreen devices was assessed with a questionnaire tailored to this study covering the following aspects: (1) use of a smartphone and tablets (“On average how often would you say you have been using a smartphone or tablet during the last 3 months?”), (2) use of the internet (“How often do you use the internet on your smartphone or tablet?”), (3) knowledge (“How knowledgeable do you consider yourself when it comes to using a smartphone or a tablet?”), (4) use of technology to support memory (“Are you using your mobile phone or tablet as a way to support your memory today?”), (5) use of specific apps for memory (“Do you have any special app or software on your mobile phone or tablet that you use to support your memory?”), and (6) perspective on the helpfulness of the technology for their memory (“Do you think that using your mobile phone or tablet to support your memory helps you to remember things?”).

Independent Variables in People With Dementia/Mild Cognitive Impairment

Health-related quality of life was measured using the total score of the Quality of Life in Alzheimer's Disease (QoL-AD) questionnaire [20], which is a 13-item measure with a 4-point Likert scale. The EuroQoL-5D-3L [21] (EQ5D) questionnaire was also administered, which is a self-completion questionnaire that consists of 5 questions along with a scale for the participant to rate their health state on a scale thermometer of 0 to 100 (EQ-VAS). The European value set of Köning et al [22] was used for calculation of the EQ5D score.

Functional decline was assessed with the Lawton Instrumental Activities of Daily Living (IADL) [23] scale in people with dementia/mild cognitive impairment. The IADL uses 8 items for women and 5 items for men. The score was rescaled to range between 0 and 1.

The severity of cognitive impairment was assessed with the MMSE [24]. Depression was scored with the GDS-15 [25], which is a widely used scale to assess geriatric depression with 15 items and a range of 0-4 (normal), 5-8 (mildly depressed), 9-11 (moderately depressed), and 12-15 (severely depressed). Since an inclusion criterion for study participation was a GDS-15 score<11, the range in this study was 0–11.

Sociodemographic data included age, gender, education, living arrangement, marital status, and country of residence. Medical history included a diagnosis of dementia and comorbidity based on the International Classification of Diseases-10 [26].

Independent Variables for Caregivers

The independent variables for caregivers included the caregiver burden, which was assessed using the Zarit Caregiver Burden Interview (ZBI-12) [27,28], a 12-item 5-point Likert scale questionnaire. This is the short version of the original scale and was designed specifically for caregivers of people with dementia/mild cognitive impairment. Health-related quality of life was measured using the EQ5D as described above. The sociodemographic data included age, gender, education, living arrangements, marital status, country of residence, and relationship with the patient.

Statistical Analysis

The mean (SD) and frequency and percentages are used to describe continuous and categorical variables, respectively. To compare groups with high and low technophilia, we used the median TechPh Index (2.83 in people with dementia/mild cognitive impairment and 3.00 in caregivers) as the cut-off point. To compare groups, we used the Chi square test for categorical variables and the Mann-Whitney test for continuous variables. A multivariate linear regression analysis was conducted to analyze the factors associated with technophilia. To check the assumptions of the linear model, the Breusch-Pagan test (homoscedasticity), Shapiro-Wilk normality test of the residuals, variance inflation factor (VIF; multicollinearity), and a scatter plot for the linearity of the variables were used. We also analyzed the factors associated with the use of specific apps or software to support memory using multivariate logistic regression. To check the goodness of fit, we used the Hosmer-Lemeshow test and the VIF for multicollinearity. If the VIF was higher than 2, the variable was taken out of the model. A 95% level of significance was used for assessment. R program version 3.6.1 (R Foundation for Statistical Computing, Vienna, Austria) and the R Commander package were used for these analyses.


The basic characteristics of the sample of people with dementia/mild cognitive impairment are summarized in Table 1. Nearly a third of the sample were diagnosed with dementia. A total of 588 of the 1086 people with dementia/mild cognitive impairment (54.14%) used smartphones and tablets almost every day and 284 (26.15%) had never used these technologies. Only 381/1086 (35.08%) used the internet on smartphones and tablets almost every day and 470/1086 (43.28%) never used the internet on these gadgets. More than half (706/1086, 65.01%) considered themselves not at all or quite knowledgeable when it comes to using a smartphone or a tablet. A total of 106 (9.76%) had a special app or software on their mobile phone or tablet to support memory, and 669/1086 (61.60%) believed that using a mobile phone or tablet to support their memory helps them remember things. Additional information about the use of smartphones and tablets by the people with dementia/mild cognitive impairment is provided in Multimedia Appendix 1. Information about groups with high and low technophilia and groups that use or do not use specific apps or software on their mobile phone or tablet to support memory is shown in Table 1.

Table 1. Characteristics of participants with dementia or mild cognitive impairment.
VariablesAll participants (N=1086)aHigh technophilia (N=591)bLow technophilia (N=493)cP valueUse apps to support memory (N=106)dDo not use apps to support memory (N=877)eP value
Age (years), mean (SD)74.48 (7.24)74.32 (7.32)74.57 (7.16).7270.22 (7.46)74.76 (7.11)<.001
Gender, n (%)


<.001

.29

Female576 (53.0)276 (46.7)298 (60.4)
51 (48.1)470 (53.6)

Male510 (47.0)315 (53.3)195 (39.6)
55 (51.9)407 (46.4)
Education level, n (%)


.003

.03

Elementary school653 (60.4)

331 (56.3)320 (65.2)
55 (51.9)527 (60.4)

Secondary school225 (20.8)126 (21.4)99 (20.2)
21 (19.8)190 (21.8)

Higher education203 (18.8)131 (22.2)72 (14.7)
30 (28.3)155 (17.8)
Marital status, n (%)


.06

.06

Unmarried41 (3.8)20 (3.4)21 (4.3)
6 (5.7)31 (3.5)

Married697 (64.2)377 (43.9)320 (64.9)
75 (70.8)555 (63.4)

Common law partner44 (4.1)33 (5.6)11 (2.2)
7 (6.6)35 (4.0)

Divorced58 (5.3)34 (5.8)24 (4.9)
5 (4.7)49 (5.6)

Widowed245 (22.6)126 (21.4)117 (23.7)
13 (12.3)206 (23.5)
Living arrangement, n (%)


.22

.05

Single222 (20.6)109 (18.6)113 (23.1)
19 (17.9)177 (20.4)

Spouse/partner686 (63.6)

385 (65.7)301 (61.6)


73 (68.9)549 (63.2)

Children97 (9.0)57 (9.7)40 (8.2)
3 (2.8)85 (9.8)

Other73 (6.8)36 (6.1)35 (7.2)
11 (10.4)58 (6.7)
Country of residence, n (%)


<.001

.12

Sweden345 (31.8)231 (39.1)114 (23.1)
39 (36.8)258 (29.4)

Spain741 (68.2)360 (60.9)379 (76.9)
67 (63.2)619 (70.6)
Diagnosis of dementia, n (%)


.73

.007

Yes299 (28.5)165 (28.8)132 (27.8)
87 (82.9)594 (70.2)

No750 (71.5)408 (71.2)342 (72.2)
18 (17.1)252 (29.8)
Diagnosis of other medical condition, n (%)


.04

.96

Yes267 (24.6)431 (72.9)386 (78.3)
26 (24.5)213 (24.3)

No819 (75.4)160 (27.1)107 (21.7)
80 (75.5)664 (75.7)
Cognitive Status (MMSEf), mean (SD)

25.41 (2.48)25.51 (2.53)25.32 (2.40).0525.97 (2.05)25.30 (2.56).03
Depression (GDS-15g), mean (SD)3.03 (2.84)2.47 (2.53)3.70 (3.05)<.0013.06 (3.01)3.11 (2.84).53
Instrumental activities (IADLh), mean (SD)0.87 (0.19)0.87 (0.20)

0.86 (0.19).070.91 (0.17)0.86 (0.19).008
Quality of life (QoL-ADi), mean (SD)36.08 (6.52)37.67 (6.31)34.21 (6.27)<.00136.93 (7.32)36.00 (6.49).21
Quality of life (EQ5Dj), mean (SD)0.75 (0.22)0.78 (0.22)0.72 (0.22)<.0010.69 (0.25)0.75 (0.22).08
Health State thermometer, mean (SD)69.45 (19.60)72.98 (18.70)65.27 (19.88)<.00170.67 (19.03)69.22 (19.80).43

aEducation level N=1081; Living arrangement N=1078; Marital status N=1085; Diagnosis of dementia N=1049.

bEducation level N=588; Marital status N=590; Living arrangement N=586; Diagnosis of dementia N=573.

cEducation level N=491; Living arrangement N=489; Diagnosis of dementia N=474.

dDiagnosis of dementia N=105.

eEducation level N=872; Marital status N=876; Living arrangement N=869; Diagnosis of dementia N=846.

fMMSE: Mini Mental State Examination.

gGDS-15: Geriatric Depression Scale.

hIADL: Lawton Instrumental Activities of Daily Living.

iQoL-AD: Quality of Life in Alzheimer’s Disease.

jEQ5D: EuroQoL-5D-3L.

The basic characteristics of the caregivers are summarized in Table 2. Among the 1086 caregivers, 839 (77.26%) used touchscreen technologies almost every day, and 123 (11.33%) had never used smartphones or tablets. A total of 721 (66.39%) used the internet on smartphones and tablets almost every day, and 191 (17.59%) never used the internet on touchscreen gadgets. A total of 433 (39.87%) considered themselves not at all or quite knowledgeable when it comes to using a smartphone or a tablet. A total of 181 (16.67%) had a special app or software on their mobile phone or tablet to support their memory, and 773 (71.18%) believed that using a mobile phone or tablet to support their memory helps them to remember things. Additional information about the use of smartphones and tablets by the caregivers is provided in Multimedia Appendix 1. Information about the caregivers overall, groups with high and low technophilia, and groups that use or do not use specific apps or software on their tablet or mobile phone to support memory is shown in Table 2.

Table 2. Characteristics of caregivers.
VariablesTotal (N=1086)aHigh technophilia (N=464)bLow technophilia (N=598)cP valueUse apps to support memory (N=181)dDo not use apps to support memory (N=840)eP value
Age (years), mean (SD)62.29 (14.68)58.35 (15.84)65.24 (12.95)<.00154.27 (14.49)63.34 (14.20)<.001
Gender, n (%)


.003

.11

Female741 (68.2)295 (63.6)167 (27.9)
133 (73.5)566 (67.4)

Male345 (31.8)169 (36.4)431 (72.1)
48 (26.5)274 (32.6)
Education level, n (%)


<.001

<.001

Elementary school378 (35.7)111 (24.3)257 (44.2)
29 (16.2)318 (38.8)

Secondary school345 (32.5)167 (36.6)173 (29.3)
73 (40.8)256 (31.3)

Higher education337 (31.8)178 (39.0)152 (26.1)
77 (43.0)245 (29.9)
Marital status, n (%)


<.001

.005

Unmarried126 (11.6)75 (16.2)46 (7.7)
33 (18.2)88 (10.5)

Married804 (74.2)312 (67.4)475 (79.6)
121 (66.9)633 (75.5)

Common law partner76 (7.0)38 (8.2)37 (6.2)
15 (8.3)55 (6.6)

Divorced47 (4.3)23 (5.0)24 (4.0)
11 (6.1)34 (4.1)

Widowed30 (2.8)15 (3.2)15 (2.5)
1 (0.5)28 (3.3)
Living arrangement, n (%)


.005

.30

Single98 (9.0)50 (10.8)45 (7.5)
17 (9.4)75 (8.9)

Spouse/partner738 (68.1)291 (62.9)436 (73.0)
112 (61.9)576 (68.7)

Children100 (9.2)52 (11.2)45 (7.5)
21 (11.6)77 (9.2)

Other147 (13.6)70 (15.1)71 (11.9)
31 (17.1)110 (13.1)
Country of residence, n (%)


.03

.05

Sweden345 (31.8)167 (36.0)177 (29.6)
44 (24.3)266 (31.7)

Spain741 (68.2)297 (64.0)421 (70.4)
137 (75.7)574 (68.3)
Relation with the patient, n (%)


<.001

<.001

Spouse/partner590 (55.5)204 (44.7)374 (63.9)
59 (33.7)483 (58.6)

Child320 (30.1)179 (39.3)134 (22.9)
97 (55.4)214 (26.0)

Other153 (14.4)73 (16.0)77 (13.2)
19 (10.9)127 (15.4)
Caregiver Burden (ZBI-12f), mean (SD)6.86 (7.70)6.31 (7.21)7.28 (8.08).139.02 (7.62)6.53 (7.68)<.001
Quality of life (EQ5Dg), mean (SD)0.77 (0.21)0.81 (0.20)0.75 (0.22)<.0010.77 (0.22)0.78 (0.21).72
Health State (thermometer), mean (SD)72.24 (18.62)74.95 (17.36)70.07 (19.22)<.00170.94 (18.21)72.46 (18.67).22

aEducation level N=1060; Marital status N=1083; Living arrangement N=1083; Relation with the patient N=1063.

bEducation level N=456; Marital status N=463; Living arrangement N=463; Relation with the patient N=456.

cEducation level N=582; Marital status N=597; Living arrangement N=597; Relation with the patient N=585.

dEducation level N=179; Relation with the patient N=175.

eEducation level N=819; Marital status N=838; Living arrangement N=838; Relation with the patient N=824.

fZBI-12: Zarit Caregiver Burden Interview.

gEQ5D: EuroQoL-5D-3L.

The people with dementia/mild cognitive impairment had a mean TechPH index of 2.84 (SD 0.69). The TechEnthusiasm score was 2.95 (SD 1.07) and the TechAnxiety score was 3.30 (SD 0.95). The caregivers had a TechPH index score of 3.07 (SD 0.68); the TechEnthusiasm score was 3.31 (SD 1.00) and the TechAnxiety score was 3.19 (SD 0.95). There were differences in the TechPH index between patients and caregivers (B=0.223, SE 0.030, P<.001), which remained significant after adjusting by age, gender, education level, and health status (B=0.074, SE 0.032, P=.02). There were significant differences in TechAnxiety between patients and caregivers (B=–0.120, SE 0.041, P=0.004); however, these differences were not significant after adjusting for age, gender, education level, and health status (B=–0.042, SE 0.047, P=.37). There was a significant difference in TechEnthusiasm between patients and caregivers (B=0.360, SE 0.045, P<.001), and this difference was still significant after adjusting for confounders (B=0.128, SE 0.050, P=.01). There were significant differences in the use of specifics apps or software to support memory between patients and caregivers (odds ratio [OR]=1.783, SE 0.131, P<.001); however, these differences were not significant after adjusting for confounders (OR=0.818, SE 0.167, P=.23).

In the multivariate analysis, the TechPH index in people with dementia/mild cognitive impairment was related to lower age, male gender, higher education level, living arrangement (living with children vs single), country of residence (Sweden vs Spain), depression, and health status (Table 3). In caregivers, the TechPH index was associated with lower age, male gender, higher education level, country of residence (Sweden vs Spain), lower caregiver burden, and better quality of life (Table 4). The use of specific apps or software on tablets or mobile phones to support memory was only associated with age in people with dementia/mild cognitive impairment (Table 5) and was associated with age, education level, and caregiver burden in caregivers (Table 6).

Table 3. Multivariate linear regression modela of factors associated with technophilia in people with dementia/mild cognitive impairment.
VariablesEstimateSE t1008P valueVIFb
Intercept3.7630.4029.359<.001
Age–0.0090.003–2.850.0041.248
Gender: female (male=reference)–0.1600.044–3.630<.0011.211
Education level (elementary school=reference)


.011.252

Secondary school0.0210.0520.400.69

Higher education0.1670.0582.886.004
Living arrangement (children=reference)


.011.342

Single–0.2100.083–2.538.01

Spouse/partner–0.1490.074–1.999.05

Other–0.1370.103–1.328.18
Country: Sweden (Spain=reference)0.2560.0584.414<.0011.855
Diagnosis of dementia: Yes (no=reference)0.0600.0541.116.261.483
Diagnosis of other medical condition: Yes (no=reference)0.0470.0500.939.351.180
Cognitive Status (MMSEc)–0.0130.010–1.356.171.438
Depression (GDS-15d)–0.0460.008–5.625<.0011.360
Instrumental activities (IADLe)–0.0200.125–0.164.871.450
Health State (Thermometer)0.0040.0013.570<.0011.271

aBreusch-Pagan test P=.06; Shapiro-Wilk normality test of the residuals P=0.39; adjusted R2=0.146.

bVIF: variance inflation factor.

cMMSE: Mini Mental State Examination.

dGDS-15: Geriatric Depression Scale.

eIADL: Lawton Instrumental Activities of Daily Living.

Table 4. Multivariate linear regression modela of factors associated with technophilia in caregivers.
VariablesEstimateSE t1021P valueVIFb
Intercept3.8240.15924.014<.001
Age–0.0160.002–9.560<.0011.603
Female gender (male=reference)–0.1910.043–4.470<.0011.081
Education level (elementary school=reference)


<.0011.278

Secondary school0.0840.0501.677.01

Higher education0.2010.0523.879<.001
Living arrangement (children=reference)


.271.379

Single0.0500.0920.537.59

Spouse/partner–0.0620.074–0.843.40

Other–0.0940.083–1.128.26
Country: Sweden (Spain=reference)0.2450.0485.140<.0011.375
Caregiver Burden (ZBI-12c)–0.0050.003–2.055.041.154
Quality of life (EQ5Dd)0.3480.0973.592<.0011.165

aBreusch-Pagan test P=.07; Shapiro-Wilk normality test of the residuals P=0.84; adjusted R2=0.194.

bVIF: variance inflation factor.

cZBI-12: Zarit Caregiver Burden Interview.

dEQ5D: EuroQoL-5D-3L.

Table 5. Multivariate logistic regression modela of factors associated with use of apps or specific software to support memory in people with dementia/mild cognitive impairment.
VariablesEstimateSEzP valueORb95% CIVIFc
Intercept3.1892.2471.419.16


Age–0.0930.017–5.387<.0010.9110.880-0.9421.366
Gender: female (male=reference)–0.1190.233–0.512.610.8770.562-1.4021.183
Education level (elementary school= reference)


.08

1.362

Secondary school–0.1450.292–0.496.620.8650.488-1.534

Higher education0.5480.2901.886.061.7290.979-3.056
Living arrangement (children=reference)


.16

1.357

Single1.2070.6631.822.073.3450.912-12.262

Spouse/partner1.0900.6251.745.082.9750.874-10.121

Other1.3470.6991.926.053.8440.976-15.139
Country: Sweden (Spain=reference)0.3190.3121.019.311.3750.745-2.5381.990
Diagnosis of dementia: Yes (No=reference)–0.3910.330–1.187.230.6760.354-1.2901.431
Diagnosis of other medical condition: Yes (No=reference)0.1680.2620.642.521.1830.708-1.9771.132
Cognitive Status (MMSEd)–0.0010.056–0.022.980.9990.894-1.1151.373
Depression (GDS-15e)–0.0080.047–0.165.870.9920.906-1.0871.565
Instrumental activities (IADLf)0.1650.7440.222.821.1800.274-5.0741.360
Health State (Thermometer)0.0020.0060.367.711.0020.990-1.0151.411

aHosmer and Lemeshow goodness of fit test P=.23.

bOR: odds ratio.

cVIF: variance inflation factor.

dMMSE: Mini Mental State Examination.

eGDS-15: Geriatric Depression Scale.

fIADL: Lawton Instrumental Activities of Daily Living.

Table 6. Multivariate logistic regression modela of factors associated with use of apps or specific software to support memory in caregivers.
VariablesEstimateSEzP valueORb95% CIVIFc
Intercept0.0890.6830.131.90


Age–0.0380.007-5.441<.0010.9620.949-0.9761.519
Female gender (male=reference)0.0690.1990.346.731.0720.725-1.5851.068
Education level (elementary school=reference)


<.001

1.180

Secondary school0.8280.2523.279.0012.2891.395-3.754

Higher education0.9600.2573.738<.0012.6111.579-4.319
Living arrangement (children=reference)


.43



1.375

Single0.0800.3860.208.831.0830.508-2.311

Spouse/partner0.3320.2981.113.271.3940.777-2.502

Other–0.0520.337–0.153.880.9500.491-1.838
Country: Sweden (Spain=reference)–0.1260.225–0.561.570.8810.567-1.3701.327
Caregiver Burden (ZBI-12d)0.0230.0112.084.041.0231.001-1.0461.141
Quality of life (EQ5De)–0.5400.429–1.259.210.5820.251-1.3511.194

aHosmer and Lemeshow goodness of fit test P=.68.

bOR: odds ratio.

cVIF: variance inflation factor.

dZBI-12: Zarit Caregiver Burden Interview.

eEQ5D: EuroQoL-5D-3L.


In people with dementia/mild cognitive impairment, technophilia was associated with less depression, better health status, as well as with sociodemographic variables. In caregivers, technophilia was associated with a better quality of life, less care burden, and other sociodemographic variables. The results of this study indicate that people with dementia/mild cognitive impairment have less technophilia than their caregivers, specifically less TechEnthusiasm, but no differences were found in TechAnxiety. The people with dementia/mild cognitive impairment used smartphones and tablets with specific apps or software to support their memory less than their caregivers, despite being a population who would benefit most from these apps. These differences are likely due to age barriers.

The use of smartphones and tablets in our study was lower than that reported in a recent study in Australia in which 91.4% of people with dementia/mild cognitive impairment reported routinely using smartphones [29]. Likewise, in other studies that included older adults and populations with cognitive impairment and dementia, attitudes toward technology were associated with factors such as age [30-32], male gender [33], higher education level [29,30,33,34], depression and negative cognitions [35], and health status [30], and variability was found between countries [36]. A correlation between the use of health apps and age has also been reported [32,37]. However, some results in other studies have not been replicated. For example, one study reported an association between cognitive functioning and the use of technologies in a cognitively impaired population [38], and another study found that women used more health apps than men [39].

The differences in technophilia between people with dementia/mild cognitive impairment and caregivers, mainly due to the lack of technology enthusiasm, could be related to dementia or mild cognitive impairment itself. In fact, other studies have found that dementia and mild cognitive impairment were related to more perceived difficulties in everyday technology use [40]. In addition, this lack of enthusiasm for technology could be related to the apathy associated with dementia and mild cognitive impairment, which is a persistent behavioral symptom [41,42]. This apathy could affect the enthusiasm toward new technologies and may be a barrier for interventions that demand learning and adaptation to the use of these technologies.

Other studies have also found that barriers to the use of ICTs and assistive technologies in older adults are due to the lack of “interest or relevance to life” and the perception of “no need” [15,43]. Indeed, these ICTs are often not designed specifically to cover the real needs of this population and they do not arouse interest. Other studies have also found that a lack of knowledge in older adults is a barrier to the use of technologies [11,16]. In addition, age was identified as one of the most critical determinants of the use of apps to support memory after adjusting for health status. This result coincides with other studies [15,30] in the general population.

Interestingly, technophilia was also strongly associated with health status and depression after adjusting for possible confounders such as gender, age, education level, and cognitive function. One longitudinal study conducted by Cotten et al [44] and other studies [45,46] have found that use of the internet and ICTs in older individuals is associated with less depression and a better quality of life, and there is some evidence that this association is mediated by loneliness and social isolation [44,45]. In conclusion, these results highlight the potential barriers of interventions for the use of ICTs, including smartphones and tablets, to improve the quality of life of people with dementia/mild cognitive impairment. The nature of the relation between use and technophilia and quality of life and mood needs to be clarified with longitudinal and experimental studies.

The use of specific apps or software to support memory was not associated with better cognitive function, better quality of life, or less depression. However, some positive results have emerged from several meta-analyses reporting that computerized cognitive training and electronic health apps are effective in improving cognition and quality of life in people with dementia/mild cognitive impairment [47,48]. This result comes from formal interventions, and the results of the present study could be due to possible confounders not assessed in the study or because the participants were not using the apps or software adequately (eg, insufficient training time or not using well-designed apps). Other studies have also indicated inconsistencies and lack evidence regarding the effectiveness of ICT interventions to improve cognition and other health-related variables [49,50].

The results of this study need to be considered in the context of several limitations. This was a cross-sectional study, and therefore the causal relation of the variables could not be established. More longitudinal and interventional studies are necessary to determine the effects of ICT use and the attitudes toward technology in people with dementia/mild cognitive impairment and their caregivers and to identify the factors that influence these variables in the other direction. In addition, this was a secondary analysis from a clinical trial, and the sample may not be representative of all people with dementia/mild cognitive impairment and caregivers. Individuals with severe depression were excluded and more than half of the sample was from only one country. There was no established cut-off point for the definition of high or low technophilia, which limits the use of the mean as the cutoff. Another limitation is that only a small set of variables was assessed. However, the strength of the study is the large sample used with more than 2000 individuals.

In conclusion, the factors associated with technophilia suggest potential barriers to technological interventions in people with dementia/mild cognitive impairment and their caregivers. These results have implications on the possible usefulness of considering technophilia and enthusiasm toward ICTs as determinants and moderator elements in digital interventions in the elderly population. The results suggest that designers of apps for older adults with cognitive impairment should create apps that engage users and are designed specifically for their needs as this population frequently lacks enthusiasm for technology. The use of touchscreen technologies was also frequent among people with dementia/mild cognitive impairment, whereas the use of specific apps or software to support memory was reduced in people with dementia/mild cognitive impairment and caregivers, and they face age barriers for the use of these apps. However, the majority of this population considers that use of smartphones or tablets is helpful for memory, which highlights the presence of a gap between the perceived potential and actual use of these technologies.

Acknowledgments

The study is supported by a grant from European Commission, Horizon 2020 (code 643399). This work is a collaboration among several members of the SMART4MD consortium. The consortium acknowledges the contributions of members not reaching full ICMJE standards for authorship in this manuscript: Karen Hayden, Tony Fenton, Jufen Zhang, Selim Cellek, Claire Preston, Tomer Regev, and Stephen Moore (Anglia Ruskin University, UK); Dominique Manhaeve and Jessie Schrijvers (University Colleges Leuven-Limburg, Belgium); Professor María Teresa Arredondo Waldmeyer (Universidad Politécnica de Madrid, UPM, Spain); Ifty Ahmed (Healthbit Ltd, UK); Michal Stefan, Karel Huk, and Jan Kubalik (DEX innovation Center, Czech Republic); Lisa Seeman (Athena ICT, Israel); and Carmen Sanmartin, Line Christensson, Eric Picullel, Joakim Frögren, and Doris Bohman (Blekinge Institute of Technology, Sweden).

Conflicts of Interest

None declared.

Multimedia Appendix 1

Detailed data on use of smartphones and tablets by people with dementia/mild cognitive impairment (PwD/MCI).

DOCX File , 14 KB

References

  1. World Health Organization and Alzheimer’s Disease International. 2012. Dementia: a public health priority Internet   URL: https://www.who.int/mental_health/publications/dementia_report_2012/en/ [accessed 2019-11-01]
  2. Forrester SN, Gallo JJ, Smith GS, Leoutsakos JS. Patterns of Neuropsychiatric Symptoms in Mild Cognitive Impairment and Risk of Dementia. Am J Geriatr Psychiatry 2016 Feb;24(2):117-125 [FREE Full text] [CrossRef] [Medline]
  3. Cerejeira J, Lagarto L, Mukaetova-Ladinska EB. Behavioral and psychological symptoms of dementia. Front Neurol 2012;3:73. [CrossRef] [Medline]
  4. Szeto JYY, Lewis SJG. Current Treatment Options for Alzheimer's Disease and Parkinson's Disease Dementia. Curr Neuropharmacol 2016;14(4):326-338 [FREE Full text] [CrossRef] [Medline]
  5. Khosravi P, Ghapanchi AH. Investigating the effectiveness of technologies applied to assist seniors: A systematic literature review. Int J Med Inform 2016 Jan;85(1):17-26. [CrossRef] [Medline]
  6. Tyack C, Camic PM. Touchscreen interventions and the well-being of people with dementia and caregivers: a systematic review. Int Psychogeriatr 2017 Aug;29(8):1261-1280. [CrossRef] [Medline]
  7. Berner J, Comijs H, Elmståhl S, Welmer A, Sanmartin Berglund J, Anderberg P, et al. Maintaining cognitive function with internet use: a two-country, six-year longitudinal study. Int Psychogeriatr 2019 Aug 9;31(07):929-936. [CrossRef]
  8. Koo BM, Vizer LM. Examining Mobile Technologies to Support Older Adults With Dementia Through the Lens of Personhood and Human Needs: Scoping Review. JMIR Mhealth Uhealth 2019 Nov 11;7(11):e15122 [FREE Full text] [CrossRef] [Medline]
  9. Grossman MR, Zak DK, Zelinski EM. Mobile Apps for Caregivers of Older Adults: Quantitative Content Analysis. JMIR Mhealth Uhealth 2018 Jul 30;6(7):e162 [FREE Full text] [CrossRef] [Medline]
  10. Klimova B. Mobile Phone Apps in the Management and Assessment of Mild Cognitive Impairment and/or Mild-to-Moderate Dementia: An Opinion Article on Recent Findings. Front Hum Neurosci 2017;11:461. [CrossRef] [Medline]
  11. Klimova B, Simonova I, Poulova P, Truhlarova Z, Kuca K. Older people and their attitude to the use of information and communication technologies ? A review study with special focus on the Czech Republic (older people and their attitude to ICT). Educ Gerontol 2016;42(5):361-369. [CrossRef]
  12. Joe J, Demiris G. Older adults and mobile phones for health: a review. J Biomed Inform 2013 Oct;46(5):947-954 [FREE Full text] [CrossRef] [Medline]
  13. Hitch D, Swan J, Pattison R, Stefaniak R. Use of touchscreen tablet technology by people with dementia in homes: A scoping review. J Rehabil Assist Technol Eng 2017;4:2055668317733382 [FREE Full text] [CrossRef] [Medline]
  14. Zhang MWB, Chan S, Wynne O, Jeong S, Hunter S, Wilson A, et al. Conceptualization of an evidence-based smartphone innovation for caregivers and persons living with dementia. Technol Health Care 2016 Sep 14;24(5):769-773. [CrossRef] [Medline]
  15. Selwyn N, Gorard S, Furlong J, Madden L. Older adults' use of information and communications technology in everyday life. Age Soc 2003 Sep 02;23(5):561-582. [CrossRef]
  16. O'Connor S, Bouamrane M, O'Donnell CA, Mair FS. Barriers to Co-Designing Mobile Technology with Persons with Dementia and Their Carers. Stud Health Technol Inform 2016;225:1028-1029. [Medline]
  17. Osiceanu M. Psychological Implications of Modern Technologies: “Technofobia” versus “Technophilia”. Procedia Soc Behav Sci 2015 May;180:1137-1144. [CrossRef]
  18. Anderberg P, Barnestein-Fonseca P, Guzman-Parra J, Garolera M, Quintana M, Mayoral-Cleries F, et al. The Effects of the Digital Platform Support Monitoring and Reminder Technology for Mild Dementia (SMART4MD) for People With Mild Cognitive Impairment and Their Informal Carers: Protocol for a Pilot Randomized Controlled Trial. JMIR Res Protoc 2019 Jun 21;8(6):e13711 [FREE Full text] [CrossRef] [Medline]
  19. Anderberg P, Eivazzadeh S, Berglund JS. A Novel Instrument for Measuring Older People's Attitudes Toward Technology (TechPH): Development and Validation. J Med Internet Res 2019 May 23;21(5):e13951 [FREE Full text] [CrossRef] [Medline]
  20. Logsdon RG, Gibbons LE, McCurry SM, Teri L. Quality of life in Alzheimer's disease: Patient and caregiver reports. Journal of Mental Health and Aging 1999;5(1):21-32 [FREE Full text]
  21. van Reenen M, Oppe M. EQ-5D. 2015. EQ-5D-3L user guide   URL: https://euroqol.org/publications/user-guides/ [accessed 2020-05-10]
  22. König H, Bernert S, Angermeyer MC, Matschinger H, Martinez M, Vilagut G, ESEMeD/MHEDEA 2000 Investigators. Comparison of population health status in six european countries: results of a representative survey using the EQ-5D questionnaire. Med Care 2009 Feb;47(2):255-261. [CrossRef] [Medline]
  23. Lawton MP, Brody EM. Assessment of older people: self-maintaining and instrumental activities of daily living. Gerontologist 1969;9(3):179-186. [Medline]
  24. Folstein MF, Folstein SE, McHugh PR. "Mini-mental state". A practical method for grading the cognitive state of patients for the clinician. J Psychiatr Res 1975 Nov;12(3):189-198. [CrossRef] [Medline]
  25. Yesavage JA, Brink TL, Rose TL, Lum O, Huang V, Adey M, et al. Development and validation of a geriatric depression screening scale: a preliminary report. J Psychiatr Res 1982;17(1):37-49. [CrossRef] [Medline]
  26. World Health Organization. 2004. ICD-10 : international statistical classification of diseases and related health problems : tenth revision, 2nd ed   URL: https://apps.who.int/iris/handle/10665/42980 [accessed 2020-05-10]
  27. Zarit SH, Reever KE, Bach-Peterson J. Relatives of the impaired elderly: correlates of feelings of burden. Gerontologist 1980 Dec;20(6):649-655. [CrossRef] [Medline]
  28. Bédard M, Molloy DW, Squire L, Dubois S, Lever JA, O'Donnell M. The Zarit Burden Interview: a new short version and screening version. Gerontologist 2001 Oct;41(5):652-657. [CrossRef] [Medline]
  29. LaMonica HM, English A, Hickie IB, Ip J, Ireland C, West S, et al. Examining Internet and eHealth Practices and Preferences: Survey Study of Australian Older Adults With Subjective Memory Complaints, Mild Cognitive Impairment, or Dementia. J Med Internet Res 2017 Oct 25;19(10):e358 [FREE Full text] [CrossRef] [Medline]
  30. Heart T, Kalderon E. Older adults: are they ready to adopt health-related ICT? Int J Med Inform 2013 Nov;82(11):e209-e231. [CrossRef] [Medline]
  31. Halmdienst N, Radhuber M, Winter-Ebmer R. Attitudes of elderly Austrians towards new technologies: communication and entertainment versus health and support use. Eur J Ageing 2019 Dec;16(4):513-523 [FREE Full text] [CrossRef] [Medline]
  32. Fischer SH, David D, Crotty BH, Dierks M, Safran C. Acceptance and use of health information technology by community-dwelling elders. Int J Med Inform 2014 Sep;83(9):624-635 [FREE Full text] [CrossRef] [Medline]
  33. Gell NM, Rosenberg DE, Demiris G, LaCroix AZ, Patel KV. Patterns of technology use among older adults with and without disabilities. Gerontologist 2015 Jun;55(3):412-421. [CrossRef] [Medline]
  34. Tacken M, Marcellini F, Mollenkopf H, Ruoppila I, Széman Z. Use and acceptance of new technology by older people. Findings of the international MOBILATE survey: 'Enhancing mobility in later life.'. Gerontechnology 2005;3(3):126-137. [CrossRef]
  35. Conrad AM, Munro D. Relationships between Computer Self-Efficacy, Technology, Attitudes and Anxiety: Development of the Computer Technology Use Scale (CTUS). J Educ Comput Res 2008 Oct 20;39(1):51-73. [CrossRef]
  36. König R, Seifert A, Doh M. Internet use among older Europeans: an analysis based on SHARE data. Univ Access Inf Soc 2018 Jan 19;17(3):621-633. [CrossRef]
  37. Rasche P, Wille M, Bröhl C, Theis S, Schäfer K, Knobe M, et al. Prevalence of Health App Use Among Older Adults in Germany: National Survey. JMIR Mhealth Uhealth 2018 Jan 23;6(1):e26 [FREE Full text] [CrossRef] [Medline]
  38. Hedman A, Nygård L, Almkvist O, Kottorp A. Amount and type of everyday technology use over time in older adults with cognitive impairment. Scand J Occup Ther 2015 May;22(3):196-206. [CrossRef] [Medline]
  39. Xie Z, Nacioglu A, Or C. Prevalence, Demographic Correlates, and Perceived Impacts of Mobile Health App Use Amongst Chinese Adults: Cross-Sectional Survey Study. JMIR Mhealth Uhealth 2018 Dec 26;6(4):e103 [FREE Full text] [CrossRef] [Medline]
  40. Rosenberg L, Kottorp A, Winblad B, Nygård L. Perceived difficulty in everyday technology use among older adults with or without cognitive deficits. Scand J Occup Ther 2009;16(4):216-226. [CrossRef] [Medline]
  41. Sherman C, Liu CS, Herrmann N, Lanctôt KL. Prevalence, neurobiology, and treatments for apathy in prodromal dementia. Int Psychogeriatr 2018 Feb;30(2):177-184. [CrossRef] [Medline]
  42. van der Linde RM, Matthews FE, Dening T, Brayne C. Patterns and persistence of behavioural and psychological symptoms in those with cognitive impairment: the importance of apathy. Int J Geriatr Psychiatry 2017 Mar;32(3):306-315. [CrossRef] [Medline]
  43. Yusif S, Soar J, Hafeez-Baig A. Older people, assistive technologies, and the barriers to adoption: A systematic review. Int J Med Inform 2016 Oct;94:112-116. [CrossRef] [Medline]
  44. Cotten SR, Ford G, Ford S, Hale TM. Internet use and depression among retired older adults in the United States: a longitudinal analysis. J Gerontol B Psychol Sci Soc Sci 2014 Sep;69(5):763-771. [CrossRef] [Medline]
  45. Khalaila R, Vitman-Schorr A. Internet use, social networks, loneliness, and quality of life among adults aged 50 and older: mediating and moderating effects. Qual Life Res 2018 Feb;27(2):479-489. [CrossRef] [Medline]
  46. Lee HY, Kim J, Sharratt M. Technology use and its association with health and depressive symptoms in older cancer survivors. Qual Life Res 2018 Feb;27(2):467-477. [CrossRef] [Medline]
  47. Hill NTM, Mowszowski L, Naismith SL, Chadwick VL, Valenzuela M, Lampit A. Computerized Cognitive Training in Older Adults With Mild Cognitive Impairment or Dementia: A Systematic Review and Meta-Analysis. Am J Psychiatry 2017 Apr 01;174(4):329-340. [CrossRef] [Medline]
  48. Coyle H, Traynor V, Solowij N. Computerized and virtual reality cognitive training for individuals at high risk of cognitive decline: systematic review of the literature. Am J Geriatr Psychiatry 2015 Apr;23(4):335-359. [CrossRef] [Medline]
  49. Butler M, McCreedy E, Nelson VA, Desai P, Ratner E, Fink HA, et al. Does Cognitive Training Prevent Cognitive Decline?: A Systematic Review. Ann Intern Med 2018 Jan 02;168(1):63-68. [CrossRef] [Medline]
  50. Ge S, Zhu Z, Wu B, McConnell ES. Technology-based cognitive training and rehabilitation interventions for individuals with mild cognitive impairment: a systematic review. BMC Geriatr 2018 Sep 15;18(1):213 [FREE Full text] [CrossRef] [Medline]


EQ5D: EuroQoL-5D-3L
EQ-VAS: Scale thermometer of 0-100
GDS-15: Geriatric Depression Scale
IADL: Lawton Instrumental Activities of Daily Living
ICT: information and communication technology
MMSE: Mini-Mental State Examination
OR: odds ratio
QoL-AD: Quality of Life in Alzheimer's Disease
SMART4MD: Support Monitoring and Reminder for Mild Dementia
VIF: variance inflation factor
ZBI-12: Zarit Caregiver Burden Interview


Edited by G Eysenbach; submitted 28.11.19; peer-reviewed by S Yusif, J Synnott, A Olsson; comments to author 09.01.20; revised version received 18.02.20; accepted 22.03.20; published 01.06.20

Copyright

©Jose Guzman-Parra, Pilar Barnestein-Fonseca, Gloria Guerrero-Pertiñez, Peter Anderberg, Luis Jimenez-Fernandez, Esperanza Valero-Moreno, Jessica Marian Goodman-Casanova, Antonio Cuesta-Vargas, Maite Garolera, Maria Quintana, Rebeca I García-Betances, Evi Lemmens, Johan Sanmartin Berglund, Fermin Mayoral-Cleries. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 01.06.2020.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.