Assessment of Frailty by the French Version of the Vulnerable Elders Survey-13 on Digital Tablet: Validation Study

Introduction

The concept of frailty has been widely studied in aging research for the past decades [1,2]. Frailty is characterized by a vulnerability related to advancing age, along with a reduction in the functional reserves of physiological systems that impairs responses to stress. Older individuals identified as frail are considered vulnerable in terms of their medical and social status, and compared to age-matched nonfrail people, they are more likely to experience geriatric deleterious events in the future [3]. Several studies found that frailty status is a significant predictor of decline in functional independence, onset of geriatric syndromes (such as falls, malnutrition, and depression), but also institutionalization, and even death [4-6]. Thus, the identification of frailty in older adults opens the way to targeted interventions to prevent or delay these geriatric events, whose individual and societal consequences are considerable [7-11]. These issues have been widely emphasized by the World Health Organization, which has initiated proactive programs to promote healthy and active aging [12,13].

The search for frailty is no longer restricted to the field of research, but is becoming part of routine care for an increasing number of geriatric teams, and the question of diagnostic strategies for this condition is clearly raised in these settings and also in primary care. Two instruments, the Fried Frailty Index [2] and the Rockwood Clinical Frailty Scale [14], are reference methods to diagnose frailty in older adults. Many other instruments have been developed to identify frailty in older individuals, and the benefits and limitations of these instruments are under debate [15,16]. However, these are relatively complex tools that are time-consuming and require a high level of geriatric expertise [9], making their use difficult for some geriatric teams and inappropriate for primary care. The Vulnerable Elders Survey-13 (VES-13) is a validated tool to identify frailty in older adults and is a predictor of the risk of functional decline, institutionalization, or mortality [17,18]. This tool is easy-to-use, brief (less than 5 minutes), and can be completed during a face-to-face interview or by telephone with the older individual or with his or her proxy. The scale has 13 simple items or questions, provides a score ranging from 0 to 10, and frailty is diagnosed by a score of 3 or more. The VES-13 can be completed without the need to involve health care staff, and for example, a trained secretary can interview the respondent or supervise the completion of the self-administrated questionnaire. Recently, the original English-language VES-13 has been validated in French, named Echelle de vulnérabilité des aînés-13 (EVA-13), and can now be used to assess frailty in France [19].

With the increasing use of informatics in health care, a number of studies have focused on the design and validation of digital versions of known screening or assessment instruments originally developed on paper. Various electronic media have been used in this research, such as computers, tactile tablets, and smartphones. Digital assessment tools have been developed for pain assessment [20,21], the function of the shoulder joint [22], food intake [23], quality of life measurement [24], dermatological monitoring [25], or erectile function [26]. These studies have documented the reliability of these digital instruments, their equivalence to the original version, and have evaluated the users’ experience with the original version. These digital instruments appear to be useful for screening or assessment because of their accuracy of detection, their ecological aspects, and their speed of execution, which saves time for physicians and can have positive effects on health [27]. Few studies have so far evaluated the feasibility and acceptability of a digital assessment tool in an older population.

This study aims to develop an electronic version of the French version of the VES-13 on a digital tablet. To do this, we will compare the results obtained by the original paper version of the French version of the VES-13 and its digital version, named Echelle électronique de vulnérabilité des ainés-13 (eEVA-13), with a population of older people and their caregivers. We wish to measure the reliability of the digital version in detecting frailty, but also its evaluation and acceptability, compared to a scale already used in its paper version in routine care.

Methods

Elaboration of eEVA-13

In this study, eEVA-13 was displayed on an Android tablet with a width of 14.9 cm (5.5 inches), a height of 25.8 cm (10.2 inches), a depth of 1.06 cm (0.42 inches), and a total weight of 515 g. This tablet has a 25.6 cm (10.1 inch) multitouch screen. A secured web application was elaborated to host the data on a personal health data host certified by the Agence Numérique en Santé (Microsoft Azure). It comprised secure web access (strong authentication) through a supervision console [28]. Once the authentication has been successfully completed by the research assistant, he or she identifies the data collector and the latter fills in the questionnaire in accordance with the illustrative input screens provided on the digital tablet. In designing the screens, emphasis was placed on the appearance and clarity of the messages. The answer possibilities are specified for each question, with attention paid to the visibility of the proposed answers.

The original scale was organized in 3 screens in eEVA-13. The 2 first questions (age group and health status) were displayed on the first screen (Figure 1), the 6 following questions were displayed on the second screen which requires scrolling (Figure 2), and the 5 last items on the third screen, which also require scrolling. For the items on the third screen inquiring about limitations in some physical activities, an additional question (“is it because of your health?”) unfolded for items answered with a limitation. The score was calculated automatically and displayed in a banner at the bottom of the third screen. The cutoff point defining frailty was a score of 3 or higher, as with the paper-and-pencil or telephone interview scales. The research technician prepared the access page for the electronic questionnaire in advance so that when the digital tablet was handed over to the participants, the form was already ready for input. In this access page, the research technician recorded if the respondent was the person assessed or a proxy, and noted an id code for the person assessed and his or her age (years).

‎

‎

Participants and Randomization

Participants were geriatric patients and family caregivers of geriatric patients of the Hôpital Charles Foix, an academic geriatric hospital located near Paris, France. From March 17, 2021, to June 28, 2021, we recruited consecutive patients attending the outpatient geriatric clinic and inpatients admitted to our rehabilitation ward. Eligible patients were included if they did not experience cognitive impairment defined by a Mini Mental Status Examination score above 22 and if they gave their written consent. During the same period, we also recruited a comparable number of family caregivers of geriatric patients attending the outpatient clinic or being hospitalized.

Each participant was assigned to 1 of 2 groups based on a randomization process stratified on the participant’s status (patient or caregiver). For the reference scale, participants in group 1 completed the self-administered paper version, and those in group 2 completed the scale as a telephone hetero-questionnaire (interview with a research technician). The eEVA-13 and the reference scale were completed within 2 days.

Data Collection

For all participants, we recorded their age, sex, status (patient or family caregiver), previous use of digital devices, and whether they owned a digital device and if so, what type. Participants were considered occasional users if they used a digital device less than once a week, and frequent users if it was more often. All participants completed 2 scales, the eEVA-13 on a digital tablet and the reference scale (self-administered paper questionnaire for group 1 participants, telephone interview for group 2 participants) in random order. Depending on the degree of comfort with the digital device, we also noted if some people required specific support to fill the eEVA-13. User experience was assessed using a self-administered questionnaire that included eight 5-level Likert-type questions, 2 questions about their preference, and 2 multiple choices questions recording positive and negative aspects of the eEVA-13.

Ethics Approval

The research has been approved by the French national ethics committee (Comité de Protection des Personnes Ile de France II; 2020-A02331-38). Written informed consent was obtained from all participants. Data were deidentified and processed in accordance with the European Union General data protection regulation. No compensation was provided to participants.

Statistical Analysis

Characteristic of patients and caregivers were compared using the Student t test and chi-square test. Agreement for diagnosis of frailty was assessed by the κ coefficient, and scores obtained at the eEVA-13 were compared to that obtained by the reference tools by Bland and Altman plots and interclass correlation coefficients. Statistics were realized using STATA (version 16.1; StataCorp).

Results

Selection and Characteristics of the Participants

During the study period, 124 people were offered the study and 37 of them declined to participate, including 31 patients and 6 family caregivers. Finally, 87 people were included in the study. One of them completed the eEVA-13, but did not complete the reference scale and, therefore, was not included in the analysis. Finally, 86 people completed the study and were included in the analysis. They comprised 40 patients and 46 caregivers (Figure 3).

‎

Participants’ characteristics are shown in Table 1. Family caregivers were significantly younger than the patients (mean 52.8, SD 11.1 years vs mean 79.9, SD 7.5 years, P<.001) and were more frequently owners of digital devices and more frequently users of digital devices. Indeed, all the family caregivers say they use digital devices very regularly. By contrast, among the 40 patients, 13 (32.5%) never used digital devices, 10 (25%) were occasional users and 17 (42.5%) were frequent users. All the caregivers had a smartphone, and 39 (87.9%) had a computer and 27 (67.5%) had a digital tablet. Among the patients, 29 had a smartphone (72.5%), 15 (37.5%) a computer, and 4 had a digital tablet (10%).

Feasibility and Support to Complete the eEVA-13

We observed no failure to complete the eEVA-13. Of the 40 patients, 28 (70%) needed support from the research assistant to complete the eEVA-13, while no caregivers needed support to complete the scale. The support consisted of an explanation of how works the digital tablet (21 cases) and help with navigation from one screen to another (18 cases), scrolling through the screen (23 cases), and selecting answers (24 cases).

Comparison of eEVA-13 With the Reference Scale

Agreement for the diagnosis of frailty was observed in 83 (96.5%) participants and disagreement was observed in 3 (3.5%) participants. The κ coefficient was 0.92, indicating an excellent level of agreement between eEVA-13 and the reference scales (Table 2).

The equivalence of the scores obtained by eEVA-13 and those obtained by original EVA-13 was assessed by 2 means. First, the intraclass correlation coefficient between eEVA-13 and paper EVA-13 was 0.997 (95% CI 0.994-0.998) and that between eEVA-13 and telephone interview EVA-13 was 0.977 (95% CI 0.957-0.988; Table 3). These results indicate a high degree of agreement between the different versions. Second, we plotted Bland and Altman graphs that showed that the differences between the scores from eEVA-13 and the original scale were small and were not related to the relative value of the scores (Figure 4). The mean difference between the scores provided by the 2 scales was 0.081 (95% CI –1.263 to 1.426).

‎

User’s Experience With eEVA-13

User’s experience with eEVA-13 assessed by questionnaire is reported in Table 4. The patients who participated in the study agreed overall (85%) that the original version of the instrument and the electronic version was similar, while among family caregivers this rate was 69%. If they were to complete the scale a second time, 85% of the family caregivers would like to do so in electronic form. As for the patients, 50% of them were divided between the electronic and the original mode. The digital version of the scale was considered easy to use overall (96% of caregivers and 77.5% of patients). The electronic version was considered the most pleasant to complete by 74% of the caregivers, while 67.5% of the patients preferred the original version.

Discussion

Principal Findings

This study shows that the digital version of the eEVA-13 is appropriate for identifying frailty among older adults and that the score it provides is similar to that obtained by the original paper or telephone version of the scale. The feasibility of the eEVA-13 scale was found to be excellent, although some older people who were unfamiliar with the use of digital objects required human assistance to use the digital tablet. According to the participants’ experience questionnaires, the digital scale was found easy to use and was generally appreciated.

Our study was conducted on a special population of frail and very old patients and their relatives. One out of 2 patients was over 80 years old and 2 out of 3 had criteria for frailty. The use of digital tablets for medical assessment has not been studied before in such a population to our knowledge and we observed that it is feasible and well appreciated by the participants.

We observed some differences in usability between patients and family caregivers. Although the digital version was considered easy to use by all participants, the family caregivers stood out by preferring the digital version as being the most pleasant, as well as choosing it for possible future use. The patients were a little more reluctant to use the digital tool to answer the health questionnaire. Among the negative points mentioned by the participants, we can observe that the interest in using the electronic mode by an older population is perceived negatively by 37.5% of the patients and 22% of the caregivers. Indeed, very old patients are sometimes uncomfortable or untrained in the use of a digital tool and prefer to be able to give information in a more traditional way, using a pen and paper or interacting directly with a person. Ten family caregivers spontaneously explained that using the eEVA-13 would be too complex for an older person. These observations are consistent with the inclusion difficulties we encountered during the study. In fact, we collected significantly more refusals from eligible patients (31) than from eligible caregivers (6). Looking at the reasons for refusal among patients (Figure 3), the one that stood out was apprehension about the digital domain, fear of not knowing how to do it, and lack of knowledge in this field. Age and previous familiarity with digital devices seem to be key elements to be considered carefully when proposing the use of an assessment by an electronic instrument.

Comparison to Prior Work

We found equivalence between the digital version of the scale and the original version of the scale, and our findings are similar to those of other teams that have compared digital and paper versions of other medical tools in other settings [20-26]. Casamali et al [24] studied the use of the World Health Organization quality of life questionnaire on computer in a sample of older adults younger than those in our study (mean age 66 years) and found it to be feasible and to provide similar scores to the original paper instrument. The disadvantages of electronic patient-reported outcomes measures (PROms) have been described in the literature as potentially leading to risks of a “digital divide.” Indeed, it has been observed in some studies that the advanced age of participants was a barrier to completing the electronic version of a questionnaire, compared to the paper version. In the case of older people with cancer or those who are not used to using digital devices, they needed time to get used to the devices or were more likely to encounter barriers when using them if they were not familiar with them beforehand [29].

Strengths and Limitations

This study was conducted using the recommendations of the International Society for Pharmacoeconomics and Outcomes Research, which emphasize the importance of validating electronic PROms against traditional PROms [30]. In addition, these recommendations provide methodological guidance for documenting equivalence between digital and traditional instruments [30]. As the transition from the original scale to eEVA-13 can be considered as a minor level of modification, our validation study was limited to exploring equivalence and usability, in agreement with the International Society for Pharmacoeconomics and Outcomes Research methodological guidance. We conducted a single-center study with a small sample size of 86 participants. These 2 characteristics represent the main limitations of this work.

Future Directions

The results obtained in this study show that it is possible to offer the eEVA-13 instrument as a self-questionnaire to family caregivers and older patients. We have started to use it in the waiting room of an outpatient department of a geriatric center, and it provides physicians with immediate information on the level of frailty of patients. In addition, digital technology might the advantage of saving time and processing and store the data instantly. Care should be taken with older patients who are unfamiliar with the use of digital devices, as they may require human assistance to fill the scale, or they may prefer to use the traditional paper scale. Despite these limitations, the eEVA-13 is an effective and innovative health support system for both family caregivers and patients. Certainly, the development of other PROms collected by digital devices is a promising avenue to facilitate geriatric assessment.