JMIR J Med Internet Res Journal of Medical Internet Research 1438-8871 JMIR Publications Toronto, Canada v23i8e24015 34420918 10.2196/24015 Review Review eHealth in Geriatric Rehabilitation: Systematic Review of Effectiveness, Feasibility, and Usability Eysenbach Gunther Skjaeret Nina Broekhuis Marijke Bhattacharjya Sutanuka Kraaijkamp Jules J M MSc 1
Department of Public Health and Primary Care Leiden University Medical Center Hippocratespad 21 Leiden, 2333 ZD Netherlands 31 610599869 J.J.M.Kraaijkamp@lumc.nl
2 https://orcid.org/0000-0001-7262-8399 van Dam van Isselt Eléonore F MD, PhD 1 https://orcid.org/0000-0002-5584-6295 Persoon Anke PhD 3 https://orcid.org/0000-0003-4581-4748 Versluis Anke PhD 1 https://orcid.org/0000-0002-9489-7925 Chavannes Niels H MD, PhD 1 https://orcid.org/0000-0002-8607-9199 Achterberg Wilco P MD, PhD 1 https://orcid.org/0000-0001-9227-7135 Department of Public Health and Primary Care Leiden University Medical Center Leiden Netherlands ZZG Zorggroep Nijmegen Netherlands Department of Primary and Community Care Radboud University Medical Center Nijmegen Nijmegen Netherlands Corresponding Author: Jules J M Kraaijkamp J.J.M.Kraaijkamp@lumc.nl 8 2021 19 8 2021 23 8 e24015 31 8 2020 19 12 2020 11 2 2021 16 5 2021 ©Jules J M Kraaijkamp, Eléonore F van Dam van Isselt, Anke Persoon, Anke Versluis, Niels H Chavannes, Wilco P Achterberg. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 19.08.2021. 2021

This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.

 Background

eHealth has the potential to improve outcomes such as physical activity or balance in older adults receiving geriatric rehabilitation. However, several challenges such as scarce evidence on effectiveness, feasibility, and usability hinder the successful implementation of eHealth in geriatric rehabilitation.

 Objective

The aim of this systematic review was to assess evidence on the effectiveness, feasibility, and usability of eHealth interventions in older adults in geriatric rehabilitation.

 Methods

We searched 7 databases for randomized controlled trials, nonrandomized studies, quantitative descriptive studies, qualitative research, and mixed methods studies that applied eHealth interventions during geriatric rehabilitation. Included studies investigated a combination of effectiveness, usability, and feasibility of eHealth in older patients who received geriatric rehabilitation, with a mean age of ≥70 years. Quality was assessed using the Mixed Methods Appraisal Tool and a narrative synthesis was conducted using a harvest plot.

 Results

In total, 40 studies were selected, with clinical heterogeneity across studies. Of 40 studies, 15 studies (38%) found eHealth was at least as effective as non-eHealth interventions (56% of the 27 studies with a control group), 11 studies (41%) found eHealth interventions were more effective than non-eHealth interventions, and 1 study (4%) reported beneficial outcomes in favor of the non-eHealth interventions. Of 17 studies, 16 (94%) concluded that eHealth was feasible. However, high exclusion rates were reported in 7 studies of 40 (18%). Of 40 studies, 4 (10%) included outcomes related to usability and indicated that there were certain aging-related barriers to cognitive ability, physical ability, or perception, which led to difficulties in using eHealth.

 Conclusions

eHealth can potentially improve rehabilitation outcomes for older patients receiving geriatric rehabilitation. Simple eHealth interventions were more likely to be feasible for older patients receiving geriatric rehabilitation, especially, in combination with another non-eHealth intervention. However, a lack of evidence on usability might hamper the implementation of eHealth. eHealth applications in geriatric rehabilitation show promise, but more research is required, including research with a focus on usability and participation.

 geriatric rehabilitation eHealth mHealth digital health effectiveness feasibility usability systematic review Introduction

The world’s population is aging rapidly. Currently, 143 million people are aged 80 years or older, and this number is expected to rise to around 426 million in 2050 [1]. Although many older adults are relatively fit, functional decline, multimorbidity, and geriatric syndromes such as frailty or falls are common in older adults [2,3]. A combination of these age-associated conditions triggers an increased risk of adverse outcomes such as hospitalization, functional impairments, and even mortality [4]. Postacute care such as geriatric rehabilitation aims to diminish these age-associated risks. Evidence shows that geriatric rehabilitation can improve functional outcomes and reduce nursing home admissions and mortality [5,6]. On the other hand, the rapidly aging populations and lack of staff are putting pressure on the quality, accessibility, and affordability of geriatric rehabilitation. In regard to these problems, the use of eHealth can be seen as important and promising, as it has the potential to simultaneously improve both rehabilitation outcomes and efficiency.

eHealth can be defined as “the use of digital information and communication to support and/or improve health and health care” [7]. Some examples of eHealth are video communication, exergames (ie, active video games), and mobile apps. Although eHealth can be seen as important and promising, successful implementation of eHealth interventions in geriatric rehabilitation is complex, can be time consuming, and involves a variety of determinants on multiple levels [8-10]. To safely and successfully implement eHealth in geriatric rehabilitation, scientific evaluation of eHealth is key [11,12]. Three important outcome measures for the evaluation of eHealth in geriatric rehabilitation can be identified: effectiveness, feasibility, and usability [9,13].

In terms of effectiveness, previous reviews show that eHealth can improve physical activity, gait, and balance in community-dwelling older adults [14-17]. However, the evidence on effective eHealth in geriatric rehabilitation is scarce and fragmented. To our knowledge, no prior reviews have examined the effectiveness of eHealth in geriatric rehabilitation.

To better understand how eHealth can be used safely, feasibility testing is an important first step [18,19]. The aim of feasibility testing is to “determine whether an intervention is appropriate for further testing” [20,21], but a general accepted standard on feasibility testing is lacking. Examples of factors that can be addressed in feasibility testing are adverse events, adherence, and acceptability [10].

Additionally, usable eHealth is also an important prerequisite for successful implementation [13,19,22]. Usability can be defined as “the extent to which a system, product, or service can be used by specified users to achieve specified goals with effectiveness, efficiency, and satisfaction in a specified context of use” [23]. For older adults receiving geriatric rehabilitation, usability is especially crucial, since there are certain age-related barriers that may hamper the usability of eHealth [24-26]. These barriers can be categorized into 4 patient-related domains: cognition, physical ability, perception, and motivation [27]. For example, poor vision can make it harder to distinguish certain icons on screens, or cognitive impairment might lead to problems understanding certain eHealth interventions. Often, eHealth is insufficiently tailored to these age-related barriers [28].

Therefore, a systematic review of eHealth in geriatric rehabilitation including the concepts feasibility, usability, and effectiveness was needed. This systematic review can help speed up the implementation process of eHealth and ensure successful adoption of eHealth overall. The aim of this review was to assess evidence on the effectiveness, feasibility, and usability of eHealth interventions in older adults in geriatric rehabilitation.

 Methods Study Registration and Protocol

This systematic review is registered at PROSPERO, with registration number CRD42019133192 [29]. This systematic review was based on the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-analyses) statement, which is an evidence-based minimum set of items used for reporting in systematic reviews and meta-analyses [30]. The complete checklist for this review can be found in Multimedia Appendix 1.

 Types of Studies and Participants

In this review, we included randomized controlled trials, nonrandomized studies, quantitative descriptive studies, qualitative research, and mixed methods studies. We excluded systematic reviews, abstracts, editorials, and non-English and nonpeer-reviewed studies. Studies were included that examined older patients with a mean age of ≥70 years who received geriatric rehabilitation, which is in line with consensus statements on the organization and delivery of geriatric rehabilitation across Europe [31]. Because there is variability between countries’ health care systems and consequently also between countries’ provisions of geriatric rehabilitation [31,32], we included studies in different types of settings such as (geriatric) rehabilitation centers, skilled nursing facilities, hospitals, or ambulatory settings. Studies that included patients with a chronic disease with no acute functional decline were excluded.

 Interventions and Outcomes

Studies investigated eHealth interventions applied during postacute geriatric rehabilitation. Outcome measures related to the effectiveness of interventions were included if they could be classified based on the World Health Organization’s International Classification of Functioning, Disability, and Health (ICF) model [33], which covers the following domains: body functions and structure, activities, participation, environmental factors, and personal factors. For the purpose of this review, we chose to specify feasibility within the following domains: adverse events, adherence, and exclusion rates. Usability outcome measures were classified based on the MOLD-US framework, which is an evidence-based framework of aging barriers that influence the usability of eHealth in older adults and includes 4 categories: cognition, motivation, physical ability, and perception [27]. We included both primary and secondary outcome measures.

 Sources and Search Strategy

On March 9, 2019, March 10, 2019, and January 11, 2021, we searched the following databases: MEDLINE, PsycINFO, EMBASE, EMCARE, Cochrane Library, Web of Science, and Central databases. For this review, 3 separate search strings were compiled. The first focused on the effectiveness, the second focused on the feasibility, and the third focused on the usability of eHealth interventions in geriatric rehabilitation. The search string focusing on effectiveness included keywords related to older adults, rehabilitation, and eHealth interventions. Studies were identified when at least 2 of 3 keywords were present. The search strings focusing on feasibility and usability included an additional keyword related to feasibility or usability. In both search strings, keywords were combined using MeSH terms using the Boolean operations “or” and “and.” The complete search strings can be found in Multimedia Appendix 2.

 Selection of Studies and Data Extraction

We first screened titles of the identified studies. The abstracts of all potentially relevant studies were then screened by 2 authors independently. Next, full texts were obtained and reviewed by the same authors. We excluded studies that did not meet the inclusion criteria. Disagreements between the 2 authors were discussed until a consensus was reached. If a disagreement could not be resolved, a third reviewer was consulted. Data extraction was performed using Covidence, which is an online systematic review management tool [34]. In Covidence, a data extraction form was constructed that included details of publication (ie, author, year, title, country of study, and funding), study design, methods (ie, inclusion and exclusion criteria, population, randomization, statistical analysis, and outcome measures), sample characteristics (ie, age, number of participants, gender, and diagnosis), eHealth intervention (ie, name of intervention, goal of intervention, delivery of intervention, and application of intervention), and primary and secondary outcomes. As the complexity of eHealth interventions influences implementation, we sorted eHealth interventions ranging from simple (ie, video communications, health sensors, or gateways) to complex (ie, robotics, exergames, or virtual reality) [9,35]. One author then extracted the data. A subset of the data (10% of included studies) was also extracted by a second author to check interrater reliability.

 Quality Appraisal

The quality of included studies was assessed using the Mixed Methods Appraisal Tool (MMAT) [36], which allowed quality assessment across different study designs. The MMAT is a critical appraisal tool specifically designed to assess the quality of 5 types of study designs: qualitative research, randomized controlled trials, nonrandomized studies, quantitative descriptive studies, and mixed methods studies. For each study design, the MMAT provides 5 quality criteria that must be rated with “Yes,” “No,” or “Can’t tell.” Since the calculation of an overall score from the ratings of each criterion is discouraged [36,37], we reported a separate score for each rating. Nevertheless, an overall score was reported, because it provides a general picture of study quality. Studies were not excluded based on study quality [36]. For the randomized controlled trials and nonrandomized designs, we rated the criterion “Are there complete outcome data?” as “No” when the drop-out rate was over 20% [38]. In nonrandomized designs, we rated the criterion “Are the confounders accounted for in the design and analysis?” as “No” when there was no description of additional therapy offered during the study, functional status, or cognitive status. Quality assessment was carried out by one author, and 10% of the included studies were selected at random and additionally assessed by a second author to check interrater reliability.

 Data Analysis and Data Synthesis

In studies that reported outcomes related to effectiveness and included a control group, a narrative synthesis was conducted using a harvest plot [39]. In the harvest plot, primary and secondary outcomes were described and color coded based on ICF domain. For each study, the bars in the harvest plot indicated the total results for the different ICF domains, and the height of the bars represented the methodical quality based on the MMAT. When a study reported multiple consistent results within the same ICF domain, the results were combined in 1 bar. If a study reported conflicting results within the same ICF domain, both results were presented. Randomized controlled trails were represented by a thick contour around bars. A meta-analysis was not feasible since the included studies were too heterogeneous with regard to population, intervention, and outcome measures.

 Results Study Selection

The search strategy identified a total of 7635 unique records. After exclusion of records based on title and abstract, 331 records remained. During full-text screening, a further 291 records were excluded, resulting in the inclusion of 40 studies in this review. Reasons for exclusion are presented in the study flowchart shown in Figure 1. In 12 cases, a third reviewer was needed to achieve consensus during the process of study selection.

PRISMA flow diagram of search strategy results. ICF: International Classification of Functioning, Disability, and Health.

 Study Characteristics

Study characteristics are shown in Table 1. Of the 40 included studies, 18 (45%) were randomized controlled trails [40-57], 2 (5%) had a mixed methods design [58,59], 1 was a qualitative study [60], and 19 (48%) had a quantitative nonrandomized design [61-79], of which 9 studies (of 19, 47%) included a control group [53,61-68,79]. Of 40 studies, 17 studies (43%) were conducted in a hospital setting [41-44,46,50,51,55-57,62,64-66,68,71,79]. Of the 17 hospital-setting studies, 12 (71%) were conducted in a dedicated hospital-rehabilitation unit [41-44,46,50,51,55,56,64,71,79], 2 (12%) were in a hospital-stroke unit [57,68], and 1 (6%) was conducted in a geriatric day hospital [62]. Of the 40 studies, 10 (25%) were conducted in an ambulatory setting [47,48,52-54,60,69,75,76,78], 9 studies (23%) took place in a geriatric rehabilitation setting [40,45,49,58,59,61,63,70,74], 2 studies (5%) were at a tertiary rehabilitation center [60,73], 1 study (3%) was at a skilled nursing facility [77], and 2 studies (5%) did not report the setting [67,72].

Study characteristics.

Author, year, country Design Diagnosis; n; setting Age (SD); female (%) Intervention Use of intervention Primary outcome domain (primary outcome measure) Secondary outcome domain(s)
Barnason [53], 2009, United States RCTa Cardiac; n=55; Ambulatory 71.6 (5.1); 16 Video communication in combination with non-eHealth vs usual care Daily use, subjects responded to assessment queries, were provided with strategies Effectiveness, activities (other) Effectiveness, participation
Backman [59], 2020, United Kingdom Mixed methods Orthopedic; n=30; Geriatric rehabilitation 81 (67-96); 63 Mobile apps Providing access to discharge records during transition to home Usability —b
Bernocchi [52], 2018, Italy RCT Multiple diagnoses; n=146; Ambulatory 79 (6.5); 84 Video communication in combination with non-eHealth vs usual care Weekly calls; video communication 2×/month; fall prevention program provided by therapist Effectiveness, activities (other) Feasibility, effectiveness, activities, participation
Bernocchi [69], 2016, Italy Quantitative; nonrandomized Stroke; n=15; Ambulatory 71 (11); 47 Video communication in combination with health sensors Weekly calls with nurse; weekly video communication with physiotherapist Feasibility (n completed, n sessions) Effectiveness, body functions, activities
Cannell [44], 2017, Australia RCT Stroke; n=40; Hospital, rehabilitation unit 74 (10); 37.5 Exergames in combination with virtual reality vs usual care 1 hour/session, 5 days/week, in addition to conventional therapy Effectiveness, activities (maintaining body position) Effectiveness, activities
Chan [62], 2012, China Quantitative nonrandomized Multiple diagnoses; n=90; Geriatric Day hospital 80 (7.1); 73 Exergames vs usual care 10 min/session, 8 sessions total, in addition to conventional therapy Feasibility (total time spent, average BSc and %MHRd) Effectiveness, activities
Cimarolli [77], 2017, United States Quantitative; nonrandomized Multiple diagnoses; n=237; Skilled nurse facility 76 (10.7); 59 Exergames Recommended use: 2 sessions/week for 15 min, in addition to conventional therapy Feasibility (time spent, predictors of intense use) Effectiveness, external factors
Dakin [61], 2011, Australia Quantitative; nonrandomized Multiple diagnoses; n=34; Geriatric rehabilitation 77; 47 Health sensors vs usual care Wore health sensor daily during admission Effectiveness activities (ADLe) Effectiveness, external factors
Da-Silva [57], 2019, United Kingdom RCT Stroke; n=33; Hospital, stroke unit 71; 60.6 Health sensors with reminders vs health sensors without reminders Wore health sensor for 4 weeks, health sensor vibrated to remind patients to use affected arm Effectiveness, activities (hand and arm use) Feasibility, adherence
Doornebosch [70], 2007, Netherlands Quantitative; nonrandomized Stroke; n=10; Geriatric rehabilitation 72 (53-94); 80 Robotics 20 minutes/session, 8 sessions total, in addition to conventional therapy Personal factors (patient’s experience) Effectiveness, body functions
Edmans [68], 2009, United Kingdom Quantitative; nonrandomized Stroke; n=13; Hospital, stroke unit 73; 23 Virtual reality vs usual care 1 hour/session, 5 days/week Effectiveness, activities (other) Effectiveness, activities
Franceschini [56], 2020, Italy RCT Stroke; n=48; Hospital, rehabilitation unit 72 (64.3); 45.8 Robotics vs usual care 30 minutes/session, 5 days/week over 6 weeks, in addition to conventional therapy Effectiveness, body functions (muscle power, tone, and reflexes) Effectiveness, (muscle power, tone, and reflexes)
Gandolfi [71], 2017, Italy Quantitative; nonrandomized Stroke; n=2; Hospital, rehabilitation unit 74; 100 Robotics 20 minutes/session, 5 days/week, 10 sessions total, in addition to conventional therapy Feasibility (compliance, time to set device) Effectiveness, body functions
Goto [65], 2017, Japan Quantitative; nonrandomized Orthopedic; n=20; Hospital 74 (7.5); 90 Robotics vs usual care Every other day, in addition to conventional therapy Effectiveness, body functions (mobility of joints) Effectiveness, body functions
Hesse [42], 2014, Germany RCT Stroke; n=50; Hospital, rehabilitation unit 70 (16); 44 Robotics vs usual care 30 minutes/session, 4 days/week, in addition to conventional therapy Effectiveness, body functions (muscle power, tone, and reflexes) Effectiveness, body functions, activities, external factors
Hesse [72], 2010, Germany Quantitative; nonrandomized Stroke; n=1; Not reported 72; 0 Robotics 25 minutes/session, 5 days/week, 25 sessions in total, in addition to conventional therapy Effectiveness, body functions (ADL) —
Hicks [63], 2016, United States Quantitative; nonrandomized Cardiac; n=285; Geriatric rehabilitation 79 (48-99); 54.3 Health gateway vs usual care Encouraged daily use, in addition to conventional therapy Effectiveness, activities (ADL) Effectiveness, external factors
Iosa [46], 2015, Italy RCT Stroke; n=4; Hospital, rehabilitation unit 71.5 (4.51); 50 Exergames in combination with virtual reality vs usual care 30 minutes/session, 3 days/week, in addition to conventional therapy Feasibility (motivation, time spent, adverse events) Effectiveness, body functions, activities
Karner [55], 2019, Germany RCT Stroke; n=56.4%; Hospital, rehabilitation unit 73,7 (7.33); 56.4 Robotics vs book reading 30 minutes/session 3 days/week over 3 weeks Effectiveness, body functions (visual) —
Koneva [67], 2018, Russia Quantitative; nonrandomized Stroke; n=40; Not reported 84 (1.2); 30 Virtual reality vs usual care Task-specific training Effectiveness, body functions (neurological) Effectiveness, body functions, activities, participation
Laver [50], 2012, Australia RCT Multiple diagnoses; n=44; Hospital, rehabilitation unit 84.9 (4.5); 80 Exergames vs usual care 25 minutes/session, 5 days/week for duration of stay Effectiveness, activities (mobility) Effectiveness, body functions, activities, participation
Levinger [64], 2016, Italy Quantitative; nonrandomized Orthopedic; n=4; Hospital, rehabilitation unit 70; 76 Exergames vs usual care 2 sessions/week, in addition to conventional therapy Effectiveness, activities (mobility) Effectiveness, body functions, activities, participation
Li [54], 2020, Hong Kong RCT Orthopedic; n=31; Ambulatory 79,3 (9.1); 80.6 Mobile apps vs usual care Use of app based on rehabilitation goals, in addition to conventional therapy Effectiveness, activities (mobility) Effectiveness, feasibility, body functions, activities,
Ling [58], 2017, Netherlands Mixed methods Orthopedic; n=7; Geriatric rehabilitation 70 (8); 71 Exergames 30 minutes/session, in addition to conventional therapy Usability (ease of use) —
Marschollek [75], 2014, Germany Quantitative; nonrandomized Orthopedic; n=14; Ambulatory 83.5 (71-90) Health sensors Sensors placed at home for monitoring ADL Feasibility (installation time, downtimes) Acceptability
Oesch [49], 2017, Switzerland RCT Multiple diagnoses; n=54; Geriatric rehabilitation 74 (67-79); 45 Exergames vs self-regulated exercises 30 minutes/session, twice a day Effectiveness (personal factors) Effectiveness personal factors, activities
Peel [40], 2016, Australia RCT Multiple diagnoses; n=270; Geriatric rehabilitation 81 (8); 58 Health sensors with goal-setting vs health sensors without goal-setting Daily feedback and goal-setting by therapists, in addition to conventional therapy Effectiveness, activities (mobility) Effectiveness, activities, participation, external factors
Peel [78], 2011, Australia Quantitative; nonrandomized Multiple diagnoses; n=0; Ambulatory — Video communication All communication conducted through intervention Feasibility —
Piqueras [47], 2013, Spain RCT Orthopedic; n=142; Ambulatory 73.3 (6.5); 72.4 Video communication in combination with health sensors vs usual care 1 hour/session over 10 days Effectiveness, body functions (mobility of joints) Effectiveness, body functions, activities
Pol [48], 2019, Netherlands RCT Orthopedic; n=240; Ambulatory 83 (6.9); 79.6 Health sensors in combination with non-eHealth intervention vs non-eHealth intervention vs usual care Sensors placed at home for monitoring ADL, 4 home visits and 4 telephone consultations Effectiveness, activities (other) Effectiveness, participation
Sampson [73], 2012, New Zealand Quantitative; nonrandomized Stroke; n=1; Rehabilitation center 76; 100 Robotics in combination with virtual reality 45 minutes/session, 4 sessions/week over 6 weeks, in addition to conventional therapy Effectiveness, body functions (muscle power, tone, and reflexes) Effectiveness body functions
Schoone [45], 2011, Netherlands RCT Stroke; n=24; Geriatric rehabilitation 71.3 (8.2); 33 Robotics 10-30 minutes/sessions, 3 sessions/week over 6 weeks, in addition to conventional therapy Effectiveness, body functions, activities (hand and arm use) Effectiveness participation, external factors
Schwickert [74], 2011, Germany Quantitative; nonrandomized Orthopedic; n=8; Geriatric rehabilitation 79.5; 50 Robotics, virtual reality 30-45 minutes/session, 2-3 sessions/week for 2-4 weeks, in addition to conventional therapy Feasibility (adherence, satisfaction) Effectiveness, body functions, activities, participation
Takano [79], 2020, Japan Quantitative; nonrandomized Orthopedic; n=27; Hospital, rehabilitation unit 81 (6.3); 89 Robotics in combination with exergames 20 min/session 6 sessions/week for 2 weeks in addition to conventional therapy Effectiveness activities (mobility) Effectiveness, activities,
Taveggia [43], 2016, Italy RCT Stroke; n=28; Hospital, rehabilitation unit 72 (6); 39 Robotics vs usual care 30 minutes/session, 5 sessions/week over 5 weeks, in addition to conventional therapy Effectiveness, activities (mobility) Effectiveness, activities, participation
Tousignant [76], 2006, Canada Quantitative; nonrandomized Multiple diagnoses; n=4; Ambulatory 70,75; 50 Video communication 1 hour/session, 3 sessions/week over 4 weeks Effectiveness, activities (ADL) Effectiveness, body functions, activities
Van den Berg [51], 2015, Australia RCT Multiple diagnoses; n=58; Hospital, rehabilitation unit 80 (12); 62 Exergames vs usual care 1 hour/session, 5 session/week, in addition to conventional therapy Effectiveness, activities (mobility) Usability; Effectiveness, activities, participation
Vanoglio [41], 2017, Italy RCT Stroke; n= 30; Hospital, rehabilitation unit 71 (12); 53 Robotics vs usual care 40 minutes/session, 5 sessions/week over 6 weeks Feasibility (n completed, adverse events, difficulty) Effectiveness, body functions, external factors
White [60], 2015, Australia Qualitative Stroke; N=12; Rehabilitation center, ambulatory 73 (53-83); 33 Mobile apps Therapist installed apps; patients encouraged to explore iPad Usability —
Yoshikawa [66], 2018, Japan Quantitative; nonrandomized Orthopedic; n=19; Hospital 76 (6.85); 81 Robotics vs usual care 14 minutes/session, 12-14 session in 4 weeks, in addition to conventional therapy Effectiveness, activities (mobility) Effectiveness, body functions

aRCT: randomized controlled trial.

bNot available.

cBS: Borg Perceived Exertion Scale.

d%MHR: maximum heart rate.

eADL: activities of daily living.

Of 40 studies, 17 (43%) included participants who were diagnosed with stroke [41-46,55-57,60,67-73], 10 (25%) included participants with multiple diagnoses [40,49-52,59,61,62,76-78], 11 (28%) included participants with orthopedic problems [47,48,54,58,59,64-66,74,75,79], and 2 studies (5%) included participants with cardiac-related diagnoses [53,63]. Across all studies, the included sample size ranged from 1 to 285 participants.

Various types of eHealth interventions were used. Of 40 studies, 11 studies (28%) delivered the intervention via robotics [41-43,45,55,56,65,66,70-72], 2 studies (5%) combined robotics with virtual reality [73,74], and 1 study (3%) combined robotics with exergames [79]. Additionally, 9 studies (of 40, 23%) investigated exergames [44,46,49-51,58,62,64,77], of which 2 (of 9, 22%) combined exergames with virtual reality [44,46] and 1 (of 9, 11%) combined exergames with health sensors [51]. Of 40 studies, 2 (5%) examined video communication [76,78], 3 (8%) combined video communication with health sensors [47,53,69], and 1 (3%) combined video communication with a non-eHealth intervention [52]. Of 40 studies, health sensors were used in 6 studies (15%) [40,48,57,61,63,75], including 1 (of 6, 17%) in combination with a health gateway [63] and 1 (of 6, 17%) in combination with a non-eHealth intervention [48]. Of 40 studies, 3 studies (8%) investigated mobile apps [54,59,60], and 2 studies (5%) examined virtual reality [67,68].

Outcome measures related to effectiveness were reported in 24 of 40 studies (60%) [40,42-45,47-50,53,55,56,61,63-68, 70,72,73,76,79], and 10 of 40 studies (25%) included outcome measures related to effectiveness and feasibility [41,46,52,54,57,62,69,71,74,77]. Of 40 studies, 2 studies (5%) included outcomes related to usability [58,60], 2 studies (5%) included outcomes related only to feasibility [75,78], 1 study (3%) included outcomes related to effectiveness and usability [51], and 1 study (3%) included outcomes related to feasibility and usability [59]. A detailed description of all included studies can be found in Multimedia Appendix 3.

 Study Quality

Results of the quality assessment are presented in Figure 2 and Figure 3. The quality of the included studies ranged from –3 to 5 (on a scale ranging from –5 to 5). The mean overall score was 3 for randomized controlled trails, 1 for quantitative nonrandomized studies, 1 for a mixed methods studies, and 5 for a qualitative study (based on 1 study). In quantitative nonrandomized studies, the most frequent shortcoming was insufficient reporting of confounders; only 2 of 19 studies (11%) accounted for confounders in design and analysis [73,79]. The representativeness of the target population in quantitative nonrandomized studies was also often insufficient; 9 of the 19 studies (47%) reported insufficient information, lacking either adequate explanation of why certain eligible participants chose not to participate or a clear description of the target population [53,61,65,67,69,71,75,76,78]. Additionally, 6 of the 19 studies (32%) included a sample size of less than 20 [64,66,70,72-74].

Quality appraisal for randomized controlled trial studies.

Quality appraisal for quantitative nonrandomized, qualitative, and mixed methods studies.

 Effectiveness Main Results for Effectiveness

Across all studies with a control group (n=27; 27/40, 68%), 73 different outcome measures were reported that were related to effectiveness, including 16 (22%) within the ICF domain “body functions,” 40 (55%) in the domain “activities,” 11 (15%) in the domain “participation,” 4 (5%) in the domain “external factors,” and 2 (3%) in the domain “personal factors” (Figure 4). In 15 studies (of 27, 56%), eHealth interventions were found to be at least as effective as non-eHealth interventions when focusing on the primary outcome measure, and 11 studies (of 27, 41%) reported eHealth interventions to be more effective than non-eHealth interventions. Of 27 studies, 1 study (4%) reported beneficial outcomes in favor of the non-eHealth interventions. Results for each ICF domain are described in detail below. A harvest plot illustrating the evidence regarding effectiveness is presented in Figure 5.

Outcome measures classified by the International Classification of Functioning, Disability, and Health model.

Harvest plot: effectiveness of eHealth interventions. MMAT: Mixed Methods Appraisal Tool.

 Body Functions

Of 40 total studies, 14 studies (35%) included 16 outcomes related to body functions [41,42,45-47,50,54-57,64-67]. Of these 14 studies, 9 studies (64%) found, in 7 outcome measures, significant improvements in favor of the intervention group (Figure 5) [41,46,47,54-56,65-67]. Of 14 studies, 4 studies (29%) reported improved muscle power through robotics [56,65], exergames [46], or mobile apps [54]. Of 14 studies, 4 studies (29%) found that the addition of robotics [56,65,66] or video communication in combination with health sensors [47] improved the mobility of joints when compared with physical therapy alone. Another 2 studies (of 14, 14%) reported that the use of robotics could decrease pain when compared with conventional physiotherapy [65,66]. Koneva and colleagues [67] reported that the use of virtual reality improved neurological status, as measured by the National Institutes of Health stroke scale, when compared with usual care (5.2±0.4 vs 6.3±0.5; P<.001).

 Activities

Of all 40 included studies, 25 studies (63%) reported 40 outcomes related to activities [40-55,57,61-64,66-68,79], and 13 studies (33%) found, in 17 outcomes, a significant outcome in favor of the intervention group [40,41,46,48,50-53,62,63,66,67,79]. Of 40 studies, 5 studies (13%) demonstrated that eHealth was effective in improving activities of daily living when the intervention was delivered via video communication in combination with health sensors and a non-eHealth intervention [52] or when the intervention was delivered via health sensors in combination with health gateways [63], exergames [62], robotics [79], or virtual reality [67]. In these studies, eHealth was compared with usual care [52,67], physiotherapy [62,79], or no intervention [63]. Another 6 studies (of 40, 15%) found that eHealth could contribute to improved mobility through the use of robotics [52,79], exergames [50], virtual reality [67], video communication in combination with health sensors [52], or health sensors in combination with goal setting [40]. These interventions were compared with physiotherapy [50,66,79], usual care [52,67], or health sensors without goal setting [40]. Of 40 studies, 4 studies (10%) reported improvements in balance when the intervention was delivered via robotics [79], exergames [50], exergames in combination with health sensors [51], or video communication in combination with health sensors [52], when compared with physiotherapy [50,51,79] or usual care [52]. Another 2 studies (of 40, 5%) reported that either robotics [41] or exergames in combination with health sensors [46] could improve hand and arm function when compared with physiotherapy [41] or no intervention [46]. Pol and colleagues [48] found that patient-reported daily functioning significantly improved with the use of health sensors in combination with cognitive behavioral treatment, compared with cognitive behavioral treatment alone, reporting a difference of 1.17 (95% CI 0.47-1.87; P<.001). Bernocchi and colleagues [52] reported that the use of video communication in combination with health sensors and a non-eHealth intervention was effective in preventing falls in patients who were at high risk of falling, when compared with usual care (29 falls vs 56 falls; P<.001). Of 40 studies, 1 study (3%) demonstrated that the use of video communication in combination with health sensors improved physical activity when compared with usual care [53].

 Participation

Of 40 studies, 12 studies (30%) included 11 outcome measures within the participation domain [40,43,45,48,50-53,57,64,67,79]. Of these 12 studies, 3 studies (27%) reported a significant difference in quality of life [52], mood [67], or self-assessment [48] when the intervention was delivered via the use of video communication in combination with health sensors and a non-eHealth intervention [52], virtual reality [67], or the use of health sensors in combination with a non-eHealth intervention [48]. Particularly, Bernocchi and colleagues [52] demonstrated that the use of video communication in combination with health sensors and a non-eHealth intervention significantly improved scores on the EuroQol Visual Analog Scale at 6 months, when compared with usual care (mean 63.8 vs mean 53.5; P<.001). Koneva and colleagues [67] reported that the use of virtual reality decreased the severity of depression as measured by the Beck Depression Inventory, when compared with usual care (mean 9.5, SD 5.52 vs mean 10.3, SD 6.03; P<.05). Additionally, Pol and colleagues [48] found that the use of health sensors in combination with a non-eHealth intervention significantly improved the performance satisfaction in daily functioning at 6 months, when compared with usual care, reporting a difference of 0.94 (95% CI 0.37-1.52; P<.001).

 External Factors

Across all 40 studies, 5 studies (13%) included outcome measures related to external factors [40,42,45,61,63]. Of these 5 studies, 2 studies (40%) included robotics as interventions and found significant differences in cost, in favor of the intervention group [41,42]. Of the 5 studies, 1 study (20%) included robotics as an intervention and found a difference in favor of the control group [45]. Hesse and colleagues [42] and Vanoglio and colleagues [41] reported decreases in cost with the use of robotics in comparison with either regular arm therapy (€4.15 [US $4.92] for robotic interventions vs €10.00 [US $11.85] for regular arm therapy, for each patient per session) [42] or physiotherapy (€237 [US $280.73] for robotic intervention vs €480 [US $568.57] for physiotherapy, for each patient per 30 days) [41]. In contrast, Schoone and colleagues [45] reported an increase in total costs when compared with physiotherapy (€644.14 [US $762.99] for robotic interventions vs €423.74 [US $501.93] for physiotherapy). Across all studies, no differences were found with regard to discharge settings [40,63], readmissions [40], or lengths of stay [61].

 Personal Factors

Oesch and colleagues [49] found that self-regulated exercise using instruction leaflets was superior to exergames in terms of enjoyment (effect size: 0.88, range 0.32-1.44; P<.001) and motivation (effect size: 0.59, range 0.05-1.14; P=.046).

 Feasibility Main Results for Feasibility

Of the 40 included studies, 20 studies (50%) evaluated the feasibility of the eHealth intervention used [41,46,50-52,54,57,59,60,62,64,65,69,71,72,74-78], of which 19 (of 20, 95%) concluded that the eHealth intervention was feasible when it was delivered via robotics [41,65,71,72], robotics in combination with exergames [74], exergames [50,62,64,77], exergames in combination with health sensors [46,51], video communication [76], video communication in combination with health sensors [52,69], health sensors [57], health gateways in combination with health sensors [75], or mobile apps [54,59,60]. Peel and colleagues [78] reported that the use of video communication was not feasible due to problems related to patient limitations, staff issues, and the logistics of the system.

The outcome measures applied to evaluate feasibility varied considerably among studies, and a total of 19 different outcome measures were used. Of the 20 studies that reported feasibility, 6 studies (30%) reported outcomes related to “adverse events,” 7 studies (35%) reported outcomes related to “adherence,” and 7 studies (35%) reported outcomes related to “exclusion rate.” Another 4 studies (of 20, 20%) did not specify the outcome measure used to evaluate feasibility but used outcomes related to effectiveness to establish feasibility [54,64,65,72].

 Adverse Events

None of the included studies reported serious adverse events during the study period [41,46,50,51,74,76]. However, 2 studies (of 40, 5%) reported that some participants experienced discomfort during exergames [49,50].

 Adherence

Of 40 studies, adherence was reported in 7 studies (18%) [49-52,57,74], and 5 studies (13%) reported information regarding the number of completed sessions [41,50-52,69]. Of the 7 studies reporting adherence, 5 studies (71%) reported high levels of adherence, ranging from 76% [52] to 100% [74]. Of the 7 studies, 2 studies (29%) reported low adherence in patients assigned to an exergame intervention when compared with either a non-eHealth intervention [49] or use of the exergame intervention below the recommended level (<30 minutes per week) [77].

 Exclusion Rate

Of 40 studies, high exclusion rates were found in 7 studies (18%). Specifically, of these 7 studies, 1 study (14%) reported an exclusion rate of 64% [47], 2 studies (29%) reported an exclusion rate of 75% [49,51], and 4 studies (57%) reported an exclusion rate over 80% [42,45,50,68]. In these latter studies, eHealth was delivered through complex eHealth interventions: robotics [42,45], exergames [50], and virtual reality [68]. The most commonly reported reasons for exclusion were cognitive impairment [45,47,49,51], physical impairment [45,49], and refusal to participate [42,47,49-51,68]. Of these 7 studies, in 2 studies (29%), the reason given for declining to participate was “no interest” in eHealth [50,51].

 Usability Main Results for Usability

Of 40 studies, outcomes related to the usability of eHealth interventions were addressed in 4 studies (10%): 2 studies (5%) evaluated the usability of exergames [51,58], and another 2 studies (5%) evaluated mobile apps [59,60]. Evaluation of usability consisted of a system usability scale [51], a survey of patients and therapists [58,59], or semistructured interviews [59,60]. Of the 4 studies that reported usability, 2 studies (50%) included outcomes related to the barrier “cognition,” 4 studies (100%) included outcomes related to the aging barrier “motivation,” and 1 study (25%) included outcomes related to the barrier “physical ability.” None of the studies included outcomes related to the barrier “perception.”

 Cognition

Ling and colleagues [58] reported that some patients found exergames too complicated because of the requirement to engage in multiple activities simultaneously, and they experienced difficulties in following instructions. To tailor the exergames to older patients with cognitive impairments, the authors advised to minimize the amount of information presented on the screen, which might help older patients to perceive the information better [58]. Additionally, White and colleagues [60] reported that patients with cognitive impairments experienced difficulties in operating mobile apps and needed their partner for support.

 Motivation

Van den Berg and colleagues [51] reported a mean score of 62 (SD 21), on the system usability scale (scores ranging from 0 to 100), indicating that participants were generally comfortable with exergames and that they would like to use exergames more frequently. Similar findings were reported by Ling and colleagues [58], who concluded that patients and therapists both found exergames easy to use and therapists intended to use the exergame in the future. Therapists rated the exergame as highly satisfactory for motor rehabilitation in older patients after hip surgery. Findings regarding mobile apps indicated that patients readily grasped the skills required for use and that this was a beneficial source of extrinsic motivation [59,60].

 Physical Ability

Ling and colleagues [58] reported that some patients with physical disabilities had difficulties playing certain exergames that required stepping exercises because these patients were unable to maintain balance during exergames.

 Discussion Principal Findings

This review aimed to provide an overview of the effectiveness, feasibility, and usability of eHealth in geriatric rehabilitation. The review included 40 studies that applied eHealth interventions in older patients receiving geriatric rehabilitation. The majority of the included studies showed that eHealth interventions in geriatric rehabilitation are at least as effective as non-eHealth interventions. All studies that delivered eHealth in combination with another non-eHealth intervention reported positive outcomes. Most studies included outcome measures related to the ICF domain “activities.” Very few studies included outcomes related to the ICF domain “participation.” eHealth seems to be feasible in geriatric rehabilitation, since no serious adverse events were reported and most studies reported high levels of adherence. However, high exclusion rates were found in some studies. Results related to usability indicate that there are certain age-related barriers, such as cognition and physical ability, that lead to difficulties in using eHealth. Very few studies included outcomes related to feasibility and usability. However, these are important prerequisites to maximize the likelihood of successful implementation, and they thereby influence the effectiveness of eHealth.

 Comparison With Prior Work

Our findings suggest that eHealth delivered via robotics, exergames, or health sensors is often found to be at least as effective as non-eHealth. Previous reviews that examined robotics [80], exergames [16], or health sensors [81,82] often found more beneficial results in favor of the intervention group. These reviews did not focus on older adults who were admitted for geriatric rehabilitation, and this could indicate that there are certain age-related barriers that affect the effectiveness of eHealth in older adults receiving geriatric rehabilitation. All of the included studies that delivered eHealth in combination with a non-eHealth intervention reported beneficial outcomes in favor of the intervention group. This is in line with other studies in which eHealth was delivered in combination with a non-eHealth intervention [83-85]. This indicates that eHealth is more beneficial when provided through blended care, where eHealth is delivered in combination with face-to-face treatment. This may provide a better quality of care by combining the best of the two types of interventions. This seems to especially be the case when blended care is delivered via video communication [52] or health sensors [48], since it offers the possibility to monitor and treat patients remotely.

Almost all of the studies that included outcomes related to feasibility concluded that eHealth was feasible in older adults receiving geriatric rehabilitation. None of the studies reported serious adverse events, which is in line with other reviews concerning feasibility of exergames [15,86]. The majority of the studies that included outcomes related to adherence or completed sessions reported high levels of adherence. Previous reviews that examined exergames also reported high adherence rates [86]. Some studies where eHealth was delivered via robotics or exergames reported a high exclusion rate (up to 88%). All studies with exclusion rates of ≥75% were conducted in a geriatric rehabilitation setting [45,49] or in a hospital with a dedicated rehabilitation unit [50,51]. Reasons for exclusion were mostly cognitive or physical impairments, problems that are often present in older patients receiving geriatric rehabilitation. These findings indicate that eHealth in geriatric rehabilitation is safe to use and overall adherence is expected to be high, but complex eHealth interventions such as robotics and exergames might only be feasible in a selective group of older patients receiving geriatric rehabilitation.

There is limited available evidence on the usability of eHealth interventions. The studies included in our review indicate that exergames and mobile apps are usable once older patients have been trained in their use. However, there were certain age-related barriers associated with cognitive or physical ability that led to difficulties in using eHealth. While we did not find studies that reported problems in the use of eHealth due to problems in perception, 2 of 4 studies (50%) that included usability outcome measures explicitly excluded patients with visual impairments [51,58]. This might suggest that poor usability was expected in patients with visual impairments; this is in line with findings from other studies [27]. These findings suggest that usability problems are expected in older patients receiving geriatric rehabilitation, since they often suffer from cognitive, physical, or visual impairments. eHealth should be tailored to these specific age-related barriers to maximize the probability of successful use and implementation [22,27]. Furthermore, most studies did not incorporate clear usability endpoints, and the evaluation of usability varied considerably among studies. The lack of using clear endpoints or reliable and validated questionnaires combined with task metrics (preferably, task completion) to evaluate usability hampers the ability to pinpoint usability issues and prevents comparisons across different eHealth types [25,87].

 Strengths and Limitations

The first strength of this review is the extensive search strategy that covered a broad range of search databases and included all types of research designs. Another strength of this review is the categorization of outcome measures based on the ICF model, providing a clear overview of different types of outcome domains evaluated in the included studies. Nonetheless, several limitations of this systematic review should be noted. While this review provides a broad overview of the literature on 3 different concepts, our study design led to a vast variety of different outcome measures related to effectiveness. The inclusion of various outcomes measures, in combination with various eHealth interventions and diagnoses, limited our ability to draw definitive conclusions. Since a meta-analysis was not feasible, we were unable to report an effect size and publication bias. We instead provided an overview of the effectiveness of eHealth interventions using a harvest plot. Lastly, while we used a separate search string that included keywords related to usability, we only found 4 studies that included outcomes on usability. A possible explanation might be that we did not include specific Computer Science search databases, which might include more studies that are related to usability [88]. Furthermore, despite the massive growth in eHealth studies, only a small portion publish their usability results [89].

 Conclusions

In conclusion, eHealth can improve rehabilitation outcomes in older adults receiving geriatric rehabilitation. Based on our findings, comparisons to literature, and the strengths and limitations of our review, our main results and recommendations for further research and the use of eHealth in clinical practice are (1) keep it simple, (2) include evidence on usability, (3) focus on participation, and (4) ensure consensus. First, simple interventions have the most potential to improve rehabilitation outcomes in older adults receiving geriatric rehabilitation, especially, when they are provided as blended care. Additionally, simple eHealth interventions have a higher chance of feasibility in older patients receiving geriatric rehabilitation who often suffer from cognitive or physical impairments. Second, scarce evidence on the usability of eHealth might hamper the implementation of eHealth in older patients receiving geriatric rehabilitation and could negatively influence effectiveness and feasibility. Further research on this topic with clear endpoints is needed. Health care professionals need to be aware of the usability of eHealth interventions they are providing. Third, participation is a key concept in geriatric rehabilitation and plays an important role in enabling older patients to continue living as independently as possible. Future research on eHealth interventions should consider including outcome measures related to participation. Fourth, current evidence on the use and evaluation of eHealth in geriatric rehabilitation is diverse, making it hard to compare outcomes and draw evident conclusions. Consensus on the use and evaluation of eHealth is needed for further development and implementation of eHealth in geriatric rehabilitation.

PRISMA checklist.

Keyword strings used for searching databases.

Detailed description of all included studies.

 Abbreviations ICF

International Classification of Functioning, Disability, and Health.

 MMAT

Mixed Methods Appraisal Tool.

 PRISMA

Preferred Reporting Items for Systematic Reviews and Meta-analyses.

JJMK screened titles of the identified studies. JJMK and AV screened the abstracts of all potentially relevant studies and obtained and reviewed the full texts. Disagreements between JJMK and AV were discussed until a consensus was reached. If a disagreement could not be resolved, EFvDvI was consulted. JJMK extracted the data. AV also extracted a subset of the data (10% of included studies) to check interrater reliability. JJMK performed quality assessment, and 10% of the included studies were selected at random and additionally assessed by AV to check interrater reliability. In 12 cases, a third reviewer, EFvDvI, was needed to achieve consensus during the process of study selection.

None declared.

 United Nations Department of Economic and Social affairs, Population division World Population Ageing 2019 2020-11-06 https://www.un.org/en/development/desa/population/publications/pdf/ageing/WorldPopulationAgeing2019-Report.pdf Salive ME Multimorbidity in older adults Epidemiol Rev 2013 01 31 35 1 75 83 10.1093/epirev/mxs009 23372025 mxs009 Inouye S Studenski S Tinetti M Kuchel G Geriatric syndromes: clinical, research, and policy implications of a core geriatric concept J Am Geriatr Soc 2007 05 55 5 780 91 10.1111/j.1532-5415.2007.01156.x 17493201 JGS1156 PMC2409147 Covinsky KE Pierluissi E Johnston CB Hospitalization-associated disability: "She was probably able to ambulate, but I'm not sure" JAMA 2011 10 26 306 16 1782 93 10.1001/jama.2011.1556 22028354 306/16/1782 Bachmann S Finger C Huss A Egger M Stuck A Clough-Gorr K Inpatient rehabilitation specifically designed for geriatric patients: systematic review and meta-analysis of randomised controlled trials BMJ 2010 04 20 340 c1718 10.1136/bmj.c1718 20406866 PMC2857746 Holstege MS Caljouw MA Zekveld IG van Balen R de Groot AJ van Haastregt JC Schols JM Hertogh CM Gussekloo J Achterberg WP Successful Geriatric Rehabilitation: Effects on Patients' Outcome of a National Program to Improve Quality of Care, the SINGER Study J Am Med Dir Assoc 2017 05 01 18 5 383 387 10.1016/j.jamda.2016.10.011 27939318 S1525-8610(16)30489-3 Lettow B Wouters M Sinnige J E-Health, wat is dat? Nictiz 2020-10-07 https://www.nictiz.nl/wp-content/uploads/E-health-Wat-is-dat.pdf Glasgow RE Phillips SM Sanchez MA Implementation science approaches for integrating eHealth research into practice and policy Int J Med Inform 2014 07 83 7 e1 11 10.1016/j.ijmedinf.2013.07.002 23910896 S1386-5056(13)00157-3 Ross J Stevenson F Lau R Murray E Factors that influence the implementation of e-health: a systematic review of systematic reviews (an update) Implement Sci 2016 10 26 11 1 146 10.1186/s13012-016-0510-7 27782832 10.1186/s13012-016-0510-7 PMC5080780 Ossebaard HC De Bruijn ACP van Gemert-Pijnen JEWC Geertsma RE Risks related to the use of eHealth technologies: An exploratory study National Institute for Public Health and the Environment 2020-10-06 https://www.rivm.nl/bibliotheek/rapporten/360127001.pdf Bonten TN Rauwerdink A Wyatt JC Kasteleyn MJ Witkamp L Riper H van Gemert-Pijnen LJ Cresswell K Sheikh A Schijven MP Chavannes NH EHealth Evaluation Research Group Online Guide for Electronic Health Evaluation Approaches: Systematic Scoping Review and Concept Mapping Study J Med Internet Res 2020 08 12 22 8 e17774 10.2196/17774 32784173 v22i8e17774 PMC7450369 Enam A Torres-Bonilla J Eriksson H Evidence-Based Evaluation of eHealth Interventions: Systematic Literature Review J Med Internet Res 2018 11 23 20 11 e10971 10.2196/10971 30470678 v20i11e10971 PMC6286426 Broens THF Huis in't Veld Rianne M H A Vollenbroek-Hutten MMR Hermens HJ van Halteren Aart T Nieuwenhuis LJM Determinants of successful telemedicine implementations: a literature study J Telemed Telecare 2007 06 13 6 303 9 10.1258/135763307781644951 17785027 Muellmann S Forberger S Möllers Tobias Bröring Eileen Zeeb H Pischke CR Effectiveness of eHealth interventions for the promotion of physical activity in older adults: A systematic review Prev Med 2018 03 108 93 110 10.1016/j.ypmed.2017.12.026 29289643 S0091-7435(17)30514-5 Skjæret Nina Nawaz A Morat T Schoene D Helbostad JL Vereijken B Exercise and rehabilitation delivered through exergames in older adults: An integrative review of technologies, safety and efficacy Int J Med Inform 2016 01 85 1 1 16 10.1016/j.ijmedinf.2015.10.008 26559887 S1386-5056(15)30051-4 Zeng N Pope Z Lee J Gao Z A systematic review of active video games on rehabilitative outcomes among older patients Journal of Sport and Health Science 2017 03 6 1 33 43 10.1016/j.jshs.2016.12.002 Reis E Postolache G Teixeira L Arriaga P Lima ML Postolache O Exergames for motor rehabilitation in older adults: an umbrella review Physical Therapy Reviews 2019 07 19 24 3-4 84 99 10.1080/10833196.2019.1639012 Orsmond GI Cohn ES The Distinctive Features of a Feasibility Study: Objectives and Guiding Questions OTJR (Thorofare N J) 2015 07 06 35 3 169 77 10.1177/1539449215578649 26594739 Matthew-Maich N Harris L Ploeg J Markle-Reid M Valaitis R Ibrahim S Gafni A Isaacs S Designing, Implementing, and Evaluating Mobile Health Technologies for Managing Chronic Conditions in Older Adults: A Scoping Review JMIR mHealth uHealth 2016 06 09 4 2 e29 10.2196/mhealth.5127 27282195 Bowen DJ Kreuter M Spring B Cofta-Woerpel L Linnan L Weiner D Bakken S Kaplan CP Squiers L Fabrizio C Fernandez M How we design feasibility studies Am J Prev Med 2009 05 36 5 452 7 10.1016/j.amepre.2009.02.002 19362699 S0749-3797(09)00096-8 PMC2859314 Kingston J Conducting feasibility studies for knowledge based systems Knowledge-Based Systems 2004 5 17 2-4 157 164 10.1016/j.knosys.2004.03.011 Hawley-Hague H Boulton E Hall A Pfeiffer K Todd C Older adults' perceptions of technologies aimed at falls prevention, detection or monitoring: a systematic review Int J Med Inform 2014 06 83 6 416 26 10.1016/j.ijmedinf.2014.03.002 24798946 S1386-5056(14)00058-6 Bevan N Carter J Earthy J Geis T Harker S New ISO standards for usability, usability reports and usability measures 2016 07 17 Paper presented at: International conference on human-computer interaction 2016 Toronto, Canada 10.1007/978-3-319-39510-4_25 Broekhuis M Velsen L Stal S Weldink J Tabak M Why My Grandfather Finds Difficulty in using Ehealth: Differences in Usability Evaluations between Older Age Groups Paper presented at: ICT4AW 2019 04 23 ICT4AW 2019 prague czech republic E2019 10.5220/0007680800480057 Sousa V Lopez KD Towards Usable E-Health Appl Clin Inform 2017 12 21 08 02 470 490 10.4338/aci-2016-10-r-0170 Alley SJ Samra P Rebar AL Schoeppe S Parkinson L Power D van Uffelen JGZ Schneiders A Vandelanotte C A focus group study of older adults' perceptions and preferences towards web-based physical activity interventions Inform Health Soc Care 2020 09 06 45 3 273 281 10.1080/17538157.2019.1656210 31690152 Wildenbos GA Peute L Jaspers M Aging barriers influencing mobile health usability for older adults: A literature based framework (MOLD-US) Int J Med Inform 2018 06 114 66 75 10.1016/j.ijmedinf.2018.03.012 29673606 S1386-5056(18)30245-4 Demiris G Finkelstein SM Speedie SM Considerations for the design of a Web-based clinical monitoring and educational system for elderly patients J Am Med Inform Assoc 2001 09 01 8 5 468 72 10.1136/jamia.2001.0080468 11522767 PMC131044 Kraaijkamp J van Dam van Isselt E Chavannes N Achterberg WP Versluis A Persoon A Effectiveness of eHealth interventions in older patients who are submitted for geriatric rehabilitation: a protocol for a systematic review PROSPERO International prospective register of systematic reviews 2019 2019-12-01 https://www.crd.york.ac.uk/PROSPERO/display_record.php?ID=CRD42019133192&ID=CRD42019133192 Moher D Liberati A Tetzlaff J Altman D PRISMA Group Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement Ann Intern Med 2009 08 18 151 4 264 9, W64 10.7326/0003-4819-151-4-200908180-00135 19622511 0000605-200908180-00135 van Balen R Gordon AL Schols JMGA Drewes YM Achterberg WP What is geriatric rehabilitation and how should it be organized? A Delphi study aimed at reaching European consensus Eur Geriatr Med 2019 10 08 10 6 977 987 10.1007/s41999-019-00244-7 Grund S Gordon AL van Balen R Bachmann S Cherubini A Landi F Stuck AE Becker C Achterberg WP Bauer JM Schols JMGA European consensus on core principles and future priorities for geriatric rehabilitation: consensus statement Eur Geriatr Med 2020 04 13 11 2 233 238 10.1007/s41999-019-00274-1 32297191 10.1007/s41999-019-00274-1 WHO International classification of functioning, disability and health WHO 2001 2020-12-19 International classification of functioning, disabilityhealth World Health Organization http://apps.who.int/iris/bitstream/handle/10665/42407/9241545429.pdf;jsessionid=DD34E52C5D093BFA4F006375315882CE?sequence=1 Vertias Health Innovation Covidence systematic review software 2019 2019-10-10 In Veritas Health Innovation Melbourne, VIC https://www.covidence.org/ Krijgsman J Klein WG Ordening in de wereld van eHealth Nictiz 2012 2019-10-11 https://www.nictiz.nl/wp-content/uploads/2012/08/Whitepaper-Ordening-in-de-wereld-van-eHealth.pdf Hong QN Fàbregues S Bartlett G Boardman F Cargo M Dagenais P Gagnon M Griffiths F Nicolau B O’Cathain A Rousseau M Vedel I Pluye P The Mixed Methods Appraisal Tool (MMAT) version 2018 for information professionals and researchers EFI 2018 12 18 34 4 285 291 10.3233/EFI-180221 Higgins J Green S Cochrane handbook for systematic reviews of interventions 2011 2 1 Vol 4 John Wiley & Sons Thomas B Ciliska D Dobbins M Micucci S A Process for Systematically Reviewing the Literature: Providing the Research Evidence for Public Health Nursing Interventions Worldviews on Evidence-Based Nursing 2004 06 23 1 3 176 184 10.1111/j.1524-475x.2004.04006.x Ogilvie D Fayter D Petticrew M Sowden A Thomas S Whitehead M Worthy G The harvest plot: a method for synthesising evidence about the differential effects of interventions BMC Med Res Methodol 2008 02 25 8 1 8 10.1186/1471-2288-8-8 18298827 1471-2288-8-8 PMC2270283 Peel NM Paul SK Cameron ID Crotty M Kurrle SE Gray LC Promoting Activity in Geriatric Rehabilitation: A Randomized Controlled Trial of Accelerometry PLoS One 2016 8 26 11 8 e0160906 10.1371/journal.pone.0160906 27564857 PONE-D-16-03978 PMC5001632 Vanoglio F Bernocchi P Mulè Chiara Garofali F Mora C Taveggia G Scalvini S Luisa A Feasibility and efficacy of a robotic device for hand rehabilitation in hemiplegic stroke patients: a randomized pilot controlled study Clin Rehabil 2017 03 10 31 3 351 360 10.1177/0269215516642606 27056250 0269215516642606 Hesse S Heß Anke Werner CCordula Kabbert N Buschfort R Effect on arm function and cost of robot-assisted group therapy in subacute patients with stroke and a moderately to severely affected arm: a randomized controlled trial Clin Rehabil 2014 07 22 28 7 637 47 10.1177/0269215513516967 24452706 0269215513516967 Taveggia G Borboni A Mulé Chiara Villafañe Jorge H Negrini S Conflicting results of robot-assisted versus usual gait training during postacute rehabilitation of stroke patients: a randomized clinical trial Int J Rehabil Res 2016 03 39 1 29 35 10.1097/MRR.0000000000000137 26512928 PMC4900426 Cannell J Jovic E Rathjen A Lane K Tyson AM Callisaya ML Smith ST Ahuja KD Bird M The efficacy of interactive, motion capture-based rehabilitation on functional outcomes in an inpatient stroke population: a randomized controlled trial Clin Rehabil 2018 02 19 32 2 191 200 10.1177/0269215517720790 28719977 PMC5777543 Schoone M Dusseldorp E van den Akker-van Marle Me Doornebosch Aj Bal R Meems A Oderwald Mp van Balen R Stroke Rehabilitation in Frail Elderly with the Robotic Training Device ACRE: A Randomized Controlled Trial and Cost-Effectiveness Study Journal of Robotics 2011 2011 1 10 10.1155/2011/543060 Iosa M Morone G Fusco A Castagnoli M Fusco FR Pratesi L Paolucci S Leap motion controlled videogame-based therapy for rehabilitation of elderly patients with subacute stroke: a feasibility pilot study Topics in Stroke Rehabilitation 2015 02 25 22 4 306 316 10.1179/1074935714z.0000000036 Piqueras M Marco E Coll M Escalada F Ballester A Cinca C Belmonte R Muniesa J Effectiveness of an interactive virtual telerehabilitation system in patients after total knee arthoplasty: a randomized controlled trial J Rehabil Med 2013 04 45 4 392 6 10.2340/16501977-1119 23474735 Pol M Ter Riet Gerben van Hartingsveldt Margo Kröse Ben Buurman B Effectiveness of sensor monitoring in a rehabilitation programme for older patients after hip fracture: a three-arm stepped wedge randomised trial Age Ageing 2019 09 01 48 5 650 657 10.1093/ageing/afz074 31204776 5519556 Oesch P Kool J Fernandez-Luque L Brox E Evertsen G Civit A Hilfiker R Bachmann S Exergames versus self-regulated exercises with instruction leaflets to improve adherence during geriatric rehabilitation: a randomized controlled trial BMC Geriatr 2017 03 23 17 1 77 10.1186/s12877-017-0467-7 28330455 10.1186/s12877-017-0467-7 PMC5363020 Laver K George S Ratcliffe J Quinn S Whitehead C Davies O Crotty M Use of an interactive video gaming program compared with conventional physiotherapy for hospitalised older adults: a feasibility trial Disabil Rehabil 2012 03 12 34 21 1802 8 10.3109/09638288.2012.662570 22409245 van den Berg M Sherrington C Killington M Smith S Bongers B Hassett L Crotty M Video and computer-based interactive exercises are safe and improve task-specific balance in geriatric and neurological rehabilitation: a randomised trial J Physiother 2016 01 62 1 20 8 10.1016/j.jphys.2015.11.005 26701163 S1836-9553(15)00131-9 Bernocchi P Giordano A Pintavalle G Galli T Ballini Spoglia E Baratti D Scalvini S Feasibility and Clinical Efficacy of a Multidisciplinary Home-Telehealth Program to Prevent Falls in Older Adults: A Randomized Controlled Trial J Am Med Dir Assoc 2019 03 20 3 340 346 10.1016/j.jamda.2018.09.003 30366759 S1525-8610(18)30494-8 Barnason S Zimmerman L Schulz P Tu C Influence of an early recovery telehealth intervention on physical activity and functioning after coronary artery bypass surgery among older adults with high disease burden Heart Lung 2009 11 38 6 459 68 10.1016/j.hrtlng.2009.01.010 19944870 S0147-9563(09)00009-0 PMC2841300 Li CT Hung GK Fong KN Gonzalez PC Wah S Tsang HW Effects of a home-based occupational therapy telerehabilitation via smartphone for outpatients after hip fracture surgery: A feasibility randomised controlled study J Telemed Telecare 2020 06 28 1357633X2093243 10.1177/1357633x20932434 Karner S Stenner H Spate M Behrens J Krakow K Effects of a robot intervention on visuospatial hemineglect in postacute stroke patients: a randomized controlled trial Clin Rehabil 2019 12 13 33 12 1940 1948 10.1177/0269215519865993 31409126 Franceschini M Mazzoleni S Goffredo M Pournajaf S Galafate D Criscuolo S Agosti M Posteraro F Upper limb robot-assisted rehabilitation versus physical therapy on subacute stroke patients: A follow-up study J Bodyw Mov Ther 2020 01 24 1 194 198 10.1016/j.jbmt.2019.03.016 31987544 S1360-8592(19)30111-1 Da-Silva RH Moore SA Rodgers H Shaw L Sutcliffe L van Wijck F Price CI Wristband Accelerometers to motiVate arm Exercises after Stroke (WAVES): a pilot randomized controlled trial Clin Rehabil 2019 08 07 33 8 1391 1403 10.1177/0269215519834720 30845829 Ling Y Ter Meer LP Yumak Z Veltkamp RC Usability Test of Exercise Games Designed for Rehabilitation of Elderly Patients After Hip Replacement Surgery: Pilot Study JMIR Serious Games 2017 10 12 5 4 e19 10.2196/games.7969 29025696 v5i4e19 PMC5658642 Backman C Harley A Kuziemsky C Mercer J Peyton L MyPath to Home Web-Based Application for the Geriatric Rehabilitation Program at Bruyère Continuing Care: User-Centered Design and Feasibility Testing Study JMIR Form Res 2020 09 14 4 9 e18169 10.2196/18169 32924953 v4i9e18169 PMC7522728 White J Janssen H Jordan L Pollack M Tablet technology during stroke recovery: a survivor's perspective Disabil Rehabil 2015 09 12 37 13 1186 92 10.3109/09638288.2014.958620 25212736 Dakin L Peel N Effect of accelerometry on the functional mobility of older rehabilitation inpatients as measured by functional independence measure--locomotion (FIM) gain: a retrospective matched cohort study J Nutr Health Aging 2011 05 30 15 5 382 6 10.1007/s12603-010-0129-3 21528165 Chan T Chan F Shea Y Lin O Luk J Chan F Interactive virtual reality Wii in geriatric day hospital: a study to assess its feasibility, acceptability and efficacy Geriatr Gerontol Int 2012 10 12 4 714 21 10.1111/j.1447-0594.2012.00848.x 22469232 Hicks SA Cimarolli VR The effects of telehealth use for post-acute rehabilitation patient outcomes J Telemed Telecare 2016 12 26 24 3 179 184 10.1177/1357633x16686771 Levinger P Zeina D Teshome AK Skinner E Begg R Abbott JH A real time biofeedback using Kinect and Wii to improve gait for post-total knee replacement rehabilitation: a case study report Disabil Rehabil Assist Technol 2016 09 04 11 3 251 62 10.3109/17483107.2015.1080767 26336875 Goto K Morishita T Kamada S Saita K Fukuda H Shiota E Sankai Y Inoue T Feasibility of rehabilitation using the single-joint hybrid assistive limb to facilitate early recovery following total knee arthroplasty: A pilot study Assist Technol 2017 09 30 29 4 197 201 10.1080/10400435.2016.1219883 27689789 Yoshikawa K Mutsuzaki H Sano A Koseki K Fukaya T Mizukami M Yamazaki M Training with Hybrid Assistive Limb for walking function after total knee arthroplasty J Orthop Surg Res 2018 07 03 13 1 163 10.1186/s13018-018-0875-1 29970139 10.1186/s13018-018-0875-1 PMC6029050 Koneva E Timashkova G Shapovalenko T Lyadov K Functional Spatially-Oriented Rehabilitation Of Elderly Patients After Cerebral Stroke Reseasrch Journal of Pharmaceutical, Biological, and Chemical Sciences 2018 9 5 2232 2238 Edmans J Gladman J Hilton D Walker M Sunderland A Cobb S Pridmore T Thomas S Clinical evaluation of a non-immersive virtual environment in stroke rehabilitation Clin Rehabil 2009 02 01 23 2 106 16 10.1177/0269215508095875 19164398 23/2/106 Bernocchi P Vanoglio F Baratti D Morini R Rocchi S Luisa A Scalvini S Home-based telesurveillance and rehabilitation after stroke: a real-life study Top Stroke Rehabil 2016 04 09 23 2 106 15 10.1080/10749357.2015.1120453 27078116 Doornebosch AJ Cools HJ Slee-Turkenburg ME van Elk MG Schoone-Harmsen M Robot-mediated ACtive REhabilitation (ACRE2) for the hemiplegic upper limb after a stroke: A pilot study TAD 2008 01 03 19 4 199 203 10.3233/tad-2007-19405 Gandolfi M Geroin C Tomelleri C Maddalena I Kirilova Dimitrova E Picelli A Smania N Waldner A Feasibility and safety of early lower limb robot-assisted training in sub-acute stroke patients: a pilot study Eur J Phys Rehabil Med 2017 12 53 6 870 882 10.23736/s1973-9087.17.04468-9 Hesse S Waldner A Tomelleri C Innovative gait robot for the repetitive practice of floor walking and stair climbing up and down in stroke patients J Neuroeng Rehabil 2010 06 28 7 1 30 10.1186/1743-0003-7-30 20584307 1743-0003-7-30 PMC2914004 Sampson M Shau Y King Marcus James Bilateral upper limb trainer with virtual reality for post-stroke rehabilitation: case series report Disabil Rehabil Assist Technol 2012 03 29 7 1 55 62 10.3109/17483107.2011.562959 21446826 Schwickert L Klenk J Stähler A Becker C Lindemann U Robotic-assisted rehabilitation of proximal humerus fractures in virtual environments: a pilot study Z Gerontol Geriatr 2011 12 14 44 6 387 92 10.1007/s00391-011-0258-2 22159833 Marschollek M Becker M Bauer JM Bente P Dasenbrock L Elbers K Hein A Kolb G Künemund Harald Lammel-Polchau C Meis M Meyer Zu Schwabedissen Hubertus Remmers H Schulze M Steen E Thoben W Wang J Wolf K Haux R Multimodal activity monitoring for home rehabilitation of geriatric fracture patients--feasibility and acceptance of sensor systems in the GAL-NATARS study Inform Health Soc Care 2014 08 22 39 3-4 262 71 10.3109/17538157.2014.931852 25148561 Tousignant M Boissy P Corriveau H Moffet H In home telerehabilitation for older adults after discharge from an acute hospital or rehabilitation unit: A proof-of-concept study and costs estimation Disabil Rehabil Assist Technol 2006 09 09 1 4 209 16 10.1080/17483100600776965 19260168 Cimarolli VR Reinhardt JP Minahan J Burack O Thomas C Melly R Use of an Exercise Technology in Post-Acute Care of a Skilled Nursing Facility: A Feasibility Study J Am Med Dir Assoc 2017 11 01 18 11 991.e1 991.e4 10.1016/j.jamda.2017.07.002 28843523 S1525-8610(17)30378-X Peel NM Russell TG Gray LC Feasibility of using an in-home video conferencing system in geriatric rehabilitation J Rehabil Med 2011 03 43 4 364 6 10.2340/16501977-0675 21305228 Takano E Ozaki K Satoh K Kawamura K Maltais M Kondo I Effects of a Balance Exercise Assist Robot on Older Patients with Hip Fracture: A Preliminary Study J. Med. Biol. Eng 2020 10 08 40 6 783 789 10.1007/s40846-020-00568-x Mehrholz J Pohl M Platz T Kugler J Elsner B Electromechanical and robot-assisted arm training for improving activities of daily living, arm function, and arm muscle strength after stroke Cochrane Database Syst Rev 2018 09 03 9 CD006876 10.1002/14651858.CD006876.pub5 30175845 PMC6513114 Cooper C Gross A Brinkman C Pope Ryan Allen Kelli Hastings Susan Bogen Bard E Goode Adam P The impact of wearable motion sensing technology on physical activity in older adults Exp Gerontol 2018 10 02 112 9 19 10.1016/j.exger.2018.08.002 30103025 S0531-5565(17)30917-8 PMC6436091 Goode AP Hall KS Batch BC Huffman KM Hastings SN Allen KD Shaw RJ Kanach FA McDuffie JR Kosinski AS Williams JW Gierisch JM The Impact of Interventions that Integrate Accelerometers on Physical Activity and Weight Loss: A Systematic Review Ann Behav Med 2017 02 26 51 1 79 93 10.1007/s12160-016-9829-1 27565168 10.1007/s12160-016-9829-1 PMC5253097 Talboom-Kamp EPWA Verdijk NA Kasteleyn MJ Harmans LM Talboom IJSH Numans ME Chavannes NH Effect of a combined education and eHealth programme on the control of oral anticoagulation patients (PORTALS study): a parallel cohort design in Dutch primary care BMJ Open 2017 09 27 7 9 e017909 10.1136/bmjopen-2017-017909 28963313 bmjopen-2017-017909 PMC5623459 Wentzel J van der Vaart Rosalie Bohlmeijer ET van Gemert-Pijnen Julia E W C Mixing Online and Face-to-Face Therapy: How to Benefit From Blended Care in Mental Health Care JMIR Ment Health 2016 02 09 3 1 e9 10.2196/mental.4534 26860537 v3i1e9 PMC4764785 van der Kleij RM Kasteleyn MJ Meijer E Bonten TN Houwink EJ Teichert M van Luenen S Vedanthan R Evers A Car J Pinnock H Chavannes NH SERIES: eHealth in primary care. Part 1: Concepts, conditions and challenges Eur J Gen Pract 2019 10 10 25 4 179 189 10.1080/13814788.2019.1658190 31597502 PMC6853224 Valenzuela T Okubo Y Woodbury A Lord SR Delbaere K Adherence to Technology-Based Exercise Programs in Older Adults: A Systematic Review J Geriatr Phys Ther 2018 06 29 41 1 49 61 10.1519/JPT.0000000000000095 27362526 Broekhuis M van Velsen Lex Hermens H Assessing usability of eHealth technology: A comparison of usability benchmarking instruments Int J Med Inform 2019 08 128 24 31 10.1016/j.ijmedinf.2019.05.001 31160008 S1386-5056(19)30067-X Zapata BC Fernández-Alemán José Luis Idri A Toval A Empirical studies on usability of mHealth apps: a systematic literature review J Med Syst 2015 02 39 2 1 10.1007/s10916-014-0182-2 25600193 Maramba I Chatterjee A Newman C Methods of usability testing in the development of eHealth applications: A scoping review Int J Med Inform 2019 06 126 95 104 10.1016/j.ijmedinf.2019.03.018 31029270 S1386-5056(18)31318-2