The following databases were searched from inception to March 2016: MEDLINE, Web of Science (Core collection), CINAHL, and the Cochrane Library. Medical Subject Headings (MeSH) keywords used were cerebrovascular disorders, hemorrhage, cerebral hemorrhage, self-help devices, telemedicine, physical therapy modalities, physical and rehabilitation medicine, exercise, exercise therapy, exercise movement techniques, self-evaluation programs, sensory feedback, motor skills, gait disorders, neurologic, gait apraxia, and gait ataxia. Text terms used were stroke, technology, physiotherapy, lower limb, rehabilitation, and gait. These were combined with text term synonyms: cerebrovascular accident (CVA), poststroke, cerebrovascular, brain ischemia, IT (information technology), ICT (information and communications technology), assistive technology, telehealth, telecare, telerehabilitation, physical therapy, physiatric, exercise, lower extremity, lower limb, ambulant, walk, locomotion, mobile, move, motion, biofeedback, sensory feedback, advise, result, evaluation, observe, assess, inform, train, therapy, treat, motor skills, motor re-learn, re-educate, re-learn, recovery enhance, promote, support, function, activity, physical, ambulant, and walking. Terms were combined using Boolean logic (“AND”, “OR”). MeSH are specific recognized terms used for the purpose of indexing journal articles and books in electronic databases. Free text terms and synonyms are specific words that the search strategy looks for in the title and abstract.

A copy of the MEDLINE search strategy is presented in Multimedia Appendix 1. Electronic citations were downloaded to Endnote software. The inclusion criteria are described in Table 1.

As this is a review of effectiveness, RCTs were chosen as the appropriate study design to answer the research question. Inclusion of non-RCT evidence is outside the scope of this review.

Comparators could be exercise/physical therapy, sham stimulation, conventional gait therapy, or treatment as usual. The primary outcome for this review was changes in activity and participation assessed by any of the following methods: the Rivermead Mobility Index, the Barthel Index, the Berg Balance Scale, the Six Minute Walk Test, the Functional Ambulatory Category, the Timed Up and Go test, the Motricity Index, the Stroke Self-Efficacy Scale, and the Performance-Oriented Mobility Assessment.

Methodological quality of included RCTs was assessed using the Cochrane Collaboration risk of bias assessment criteria [30]. This tool addresses specific domains, namely, sequence generation, allocation concealment, blinding of participants and personnel, blinding of outcome assessment, incomplete outcome data, and selective outcome reporting. For the selective reporting domain, a proxy judgement was made that if a trial reported that a study protocol had been approved and the trial report described primary and secondary outcomes with results, then the trial could be considered at low risk of selective reporting bias. We classified RCTs as being at overall low risk of bias if they were rated as “low” for each of three key domains: (1) allocation concealment [31], (2) blinding of outcome assessment, and (3) completeness of outcome data. RCTs judged as being at high risk of bias for any of these domains were judged at overall high risk. Similarly, RCTs judged as being at unclear risk of bias for any of these domains were judged at overall unclear risk.

Retrieved titles, abstracts, and/or papers were screened independently by 2 review authors (LAP, JP) to identify studies that met the inclusion criteria. Disagreements were resolved between reviewers through discussion. A standardized form was used for data extraction using Excel. Details of the RCT characteristics, included participants, the intervention, and comparator. Data extraction was carried out by reviewer LP and checked for accuracy by reviewer JP. Missing data were requested from study authors.

When undertaking a systematic review, it is essential that the quality of the outcome measures used in each study is assessed in order to ensure that the results of the study are valid and reliable. In order to do this, three clear domains need to be considered for each of the outcome measures used: (1) whether the psychometric properties of the scale have been assessed previously [32], (2) whether the clinimetric properties of the scale have been considered [33-37], specifically the Minimally Clinically Important Difference (MCID) [36], and (3) whether the design and analysis of the measurement scale fulfils the requirements of measurement theory [38-40].

We identified all the outcome measures (N=19) used in the 11 trials and reviewed each individually to assess whether they fulfilled the first two domains outlined above. The outcome measures were:

This was established by reviewing the literature on each of the measuring scales. We then examined each measurement scale to establish how the data were scored and how data collected were subsequently analyzed within the results section of each trial.

We classified the measures against the three domains within the World Health Organization ICF, as the aim of this review was to assess the effectiveness of lower-limb wearable technology for improving activity and participation. We wanted to exclude any measurements of “body structures” (impairment) such as the Fugl-Meyer assessment or the Ashworth scale. All 19 outcome measures included were measures of “activity” and 2 were measure of “participation” as classified by the ICF [23].

We have presented a narrative overview of the included RCTs with supporting evidence tables and text. A meta-analysis was not undertaken.

The electronic searches identified 940 citations following de-duplication. No additional citations were identified through reference searches/other sources. We excluded 780 citations at the title and 128 at abstract stage. We then obtained 32 citations as full-text articles. Of these, 21 were excluded at the full-text stage; details of these excluded studies with the reason for exclusion are shown in Multimedia Appendix 2 [41-59]. Eleven RCTs reported across 11 publications were included in the review (see Figure 1).

Full details from the Cochrane risk of bias assessment are presented in Multimedia Appendix 3. A summary of the risk of bias assessment is presented in Table 2, and a summary of the outcome measurement quality assessment can be found in Multimedia Appendix 4.

Seven of the 11 included RCTs were considered to be at overall high risk of bias [60-66]. Six of these were judged to be at high risk of an attrition bias [60-63,65,66], and two reported that the outcome assessment was not blinded [64,66]. The remaining three RCTs were considered to be at overall unclear risk of bias. None of the included RCTs were considered to be at high risk for the concealment of allocation domain.

Eight of the 11 [14,60-62,64,66-68] included RCTs used a combination of ordinal and ratio scales of measurement all with established psychometric properties; however, it was unclear what the minimally clinically important difference (MCID) was for the ratio data. Therefore, the clinical significance of the findings is difficult to establish. Two papers [63,69] used the Bartell Index alone, which has been proven not to be a unidimensional scale. Therefore, the analysis of the data was inappropriate, putting the findings at risk. One paper [65] used ratio levels of measurement, but again the MCID was unclear. Relative results are not reported in the RCTs.

This review set out to answer the question “What is the effectiveness of lower-limb wearable technology for improving activity and participation in adult stroke survivors?” The review found that there is little evidence in the literature to support the use of wearable technologies to improve activity and participation. Following exclusions, outcome measure assessment and quality assessment of RCTs, 11 studies were included (see Table 3).

The interventions used in eight of the 11 RCTs identified made no significant between-group differences in functional and participation abilities in adults post stroke. Three of the 11 studies did demonstrate significant between-group differences. One study that found significant between-group differences recruited 109 participants [65] comparing TENs together with a task-related exercise program modified from the training programs [70] with placebo and exercise and a control with no active treatment. The study provided evidence that the subjects receiving this intervention in a home environment had a significantly greater absolute and percentage increase in gait velocity and a reduction in timed get up and go scores from Week 2 onwards.

Another study recruited 60 participants [67] where all participants wore accelerometers around both ankles and were randomly assigned to either receive feedback on the accelerometer data from their physiotherapist or to not receive feedback. The study provided evidence that providing feedback to the participants significantly improved their cadence of daily walking.

The third study recruited 40 participants [60] where all participants underwent 20 minutes of active leg cycling with or without FES application to the muscles of the paretic upper leg. The study provided evidence for the intervention improving participants gait and balance (measured using the POMA); however, these improvements were not sustained when participants were followed up. It could be argued, however, that the high dropout rate (n=19) could have affected the significance of the lasting effects of the study.

As described fully in the quality assessment section of this paper, seven of the 11 included RCTs were considered to be at overall high risk of bias [66]. However, this does not mean that the interventions were not effective for improving gait for people post stroke. A number of conclusions could be drawn from this result. One may be that interventions that rely heavily on direct clinical input may not be suitable for this population where self-managed interventions may be more appropriate.

A number of measurement scales used in the trials were not incorporated in the outcome data for the review, as they were not validated scales: the Bobath scale [61], the 5X Sit-To-Stand-Test [14], and the California Functional Evaluation 40 [14]. Of the 11 RCTs included in the review, eight used a combination of ordinal and ratio data with proven psychometric properties; however, the clinicmetric properties were not described. The lack of evidence, therefore, in eight studies could have been due to the lack of a clinically meaningful, responsive outcome measurement scale combined with a potential lack of statistical power due to small sample sizes. The three studies that did have significant results used a combination of ordinal and ratio data with only one study [60] that provided estimates of MCID together with appropriate anayisis of the FAC data. While Shamay et al did not consider the clinical meaning or significance of the change in scores, they did report research supporting the “practical significance” of the TUG [71], which found that older adults who were able to complete the TUG task in less than 20 seconds were more likely to be independent in the transfer tasks needed for activites of daily living.

The results from this systematic review should be generalized to a wider stroke population cautiously due to the low recruitment figures for the majority of the included RCTs. Observations of lack of efficacy should also be interpreted with caution, given the uncertainty surrounding the methodological quality of the existing evidence base. Only a small number of papers with small sample sizes were able to be included in this review. Three of the selected studies recruited fewer than 20 participants [61,63,64,72], and only two recruited over 100 participants [62,65]. This could be for a number of reasons including difficulty to recruit a poststroke population to such studies. Despite the plethora of research in poststroke gait research, only 11 RCTs were selected for this review. This could be due to the difficulty of including complex interventions within an RCT design.

An RCT aims to control conditions for each arm of the study, frequently aggregating group data to provide mean values. However, no stroke is the same, recovery varies across individuals, and recovery is naturally accelerated soon after the stroke compared to those who suffered a stroke a long time ago. These factors coupled with different causes and different types of stroke, make it very difficult to control each arm of a study. Therefore, it is difficult to infer if certain interventions improve functionality post stroke or if other variables are responsible. Exploring individual change over time particularly when evaluating novel technologies with complex conditions may provide more valuable information. It has been suggested [73] that the integration of a realist evaluation perspective within an RCT design may be more appropriate and a paradigm shift for evidence-based medicine where “statistically significant benefits may be marginal in clinical practice” [74].

The results of the RCTs were not combined for a meta-analysis due to the varied types and quality of data collected for the primary outcome measures. It would also be difficult to compare primary outcomes across RCTs accurately as there were a wide variety of functional and participation outcome measures used across the 11 RCTs, some of which lacked validity as a measure of activity and participation.

Evidence exists supporting the need for task specificity, intensity, and repetition of motor skills in order to promote neuroplasticity and motor relearning; however, seven of the interventions in this review of RCTs were reliant on staff presence. This automatically eliminates the ability of stroke survivors to self-manage their rehabilitation, increasing both intensity and repetition within a task-specific environment.

This review included 550 participants at baseline and 474 participants at final follow-up, 260 from two studies alone [62,65]. Stroke severity can affect the rate by which individuals recover from a stroke and how they may or may not respond to interventions. Only two [61] of 11 papers in this review reported the stroke severity of their participants. Perhaps the severity was low and therefore it was difficult to infer a significant improvement of function. One paper [65] reported clinically and statistically significant results for the use of lower-limb wearable technologies with rehabilitation, although the technology was TENs, a technology that may not support a self-management paradigm and is not always tolerated by stroke survivors.

Perhaps future research should consider larger sample sizes, with valid, reliable, and responsive measurement tools ensuring clarity when reporting outcomes. Population descriptors should be used when exploring technology enhanced self-management models of poststroke rehabilitation. Outcome measures should be chosen only if they have psychometric or clinimetric properties reported. Where possible, individuals’ change over time should be captured and analyzed to ensure we begin to understand what works for whom and in what respect [75].

This review found that there is little evidence in the literature to support the use of wearable technologies to improve activity and participation following a stroke. However, this review has highlighted a number of reasons for a lack of significant findings including low sample sizes, the appropriateness of RCT methodology for complex interventions, a lack of appropriate analysis of outcome data, and participant stroke severity.