Hand function assessment is crucial in determining the extent of functional loss in patients and the outcome of surgical and rehabilitative procedures. Subtle changes in hand function could be a good predictor for the early detection of certain neuromuscular degeneration diseases, such as Parkinson disease (PD), which could help take preventive measures to reduce the severity of the illness [1]. However, most current hand function assessments are conducted in a clinical context with the intensive involvement of rehabilitation professionals. Clinical evaluation requires frequent visits and long-duration treatment sessions [2]. Hand function is usually assessed using standard questionnaires, such as the Michigan Hand Outcome Questionnaire and Disability of the Arm, Shoulder, and Hand Index [3]. These measurements are subjective and could result in different assessment results across different test scenarios and medical professionals [4]. Clinical outcomes based on a rating scale are often insensitive to subtle hand function changes and do not support the provision of timely feedback [5]. As such, a hand assessment tool that can overcome the clinical assessment drawbacks of inconvenience, high cost, and imprecision [1,5] and automatically evaluate hand function over time would benefit patients.

Smartphones are equipped with advanced technologies, such as touchscreens, accelerometers, and gyroscopes, which can be used for measuring and evaluating hand function [6]. The application of smartphones in clinical hand dysfunction assessments can exploit built-in sensors (such as accelerometers and gyroscopes) to collect real-time hand movement data with convenience and at low cost [7]. Smartphones can precisely monitor and analyze a patient’s hand condition for dysfunction assessment using machine learning (ML) and artificial intelligence algorithms [8]. Moreover, the smartphone-based hand dysfunction assessment can be designed according to clinical criteria to improve the system’s reliability and validity [9-11]. Despite recent advances in smartphone-based hand function assessment [12,13], no systematic reviews have been conducted to provide a holistic perspective on how smartphones can be applied to hand function assessment.

Although other technologies, such as wrist-worn or finger-worn sensors, smartwatches, and specialized keyboards, also show potential for automated hand function assessment, they typically focus on simple physiological data collection with limited data processing capabilities and display of basic information [14-16]. However, smartphones offer more extensive data acquisition, accurate data processing, and richer data display options, providing a more comprehensive technological solution [17,18]. Moreover, considering the widespread availability and user-friendly nature of smartphones [19], directing research efforts toward smartphone-centric studies can enhance innovation and application possibilities. This approach not only aligns with the current prevalence of smartphones but also extends a broader scope for future technology transfer and development specific to hand function assessment. Therefore, focusing on smartphone research can lead to more innovation and application possibilities, offering a broader scope for future technology transfer and development. As such, the main goal of this review was to synthesize the present ways in which smartphones are applied in hand function assessment and the extent to which hand function evaluation is achieved using smartphones. It aimed to explore the system development guidelines for the future application of smartphones in hand function assessment.

The research questions (RQs) were as follows: (1) What types of hand dysfunctions are studied, and what assessment inventory tools are used? (2) How are smartphones applied in clinical practice in hand function assessment? (3) What sensors are integrated into smartphones to collect hand function data? (4) What statistics or ML algorithms are used for hand function assessment?

An information specialist (JB) developed and executed a comprehensive search strategy. The following electronic databases were searched: MEDLINE(R) ALL (Ovid), Embase and Embase Classic (Ovid), CENTRAL (Ovid), Scopus, Compendex (Engineering Village), INSPEC (Engineering Village), IEEE Xplore, and ACM Digital Library. The search strategy was first developed in MEDLINE ALL (Ovid) in consultation with the research team. Search terms were also sourced from a previously published review [20]. The search strategy was then adapted into other databases.

Search strategies included the use of text words and subject headings related to two major concepts: (1) smartphone or mobile phone or mHealth and (2) hand function or function assessment. Searches were limited to English-language papers. When possible, searches were also limited to human studies and excluded conference proceedings and trial register records. No date limits were applied. All searches were conducted on June 8, 2023. The complete search strategies for each database are provided in Multimedia Appendix 2.

The studies were imported into Covidence (Veritas Health Innovation) after eliminating duplicates using EndNote (Clarivate). Title and abstract screening and full-text screening were completed by 2 researchers (YZ and YF) independently based on the same inclusion and exclusion criteria. Any disagreement was first discussed and solved by the 2 researchers. Otherwise, a third researcher (BY) was involved to ensure that an agreement was reached.

Neurocognition is evaluated as an independent criterion in clinical hand assessments [21]. Therefore, neurocognitive studies were excluded from this review to focus specifically on aspects related to hand motor control and dysfunction. Although cognitive functions play a significant role in hand motor control, the primary aim of this review was to narrow its scope and focus on the specific factors directly related to the mechanics and dysfunction of the hand, with particular focus on methods and techniques for using smartphones in assessment. Neurocognitive research often involves specialized equipment and methods, for example, neuroimaging techniques such as functional magnetic resonance imaging or electroencephalogram, which may not be practical for assessing hand function in smartphone-related contexts.

After the screening stage, the research quality of selected studies was evaluated using the Mixed Methods Appraisal Tool, a tool designed for the systematic mixed research review evaluation phase [22]. The quality assessment was completed by one researcher and checked by another researcher. A conflict that arose regarding the assessment was discussed between the 2 researchers, and an agreement was reached.

The inclusion and exclusion criteria used for the screening process are presented in Textbox 1.

A total of 8898 records were retrieved from the search. After removing duplicates, 64.31% (5722/8898) of the records were filtered at the title and abstract screening stage. After title and abstract screening, 97.68% (5589/5722) of the records were removed. The remaining 2.32% (133/5722) of the records underwent full-text screening. A total of 46 studies were included after both screening stages and included in the final review. Figure 1 presents the PRISMA [23] flow diagram. Multimedia Appendix 3 [6,9-11,24-58] details the results of the evaluation of included studies based on the Mixed Methods Appraisal Tool. All 46 studies were published after 2012, and 67% (n=31) of them were published between 2017 and 2023.

Of the 46 studies, 14 (30%) recruited participants with hand dysfunction, 7 (15%) included only healthy participants, and 23 (50%) recruited both types of participants (Table 1). The summarized smartphone specification is shown in Table 2. The age range was 21 to 91 years for patients with hand dysfunction and 17 to 81 years for healthy participants; the sample size varied from 1 to 1815.

Smartphone-based hand assessment has been applied in 4 different ways. It has been used for the measurement of function parameters (ie, wrist and finger ROM and hand strength), the early detection of disease-related dysfunction, real-time assessment during rehabilitation, and function assessment and rating (Table 5).

Of the 46 studies, 18 (39%) focused on the measurement of hand function parameters such as wrist ROM [26,27,37,40,41,47,59,65], finger ROM [24,25,35,37-39,45,49], hand gesture [49], hand dexterity [9], or hand grip strength [34]. Hand grip strength measurement and hand dexterity measurement were conducted on smartphones and shown to have good constancy with traditional measurement tools [16,23,38].

Furthermore, 15 (33%) out of the 46 papers focused on dysfunction assessment for early disease detection. Dysfunctions, such as hand tremor (10/46, 22%), hand bradykinesia (3/46, 7%), fine hand use decline (5/46, 11%), and hypokinesia (2/46, 4%), were used as biomarkers for certain diseases, such as PD [6,10,29,30,42,44,48,51,52,54,55,57,61,62], CTS [30], and hand arthritis [57,65]. The detection exhibited high sensitivity and specificity, supporting personalized treatment plan adjustments and enabling early disease diagnosis and optimized management [55].

Among the 46 studies, 14 (30%) concentrated on rating hand dysfunction severity, mostly in PD- or MS-induced hand tremor (8/46, 17%) and bradykinesia (4/46, 9%). The findings demonstrate that smartphones can determine the degree to which the patient is affected by the disease, rating the severity of both the disease and hand dysfunction [45,67,68].

Furthermore, 8 (17%) of the 46 studies explored how smartphones were used for real-time hand function assessment during hand rehabilitation [25,26,32,37,39,45,46,58]. Smartphones provide an interactive interface with guided exercises, therapeutic games, and performance feedback [26,45]. The results of real-time assessment during rehabilitation can help increase patients’ motivation and interest, reduce discontinuity in the rehabilitation process, and lower treatment costs [25,26,32,37,39,45,46,58].

The literature showed that smartphones had been used in 4 ways for hand function assessment: data collection (38/46, 83% studies), data display (17/46, 37% studies), data transmission (15/46, 33% studies), and data processing (6/46, 13% studies).

In the literature, smartphones only assessed 6 types of hand dysfunctions, namely abnormal ROM, tremor, bradykinesia, fine motor skill decline, hypokinesia, and hand arthritis–related hand dysfunction. The reason might be that smartphones are limited in capturing the complexity and variety of hand movements to measure all aspects of clinically relevant hand functions [73]. Other types of hand dysfunctions such as decreased grip strength, altered sensation, and impaired coordination are important biomarkers clinically, requiring the future development of smartphones to collect related parameters [74].

ROM is a critical and objective measurement that can reflect various diseases, such as arthritis, trauma, and stroke [75]. Abnormal ROM was the most studied smartphone-based hand function assessment [24-27,32,37-41,45-47,49,51,59,60,65], indicating the advantages of smartphones in obtaining ROM parameters. Therefore, the further development of smartphones to achieve better accuracy and reliability in capturing ROM is warranted. With the advancement of built-in accelerometers and gyroscopes in smartphones, capturing and analyzing hand ROM data have become more accessible [75,76]. Furthermore, smartphones can accurately measure both dynamic ROM and static ROM, providing good potential for long-term monitoring even without the presence of professionals [27].

PD is the most studied disease that causes hand dysfunction. PD can cause multiple hand dysfunctions, such as tremors [6,9,10,42,48], bradykinesia [6,9,43,48], abnormal ROM [37,39,45], and fine hand use decline [44]. It provides evidence that smartphones have the potential to provide a comprehensive assessment platform for multiple hand dysfunctions [9,42-44].

In addition, chronic neurodegenerative diseases, such as PD, exhibit progressive symptoms that require continuous monitoring [7]. However, existing clinical assessment tools, such as MDS-UPDRS, tend to be subjective, time constrained, and time consuming [77]. Smartphone apps could exploit the multiple built-in sensors in smartphones to detect changes indicative of the disease progression or treatment response [78-82], indicating that smartphones can be prosperous tools for managing chronic hand dysfunction in the long run.

Above all, for a reliable clinical application of hand dysfunction assessment, the following should be achieved:

Real-time assessment during hand rehabilitation is beneficial in clinical practice because it allows the modification of the rehabilitation tasks and goals according to an individual’s specific needs and ongoing recovery progress [85]. In our review, studies on real-time smartphone-based assessment were primarily conducted between 2016 and 2022, indicating an emerging trend focusing on real-time hand assessment. A potential technical challenge may lie in identifying the best sensor configuration and feature extraction method for hand function assessment [6,84].

The early detection of a degenerative disease through hand assessment is important because it can help slow down further disease progression [86]. The reviewed literature discussed conditions such as PD and CTS [34,36,37]. Future work could use smartphones for biomarker acquisition to monitor disease-relevant physiological and behavioral symptoms and provide personalized rehabilitation guidance [87-89]. The use of smartphones for biomarker acquisition offers advantages, including portability, accessibility, affordability, noninvasiveness, and continuous monitoring, benefiting both patients and clinicians [90]. However, challenges exist in terms of data quality, reliability, and privacy concerns [91].

Smartphones were mostly used for data collection. With more sensors embedded in smartphones, richer and more dimensional data can be collected for function measurement. For example, the resolution of smartphones’ built-in camera is between 1920×1080 and 2400×1080 pixels, which is higher than the commonly used camera resolution in clinical settings, which typically ranges from 1280×720 to 1920×1080 pixels [8]. Compared to smartwatches and ring-shaped sensors, smartphones are more indispensable in people’s daily lives, making them an easily available assessment tool and requiring no extra investment like others [92]. While webcams provide high resolution and frame rates, they rely on a stable internet connection and can potentially raise privacy and security concerns [93]. In comparison, smartphones can collect data offline and protect the patient’s privacy by encrypting data, anonymizing personal information and storing data locally [40]. This also shows that smartphones, as general-purpose devices, do not require excessive hardware requirements, are available at a low cost, and are easy to access. Smartwatches and wearables usually feature multiple sensors similar to those found in smartphones, allowing for the collection of hand motion and physiological data with real-time feedback. However, their functionality is confined by a fixed position of the body, resulting in the limited scope of data collection [14]. In contrast, smartphones, being portable devices, are not constrained by fixed positions, granting convenience and flexibility for hand dysfunction assessment. Ring-shaped sensors offer high precision and accuracy and provide real-time data. However, their use may be limited due to comfort and portability constraints [16]. Smartphones are equipped with data processing modules, which can analyze and process data in real time, providing better accuracy at the same cost [94]. In terms of user experience, as a more familiar product, smartphones reduce the users’ learning cost and provide a more convenient, personalized, and friendly hand dysfunction evaluation experience, which helps improve user participation and satisfaction [19]. However, one of the weaknesses of using a smartphone for data collection may be data errors or biases due to the smartphone user’s lack of training, supervision, and quality control [95].

Using smartphones for data processing was the least mentioned in the studies [24-26,32,39,42]. The benefits of smartphone data processing are manifold, including mobility, real-time processing, and interactive nature [96]. This empowers users to access and process data at any time, receive real-time feedback, and seamlessly interact with their smartphones, regardless of location [97]. Despite the advantages, there are also obstacles to overcome, including short battery life, limited storage capacity, and weak processing power [98]. Therefore, most of our reviewed studies focused on the wireless transmission of data to computers or the cloud for subsequent data processing [6,10,39,46]. This approach would allow for efficient data management and processing without consuming the limited storage space available in smartphones (Figure 2) [10].

In this review, among the 46 studies, 7 (15%) exclusively involved healthy participants, while 23 (50%) recruited both patients and healthy participants. Consequently, 65% (30/46) of the studies included healthy participants, marking a noteworthy finding. In smartphone-based hand dysfunction assessment, incorporating baseline data from healthy participants is important for several reasons [37-41,59,60]. First, a standard reference range is typically derived from data collected from healthy participants, which could enable a more precise evaluation of a patient’s hand dysfunction. By comparing the hand function of patients to that of healthy participants, potential abnormalities can be identified more effectively, assisting in the accurate diagnosis of issues and facilitating the implementation of appropriate treatments [6,10,11,29,42-55,61-65]. Second, during the rehabilitation process, the patient’s recovery progress and improvement can be quantified by comparing against data from health people [45,46]. The effectiveness of the treatment can be more accurately assessed, and rehabilitation protocols could be adjusted for better outcomes. Third, it’s necessary to establish a normal reference range from healthy participants, including different ages, sex, and demographic characteristics. A broader set of data is available, ensuring that assessments are not limited to a specific group and can cover a broader population, resulting in a complete and more comprehensive understanding of hand function assessment [99]. In summary, remote assessment platforms have been developed for a wide range of users, including professionals, caregivers, and patients [2,10]. However, certain aspects need to be considered when using smartphones for hand assessment. They are as follows [28,100-102]:

Statistical methods (39/46, 85%) were more commonly used than ML methods (20/46, 43%). The most commonly used statistical method was summary statistics such as mean and SD. Summary statistics offer concise insights into data, facilitating comparisons and simplifying analysis [103]. However, they can be subjective, relying on expert experience, and may distort information [104]. In addition, due to the multiple independent variables present in hand function assessment [83], it is important to consider statistical methods that are capable of analyzing a multifactor model, such as multiple linear regression [105].

ML methods have been increasingly used in various health care apps [106]. In the studies in our review, ML methods were mainly used for detecting and classifying patient hand posture, analyzing and classifying behavior patterns (ie, tremor, bradykinesia, and ROM), and identifying disease severity and prediction. Our review found SVMs to be the most commonly used ML algorithm, particularly for disease classification. This may be attributed to the fact that SVMs are capable of effectively addressing multi-dimensional data with small sample sizes while providing a good generalization performance and the ability to work with the primary processing stage data [107]. The main limitation of the SVM algorithm is its inability to handle multiclass classification problems without additional modifications or extensions [108].

The strengths of this review are as follows: (1) the relevant database searches were conducted in a comprehensive and reproducible manner; (2) this was the first review that aimed to comprehensively discuss the role of smartphones and their functionalities in hand assessment from a holistic perspective; and (3) this review provides an analytical demonstration of the technical feasibility and advantages of using smartphones for hand function assessment across various domains, including sensor support, clinical practice, and application scenarios. It recommends potential directions for future studies in this field, such as multisensor fusion, gold-standard establishment, real-time assessment, and ML algorithms for data analysis exploration. This review also has some limitations. First, given that smartphone-based hand function assessment is at its nascent stage, the number of relevant studies is limited. This may contribute to a lack of sufficient evidence, completeness, and comprehensiveness in research materials, posing challenges in supporting viewpoints, drawing conclusions, and gaining a comprehensive understanding of the field. Second, this review encompassed only studies in the English language. Third, due to the exploratory and developmental nature of this topic, the literature quality varied, with potential limitations, such as inconsistency and a lack of high-quality reference studies and as well as meta-analyses.

This systematic review focused on how smartphones are used for hand function assessment. It covered the evaluation and measurement of hand dysfunction caused by various diseases, different embedded smartphone sensors, and statistical and artificial intelligence methods for hand function assessment. The evidence demonstrated that smartphones could facilitate a convenient, inexpensive, and reliable hand-functional assessment [9,10,44]. Future research could (1) explore how to develop a gold standard for smartphone-based hand function assessment; (2) take advantage of smartphones’ integrated systems with multiple sensors to collect patients’ data in various dimensions to assess hand function holistically; and (3) develop ML methods that are more suitable for processing data collected by smartphones. On the basis of the growing capabilities of smartphones for data collection and analysis, digital technology holds promise for bringing revolutionary changes to hand function assessment.