The 2 subjective parameters—pain and activities of daily living—were assessed using Chinese versions of the Constant-Murley questionnaire [20].

For measuring ROM, patients were asked to complete several movement tasks as defined by the CMS (Multimedia Appendices 1 and 2). Their postures were recorded with mobile phone cameras. Images of external rotation were captured from the anterior side, internal rotation from the posterior side, and elevations from the lateral side. For better presentation, images that did not include the patient’s head or hip were excluded. The eligible images were then collected for model construction.

Landmark detection and parameter extraction were performed using the HPE algorithm BlazePose [21]. The model was run in static image mode, with complexity set to 2, minimum detection confidence set to 0.5, and all other parameters left at default values. As the CMS focuses only on upper limb movement, geometric features were generated from selected landmarks, including the nose, left and right shoulders, left and right elbows, left and right wrists, and left and right hips (Multimedia Appendix 3). To efficiently represent these geometric features, we converted the landmark coordinates into angles between vectors, generating 5112 features using the following algorithm (features based on 2D [x and y] and 3D [x, y, and z] coordinates were extracted separately for comparison):

where a and b are the vectors forming the angle.

To measure strength, we used the embedded inertial sensors in mobile phones to record shoulder abduction movement data. In this procedure, a mobile phone was attached to the participant’s upper limb. The participant was then asked to repeatedly perform shoulder abduction as quickly and continuously as possible, from a natural resting position to 90° of abduction. Data collection was repeated twice for each patient, with each repetition lasting 10 seconds and sampled at 50 Hz.

The collected data were processed to extract kinematic features representing the movement characteristics of shoulder abduction. The collected sensor data were filtered using a 10 Hz low-pass Butterworth filter implemented with the butter function from the Scipy.signal package in Python (version 3.9; Python Software Foundation). The filter order was set to 8, and the index was 0.3. Next, a 2-second sliding window with a 1-second step size was used to segment the data, resulting in 8 segments per session. From these segments, we extracted a set of time- and frequency-domain parameters (Multimedia Appendix 4) for model construction.

The eligible images were assigned to 2 independent raters for annotation, both of whom had undergone standardized training before scoring. Disagreements were resolved through discussion or consultation with a third investigator until a consensus was reached. These finalized annotations served as the gold standard for the subsequent comparisons. For external rotation, if patients correctly completed the required posture, the images were annotated according to the corresponding item number (classes 1-4). If patients were unable to perform the movement as required, the images were annotated as class 5. For internal rotation, the images were categorized into 6 classes based on the CMS.

Shoulder strength was measured manually following the procedure described by Constant et al [22]. Measurements were taken at 90° of shoulder abduction in the scapular plane at wrist level using a handheld digital dynamometer (microFET 2; Hoggan Scientific). The final score was defined as the maximum value of 3 repetitions, each separated by a 1-minute interval. Patients who were unable to achieve the test position were assigned a score of 0. The strength values were then converted into 5 annotation classes: 0 (0-5 lb [0-2.3 kg]), 1 (5-10 lb [2.3-4.5 kg]), 2 (10-15 lb [4.5-6.8 kg]), 3 (15-20 lb [6.8-9.1 kg]), and 4 (>20 lb [>9.1 kg]).

With minimum detection confidence set to 0.5, the HPE algorithm successfully extracted pose landmarks from 95.7% (850/888) of the included images. The detection rates were 96.1% (533/555), 99.1% (110/111), 87.4% (97/111), and 99.1% (110/111) for external rotation, internal rotation, forward elevation, and lateral elevation, respectively.

To optimize model performance, several data processing methods were conducted, including (1) the use of 3D parameters, (2) the incorporation of positive and negative reference data during training, and (3) class-balance training through oversampling. The results indicated that the highest accuracy was achieved in the model using 2D parameters with positive reference data (Tables 2 and 3). For external rotation, the support vector machine achieved the highest accuracy (92/112, 82.1%). For internal rotation, the best accuracy was obtained by both the random forest and KNN (23/28, 82%).

Next, the classification results were converted into CMS scores for interpretation (Multimedia Appendix 5). Confusion matrices summarizing true positives, true negatives, false positives, and false negatives are presented in Figure 2.

The reliability of the proposed method was further assessed by comparison with human raters. As shown in Table 4, the κ coefficients for the Mobile Constant method ranged from 0.389 to 0.819 (P<.01) for ROM measurements, and the ICCs ranged from 0.898 to 0.956. These values indicate fair to almost perfect agreement with the gold standard. The highest consistency was observed for internal rotation, while lateral elevation showed the lowest agreement. Notably, the proposed method achieved performance comparable to that of trained human raters.

At the model training stage, 83 patients and 5 healthy volunteers completed the data collection procedure, generating 176 records. After excluding ineligible data, 163 (92.6%) of the 176 records remained for model construction. At the internal validation stage, 56 records from 28 patients were collected for model evaluation.

As shown in Multimedia Appendix 6, the KNN model achieved the highest accuracy, followed by random forest, adaptive boosting, support vector machine, decision tree, and logistic regression. The oversampling method had a negligible effect on classification accuracy. However, we validated the impact through confusion matrices, which showed that models without oversampling performed poorly in classifying minority class samples (Multimedia Appendix 7). Therefore, the oversampling method was ultimately adopted for subsequent analyses.

Next, the classification results were converted into the strength scores. The results of the classifiers were first combined using majority voting, and larger value of repetitions was identified as the final result. As shown in Figure 3, the best performance was achieved by the KNN model, with an accuracy of 78.6% (22/28).

In addition, the reliability of the proposed method was evaluated against gold standard methods. In this comparison, the KNN model demonstrated the highest reliability with a κ coefficient of 0.707 and an ICC of 0.759 (Table 5).

The scoring results for patients in the internal validation set, assessed by the Mobile Constant and the gold standard, are summarized in Table 6. The mean difference between the 2 methods was 0.51 (SD 4.51).

On the basis of the performance in the internal validation set, models trained with 2D parameters and both negative and positive reference data were used. We selected the support vector machine model for external rotation assessment, the random forest model for internal rotation scoring, and the KNN model with oversampling for strength evaluation. The BlazePose algorithm achieved landmark detection rates of 97% (29/30 patients) for lateral elevation, 90% (27/30 patients) for forward elevation, 98.7% (148/150 frames across all patients) for external rotation, and 93% (28/30 patients) for internal rotation. Subsequent comparisons with manual assessments revealed substantial agreement across all tasks. Consistent with the observations in the internal validation set, external rotation exhibited the lowest interrater agreement (κ=0.198; ICC=0.584), while internal rotation showed the highest (κ=0.699; ICC=0.922). For strength assessment, the Mobile Constant system achieved significant agreement with manual ratings (κ=0.809; ICC=0.906). Agreements between model predictions and manual assessments are presented in confusion matrices in Multimedia Appendix 8. The overall mean difference between manual evaluations and system results was 1.21 (SD 3.66; Table 7).

Our study introduced the Mobile Constant method and demonstrated that it was an objective, valid, and reliable instrument for assessing patients with shoulder concerns. This technique makes it possible for patients to perform assessments of shoulder function using only a mobile phone. Mobile Constant could be used as an objective tool in clinical work, especially for remote monitoring.

The original CMS consists of 4 subscales, including 2 subjective items and 2 objective measurements. In our method, the 2 subjective parameters were evaluated using Chinese versions of patient-reported questionnaires [20]. ROM was then assessed using the HPE-based method, which can detect human landmark coordinates and provide quantitative analysis of movements. Although the reliability of the HPE algorithm for measuring ROM has been validated [14,15], methods for conducting movement scoring using this approach remain unclear. The primary challenge stems from the subjective assignment of scores and the absence of quantitative criteria. In this study, we demonstrated that machine learning classifiers using kinematic parameters represent a reliable solution. Original scoring was converted into movement classifications, which demonstrated good reliability.

Although this performance is still far from perfect, it is still comparable to the interrater consistency reported in our study and previous research. Rocourt et al [25] reported substantial variability in interrater reliability for ROM in CMS assessment, with ICCs of 0.460 for external rotation and 0.854 for internal rotation. Blonna et al [26] further identified that physician expertise can impact scoring accuracy, with limits of agreement reaching 7.3 for external rotation and 3.7 for internal rotation.

In our study, the Mobile Constant achieved ICCs of 0.899 for external rotation and 0.956 for internal rotation, with a mean difference of <1, indicating that the proposed method demonstrated reliability comparable to that of human raters.

In addition, our results showed performance comparable, if not superior, to previous research on remote evaluation based on the CMS. Levy et al [6] designed a patient-based questionnaire, reporting consistency with the human rater ranging from 0.80 to 0.90. Similar findings were observed for the Auto-Constant questionnaire [7], which reported an ICC of 0.85 for ROM. These results suggest that our HPE-based method provides acceptable reliability and holds potential as a useful tool for the remote assessment of shoulder movements.

The remote assessment of shoulder abduction strength presents several challenges. First, patients may have difficulty understanding and adhering to the testing protocol in unsupervised settings. In addition, the standardized measuring instrument is typically unavailable outside clinical environments. Recent studies have developed simplified frameworks for remote strength assessment. The most commonly used method is to ask patients whether they can lift objects (eg, a water bottle, packages containing flour or sugar, or a container of milk) while performing abduction [6,7]. This approach is feasible for remote measurement and has demonstrated good reliability, with an ICC of 0.57 [7]. However, results could be impacted by subjectivity and variations in object weight. In this study, we used embedded IMU sensors to extract kinematic data “and train machine learning classifiers. Our method achieved an accuracy of 79% (22/28) in the internal validation set and 73% (22/30) in the external validation set, demonstrating an objective approach to evaluating shoulder strength with good reliability.

In addition, we conducted tests to evaluate the effect of different data processing approaches. First, we compared the accuracy of models constructed using 2D versus 3D coordinates. Although 3D coordinates provide more detailed pose information and thus may help identify joint positions, which is essential for distinguishing similar postures, the results surprisingly indicated that incorporating 3D information deteriorated classifier performance. We attribute this mainly to estimation errors in 3D pose estimation, which remains a challenging problem in computer vision [21]. These findings also suggest that with the evolution of 3D pose estimation methods, the performance of HPE-based functional assessment would further improve. The next comparison tested the effect of including reference data. Positive reference data, collected from healthy volunteers, were used to demonstrate standard movements to the models, while negative reference data from patients with severely impaired shoulder function were used to present compensatory movements. Interestingly, we found that using negative reference data or combining positive and negative reference data improved model accuracy. Our results suggest a possible method to improve model performance when patient data are insufficient for model training. We also evaluated the effect of oversampling, which addressed the uneven distribution of strength data. Although overall classification accuracy seemed to decline after oversampling, the classifiers exhibited better performance in distinguishing instances in minority groups, as shown in the confusion matrices. This improvement indicates that classifiers possess practical ability in clinical settings because they can detect less common cases rather than simply grouping all samples in the majority class.