Imagine an individual at home, trying to complete a simple finger-tapping or speech task for an AI-based health assessment. The setup looks straightforward, but the camera angle cuts off their hand, the lighting obscures their face, or the microphone captures more background noise than their voice. For a clinician, these errors are obvious. For an algorithm, they are harder. The system cannot easily distinguish between actual motor symptoms and poor video quality. Without safeguards, the very recordings meant to extend access can instead mislead or fail altogether.

Neurological movement disorders such as essential tremor, Huntington disease, and Parkinson disease (PD) affect tens of millions worldwide [1]. The number of people living with PD alone doubled from about 3 million in 1990 to over 6 million in 2015 and is projected to surpass 12 million by 2040 [2]. Yet, access to neurological care remains scarce: in many countries, there is less than 1 neurologist per 100,000 people [3,4], and even in well-resourced regions, rural and low-income communities face hours-long travel for appointments [5]. Older adults, the most affected group, are often least able to make these trips, leaving many undiagnosed or untreated [6,7]. Remote assessment offers a promising alternative, enabling frequent and accessible evaluations without the burden of in-person visits [8].

Compared to wearable sensors, audio/video (A/V) recordings may have clear advantages. Sensors require shipping, setup, and device literacy, which can be daunting for older users [9]. In contrast, A/V can be captured using everyday devices like smartphones or webcams, preserving the clinically familiar cues of movement and speech [10]. Recent studies show that automated A/V analysis can support screening and progression tracking for finger-tapping [9,11], facial expression [12,13], and speech tasks [14,15]. While these assessments provide a scalable means for remote screening [16], they cannot address the shortage of neurologists if clinicians must still supervise recordings. Unsupervised home-recorded multimedia data, though promising for detecting disorders such as PD [17] and ataxia [18], introduces new challenges. Older adults may struggle to follow instructions [19], and uncontrolled environments often result in poor lighting, occlusion, or distracting background noise. In clinics, staff can intervene to correct these issues, but at home, no such safeguard exists. As a result, predictive models trained on low-quality data risk basing their decisions on noise rather than genuine symptoms, undermining both trust and clinical utility. Ensuring automated quality checks is therefore essential to safeguard patient safety and preserve the reliability of machine learning systems.

Movement disorder screening is task-based, with each task targeting specific clinical symptoms [20]. While general A/V quality checks (eg, background clutter or lighting) are helpful, they are insufficient for assessing task compliance. For example, finger-tapping tasks [21,22] require clear visibility of the thumb and index finger; facial expression tasks [12] require unobstructed frontal views; speech tasks [23] require clean audio without noise. Existing quality checks [24,25] typically address generic capture issues, leaving compliance to manual review [26]. A prior study [27] has also emphasized the need for clearer instructions, camera framing assistance, and real-time quality feedback, suggesting that built-in guidance and automated checks could enhance both task compliance and user engagement.

We address this gap with NeuroSift (see Figure 1 for an overview), a task-aware quality assurance framework for 3 remotely collected tasks used in PD assessment: finger-tapping, facial expression (smile), and speech (pangram utterance) tests. Although our dataset is collected primarily in the context of PD assessment [21], these tasks capture functional domains that are also relevant across Huntington disease [28], cerebellar ataxia [29], amyotrophic lateral sclerosis (ALS) [30], dystonia [31], and other neurodegenerative conditions. Each of these tasks introduces unique quality and compliance considerations: hands may drift out of frame during tapping, facial expressions may be obscured by poor lighting, and speech may be masked by background noise. To address these challenges, we curate a dataset of 2516 home-recorded audio or video segments spanning these tasks, annotate each segment with expert-provided quality labels, and develop task-specific quality guidelines. These guidelines help raters to assess observable quality issues objectively and significantly improve interrater agreement (at least a 0.23 gain in weighted Cohen κ score across all 3 tasks; P<.001) while offering practical checklists for both clinicians and study designers.

Building on these guidelines, we engineer interpretable computational features—13 for finger-tapping, 16 for facial expression, and 6 for speech—directly tied to observable aspects of quality and compliance. Using these features, we train machine learning models that achieve 83%, 77%, and 90% ordinal accuracy in predicting 3-class quality labels (poor, borderline, and good) across the finger-tapping, facial expression, and speech tasks, respectively. Beyond performance, the feature-based models support explainability. Using Shapley additive explanations (SHAP [32]), a model-interpretation method that estimates how much each input feature contributes to a prediction, we can identify the key factors (eg, hand visibility, facial illumination, and background noise) that drive model prediction. As a result, instead of a “black-box” rejection, users can potentially receive actionable feedback: why the recording failed and how to fix it.

The aim of this study is to develop and evaluate NeuroSift as a task-aware, interpretable quality-assurance framework for user-recorded multimedia data in remote movement disorder assessment. Specifically, we ask: (1) Can structured, task-specific annotation guidelines improve expert agreement when rating recording quality and task compliance? (2) Can interpretable, task-specific features support automated classification of recording quality across finger-tapping, facial expression, and speech tasks? (3) Can model explanations identify actionable sources of recording failure that may guide users toward successful rerecording? By addressing these questions, this study evaluates whether automated quality assurance can serve as a practical safeguard for scalable, reliable, and transparent remote neurological assessment, using PD as a specific use case.

In this study, we used data collected via PARK [71], a web-based platform designed for large-scale remote assessment of PD. The PARK platform allowed participants to enroll and complete study procedures outside of a clinic, primarily from their homes. Recruitment was conducted (from 2017 to 2025) through several predefined channels, including invitations sent through the University of Rochester Brain Health Registry, referrals from clinicians and research collaborators, and public outreach through social media and related online postings. These channels were used to reach both individuals with PD and individuals without PD. Eligible participants were required to be at least 18 years old, able to provide informed consent, and have access to an internet-connected laptop. Participants who self-reported having movement disorders other than PD were not enrolled. Sex and race/ethnicity were not used as eligibility or recruitment criteria. No formal sample-size calculation or enrollment ceiling was used. The study size was determined by the availability of willing and eligible participants who enrolled through the remote platform and completed the relevant recording tasks.

Interested participants accessed the PARK platform, reviewed the study information, completed an electronic informed-consent process, and provided demographic information and self-reported PD diagnosis status before completing the recording tasks. Because enrollment and participation occurred remotely, diagnosis status was not independently verified by a clinician as part of this study. The study did not include any follow-up data collection.

A total of 1444 confirmed eligible participants contributed demographic information and A/V recordings (primarily from home) across 21 standardized tasks spanning motor functions (eg, finger-tapping, hand movement, and fist making), facial expressions (eg, disgusted face, smile, and surprise), and speech (eg, sustained phonation, pangram utterance, and tongue twister). Before each task, the platform displayed an instructional video and task-specific on-screen prompts. Recordings were completed without real-time supervision, which allowed the study to capture the types of recording quality and task-compliance issues that can naturally occur in home-based multimedia assessment.

Among the participants, 750 identified as female, 609 as male, 1 as nonbinary, and 84 did not disclose their gender. The largest self-identified racial group was White (892/1444, 61.8%), followed by Black or African American (46/1444, 3.2%), Asian (45/1444, 3.1%), and smaller proportions identifying as American Indian or Alaska Native, Native Hawaiian or Pacific Islander, or multiracial (with over 20%, 289/1444) choosing not to disclose their race. The mean age of the participants was 61.7 (range 18-89; SD 13.7) years, reflecting the older demographic most affected by movement disorders. Among all the participants, 472 (32.7%) self-reported being diagnosed with PD (we do not have information regarding other movement disorders, as data were primarily collected for PD assessment).

The study was approved by the University of Rochester Research Subjects Review Board (RSRB/IRB) under protocol STUDY00001369 (Parkinson Remote Data). All participants provided informed consent for the use of their data in this research. Participants whose images are included in this manuscript also provided consent for the use of their images in research dissemination, including scholarly publications.

We selected three standardized tasks for automated quality assurance:

We selected these 3 tasks based on their clinical relevance across movement disorders, feasibility for unsupervised home settings, and participant safety. Combined, these tasks capture complementary motor and speech modalities and are frequently used in clinical assessments of movement disorders. Finger-tapping is a core test for quantifying bradykinesia in PD [21]. It is also used to assess coordination deficits in Huntington disease [28] and cerebellar ataxia [29]. Facial expression tasks, such as smiling, capture hypomimia in PD [12], facial weakness in ALS [30], and dystonic movements in tardive dyskinesia [31]. Speech tasks, including sustained phonation and short reading passages, are routinely applied to measure dysarthria [74] and articulatory breakdowns [75] across PD, ALS, multiple system atrophy, and spinocerebellar ataxia. Tasks such as gait assessment [76], while well established in neurological research, pose practical challenges in remote contexts. Completing this task at home requires ample walking space in front of the camera and full-body capture, and can be unsafe (ie, risk of falling) for individuals with advanced mobility impairments. In contrast, the chosen tasks can be performed while seated, require only a webcam and microphone, and pose minimal safety concerns. Importantly, each task taps into functional domains commonly affected across a range of movement disorders: finger-tapping probes motor speed and coordination, smile mimicry captures facial expressivity, and pangram utterance reflects articulatory precision and prosody. In addition, the combination of these 3 tasks has recently been shown to support accurate detection of PD [17], further validating their practical utility for neurological assessments.

We recruited 3 annotators with 2-5 years of relevant expertise to establish task-specific guidelines and generate reliable datasets for automated quality classification. Two were doctoral students specializing in video-based movement disorder assessment, and the third was an undergraduate researcher experienced in analyzing home-recorded videos for PD. All annotators were familiar with the MDS-UPDRS [20] criteria, the clinical standard for evaluating PD, which supported consistency and credibility of their quality ratings.

Annotation proceeded in multiple phases. First, each annotator independently rated 50 randomly selected videos per task into three classes: 0 (poor), 1 (borderline), and 2 (good). Disagreements were resolved through collaborative discussion, during which the annotators jointly finalized labels for all videos. Based on this process, the team developed draft annotation guidelines tailored to each task. In the second round, they applied the guidelines to a new set of videos (71 for the finger-tapping task; 50 for the smile and speech tasks), again rating independently and resolving disagreements collaboratively. Only minor refinements of the guidelines were made after this stage, and interrater agreement improved significantly during the second round of ratings, with quadratic weighted Cohen κ (QWK) increasing by at least 0.23 across tasks (P<.001). Because recording quality and task compliance do not have an objective biological ground truth, we treated expert consensus labels as the reference standard for model evaluation. For the held-out test sets, 3 annotators first rated each recording independently using the task-specific guidelines. Disagreements were then resolved through collaborative adjudication, resulting in a single consensus reference label for each recording. These labels should therefore be interpreted as expert-derived consensus labels rather than absolute ground truth.

The borderline class was used for recordings that were not clear failures but contained minor quality or task-compliance issues that could reduce confidence in automated analysis. For example, speech recordings were rated as borderline when the participant remained intelligible despite background noise, a brief pause, limited repetition, or minor word omissions. Smile recordings were rated as borderline when the face remained visible despite mild blurriness, partial eyeglass glare, extra smiles, or slightly suboptimal camera distance. Finger-tapping recordings were rated as borderline when the hand remained mostly visible despite suboptimal lighting, partial wrist visibility, minor tap-count deviations, or slightly distant camera framing.

For analysis and deployment, the 3 quality classes reflect different levels of usability. Good-quality recordings can be passed confidently to downstream PD assessment models. Borderline recordings may still be usable, but their downstream predictions should be interpreted with caution; depending on the application, they may either be accepted with a warning or trigger corrective feedback and rerecording, reflecting the tradeoff between data quality and user burden. Poor-quality recordings indicate substantial quality or compliance failures and would prompt the user to rerecord.

For developing the training datasets, each annotator rated approximately one-third of the remaining videos following the final guideline. Overall, this process resulted in task-specific annotation guidelines, a reliable consensus-labeled test set, and relatively larger training sets for model development. A summary of the datasets is provided in Table 1.

In this section, we describe how each of the 3 tasks was recorded, summarize the quality issues identified by expert annotators, and outline the task-specific guidelines developed by the team.

We first assessed interrater agreement on an initial set of 50 videos (per task), independently annotated by 3 experts without guidelines. Agreement metrics were then reassessed on a second set of 50+ videos (50 each for the smile and speech tasks; 71 for the finger-tapping task) after the introduction of structured, task-specific guidelines. As shown in Table 6, the guidelines significantly improved agreement across all tasks. In the finger-tapping task, QWK nearly doubled (0.46 to 0.89), pairwise agreement increased from 62.5% to 92.7%, and complete agreement rose from 47.8% to 89.9%. For the smile task, QWK improved from 0.61 to 0.84, and for speech, from 0.64 to 0.90. ICC values also increased consistently, with finger-tapping showing the largest jump (0.51 to 0.90). All improvements (Δ) were statistically significant (P<.001), based on the Mann-Whitney U test with bootstrap resampling. These results indicate that even experts initially struggled to annotate A/V data quality consistently, but structured guidelines provided a shared reference framework that reduced subjectivity, aligned judgments across raters, and enabled the creation of reliable training and test data for downstream modeling.

Building on the improved reliability of expert annotations, we next evaluated how well machine learning models could replicate these judgments on the 3-class quality prediction. Table 7 presents the performance of the best-performing model for each task (see Table S1 in Multimedia Appendix 2 for results from all 5 models tested; we also provide the confusion matrices for the best-performing models in Figures S1-S3 of Multimedia Appendix 3). For finger-tapping, random forest achieved the highest performance with QWK=0.71 and OCA=82.1%. For speech, random forest again performed best, reaching QWK=0.72 and OCA=89.9%. In contrast, smile proved more challenging: LightGBM was the top-performing model but with a comparatively lower QWK of 0.56 and OCA of 76.9%. Accuracy followed similar patterns, with finger-tapping and speech models showing higher consistency across classes compared to smile.

To better characterize model behavior under class imbalance, we further examined per-class performance for the 3-class models. Performance was highest for the poor and good classes and lower for the borderline class across all tasks. For speech, precision/recall/F₁-score were 0.91/0.91/0.91 for poor, 0.38/0.38/0.38 for borderline, and 0.88/0.88/0.88 for good recordings. For smile, the corresponding values were 0.52/0.50/0.51 for poor, 0.30/0.23/0.26 for borderline, and 0.74/0.81/0.77 for good recordings. For finger-tapping, they were 0.82/0.77/0.79 for poor, 0.27/0.32/0.29 for borderline, and 0.78/0.78/0.78 for good recordings.

The confusion matrices in Multimedia Appendix 3 further show that most errors involved the borderline class. In the speech task, the dominant errors were borderline recordings classified as good and good recordings classified as borderline, while severe poor-versus-good confusions were rare. A similar pattern was observed for smile and finger-tapping, where many errors involved borderline recordings being assigned to an adjacent class or adjacent classes being assigned to borderline. This pattern suggests that the models generally captured the ordinal structure of the labels but struggled with intermediate-quality recordings, which are also the most ambiguous for human annotation and deployment decisions.

To support applications with different reliability requirements, we also trained models in 2 binary setups: a restrictive configuration (borderline merged with poor) and a flexible configuration (borderline merged with good). The restrictive setup is suited to contexts where ensuring data fidelity is critical. In contrast, the flexible setup allows for broader inclusion by treating borderline recordings as usable. Table 8 summarizes the results of the best-performing models for each task (refer to Table S1 in Multimedia Appendix 4, where we report the performance of all 5 models tested). In the restrictive setup, accuracies reached 81.8% for finger-tapping, 77% for smile, and 83.9% for speech, with AUROC values between 83.9 and 87.1. In the flexible setup, performance was generally higher, with accuracies of 83.5% for finger-tapping, 80.1% for smile, and 95% for speech. AUROC values were highest for speech at 99.5, demonstrating that the model was near-perfect in identifying poor speech recordings.

Overall, these results indicate that automated models can approximate expert ratings with substantial agreement, though performance varies across tasks and remains lower for the smile task.

We used SHAP [32] to examine how individual features contributed to model predictions. Figure 6 shows an example in which the model correctly predicted a finger-tapping recording as poor. In this example, the largest drivers were extended idle time and unusually long video duration, while features such as brightness ratio and wrist visibility played smaller roles.

Beyond individual predictions, SHAP also highlighted global patterns in model behavior. As shown in Figure 7, features such as wrist visibility, tap count, and idle time consistently emerged as strong indicators of finger-tapping task quality. For the smile task, waist visibility and the ratio of eyeglass and face brightness played important roles (Figure S1 in Multimedia Appendix 5). For the speech task, similarity between the synthesized and target utterance, audio duration, and number of speakers were central contributors (Figure S2 in Multimedia Appendix 5).

The aim of this study was to develop and evaluate NeuroSift as a task-aware, interpretable quality-assurance framework for user-recorded multimedia data in remote PD assessment. Specifically, we evaluated whether structured task-specific guidelines could improve expert agreement in quality annotation, whether interpretable features could support automated recording-quality classification across finger-tapping, facial expression, and speech tasks, and whether model explanations could identify actionable sources of recording failure. Overall, the findings support these aims. The annotation guidelines improved interrater agreement across all 3 tasks, suggesting that structured criteria can reduce subjectivity in quality assessment. The task-specific classifiers achieved moderate to strong performance against expert-derived consensus labels, with stronger agreement for finger-tapping and speech than for smile recordings. Finally, SHAP-based model explanations identified quality issues that were consistent with the expert guidelines, such as hand visibility, tap count, idle time, camera framing, eyeglass glare, and speech-related noise. Together, these findings suggest that task-aware quality assessment can provide a practical layer of quality control for remote multimedia-based PD assessment, while also identifying several areas that require further validation before clinical deployment.

This study addresses an important challenge in remote digital health: unsupervised, user-recorded audio and video data can contain quality and task-compliance problems that are obvious to clinicians but difficult for automated systems to handle reliably. Unlike general video quality checks, NeuroSift focuses on task-aware safeguards. For example, it evaluates whether the hand remains visible during finger-tapping, whether facial expressions are adequately captured during the smile task, and whether speech is sufficiently audible and aligned with the target utterance. This distinction is important because a recording may be technically clear but still unusable for a specific clinical task if the relevant body part, movement, or speech segment is missing or poorly captured.

A central finding is that quality assessment itself can be subjective without structured criteria. The improvement in interrater agreement after guideline implementation suggests that explicit, task-specific annotation rules can provide a shared reference for raters and reduce ambiguity. This is particularly important for the borderline class, which represents recordings that are not clear failures but contain minor quality or compliance issues that may reduce confidence in downstream analysis. In practice, good-quality recordings can be passed more confidently to downstream PD assessment models, borderline recordings may be used with caution or may trigger corrective feedback depending on the quality-versus-user-burden tradeoff, and poor-quality recordings should prompt rerecording. This 3-level structure is more informative than a simple usable-versus-unusable decision, especially in remote settings where repeated rerecording may burden older adults or people with movement disorders.

The automated quality models reproduced expert-derived labels with varying performance across tasks. The stronger performance for finger-tapping and speech suggests that many quality issues in these tasks can be captured through interpretable features, such as wrist visibility, tap count, idle time, audio duration, speech similarity, and number of speakers. Smile quality assessment was more difficult, consistent with both lower postguideline interrater agreement and weaker model performance. This may reflect the inherent difficulty of evaluating facial expression quality in PD, where hypomimia can make it challenging to distinguish incomplete task performance from disease-related reduced expressivity. The per-class results and confusion matrices further suggest that borderline recordings are the most difficult to classify. Most errors involved borderline recordings being assigned to an adjacent class, or poor or good recordings being assigned to borderline, rather than severe poor-versus-good confusions. This pattern is consistent with the ordinal nature of the labels and supports the use of ordinal evaluation metrics.

Beyond PD assessment, the need for task-aware quality assurance is relevant to many user-recorded multimedia workflows. Remote education platforms often rely on live-streaming video presentations [84], digital recruitment systems request video responses to interview questions [85], and telehealth services routinely use patient-recorded videos for screening and monitoring [86,87]. In these settings, poor lighting, occlusion, background noise, or noncompliance with task instructions can affect both human interpretation and automated analysis. Prior work on telehealth usability and video-mediated communication also highlights that older adults and nonexpert users may face challenges in configuring devices, positioning themselves correctly, and producing usable recordings [88,89]. NeuroSift contributes to this broader design space by showing how quality guidelines, interpretable features, and feedback mechanisms can be aligned around task-specific requirements.

An important strength of NeuroSift is that its predictions can be traced to interpretable, task-relevant features. By surfacing the factors contributing to a quality prediction, the model can communicate not only that a recording failed but also why it failed. This information could be translated into actionable guidance, such as reducing idle time, completing more taps, improving hand visibility, or adjusting camera position.

However, raw SHAP plots are unlikely to be appropriate as direct feedback for participants, particularly older adults completing assessments without supervision. In deployment, these explanations would need to be translated into short, plain-language prompts linked to corrective actions. For example, features indicating poor hand visibility could be presented as “move your hand closer to the camera,” speech-noise features as “try recording again in a quieter room,” and face-framing or lighting features as “adjust the camera so your face is centered and clearly visible.” Locally deployed large language models could also be fine-tuned to translate SHAP-derived feature explanations into personalized, actionable feedback while avoiding transmission of raw audio or video data to external servers. Such feedback should avoid technical terminology, minimize cognitive burden, and limit repeated rerecording requests that may frustrate users. Future work should evaluate these feedback messages through user studies with older adults and people with movement disorders, assessing not only recording quality but also usability, trust, accessibility, and willingness to complete rerecordings.

These findings also have design implications for remote health tools. Quality assessment should not function only as a rejection mechanism; it can also become a user-support mechanism. Task-specific checks can be integrated into recording interfaces to provide real-time prompts before submission, such as asking users to recenter their hands, adjust lighting, or reduce background noise. Because NeuroSift uses lightweight models and interpretable features, quality screening may also be feasible directly on user devices. On-device assessment could reduce exposure of sensitive audio and video data while still providing immediate feedback. More broadly, by decomposing each task into measurable indicators of quality and compliance, task-aware safeguards could support remote assessment workflows in other health domains, including poststroke rehabilitation, speech or language therapy, and respiratory or mental health screening.

While our findings underscore the promise of automated quality assurance, several limitations remain. The dataset shows demographic imbalances, with limited representation of some racial and ethnic groups, which may affect model generalizability. Although the preliminary subgroup analyses suggested broadly equitable performance, the small number of minority samples reduced statistical power, and we observed significant discrepancies in speech quality classification across ethnic subgroups. These differences may reflect multiple factors, including dialectal variation, accent, microphone quality, background noise, and limitations of off-the-shelf automated speech recognition (ASR—a technology that converts spoken audio into text for analysis) modules [90], where racial disparities are well documented [91]. For example, African American participants may more frequently use African American Vernacular English [92], which is underrepresented in our dataset and could contribute to disproportionate outcomes if not carefully addressed. In deployment, quality assurance systems should avoid treating dialectal variation, accent, speech impairment, or differences in technology access as automatic evidence of poor-quality data. Low-confidence or borderline predictions may be better handled through corrective feedback, optional human review, adaptive thresholds, or alternative task pathways rather than automatic exclusion. Addressing these biases is critical, particularly because underrepresented groups and older adults often face greater barriers to accessing clinical care and may benefit substantially from home-based assessment. Future work should expand data collection among underrepresented groups, evaluate subgroup-specific error rates, calibration, and rejection rates, and develop strategies to mitigate bias in automated quality classification.

Residual uncertainty remained possible in the expert-derived reference labels, particularly for borderline recordings, where recordings may contain minor quality issues without being clearly unusable. This uncertainty is reflected in the postguideline agreement metrics, including complete agreement rates that remained below 90% across tasks. For the larger training datasets, each recording was labeled by a single annotator using the finalized guidelines, potentially introducing annotator-level label noise. We addressed this limitation by using structured annotation criteria, preserving the ordinal 3-class label structure, applying ordinal evaluation metrics, and evaluating final model performance on consensus-labeled held-out test sets.

Another limitation is the close coupling among the annotation guidelines, feature design, and model training. This coupling was intentional: the goal of NeuroSift was to produce interpretable quality predictions that could be traced back to observable recording-quality and task-compliance criteria. However, this design also means that the models primarily learn the expert-defined operational criteria encoded in the guidelines, rather than an independently established or universal definition of recording quality. Although we evaluated the models on held-out participants using consensus-derived reference labels, the reference labels and engineered features were still derived from the same annotation framework. Therefore, the present results establish internal validity within this quality-assessment protocol but do not fully establish external validity across different recording platforms, populations, environments, or annotation standards. Future work should validate NeuroSift on independently collected datasets and compare its predictions against alternative or independently defined quality criteria.

Although the predictive models aligned well with human raters for the finger-tapping and speech tasks, agreement was weaker for the smile task. While our feature sets were designed to closely mirror the quality assessment guidelines, they may not capture all relevant aspects of facial recordings. Some features (eg, detected peaks) depend on reliable smile detection, which is particularly challenging in individuals with PD due to hypomimia (ie, reduced facial expressiveness). Moreover, the smile task showed the lowest interrater agreement among the 3 tasks, even when raters followed the guidelines. Together, these findings suggest that smile quality assessment is inherently difficult, underscoring the need for richer feature representations and more robust annotation protocols for facial recordings.

NeuroSift generalizability requires further validation because all recordings were collected through the PARK framework. Importantly, however, PARK is a remote, web-based platform that captures recordings in participants’ natural home environments rather than under highly controlled laboratory conditions. Consequently, the dataset includes substantial variability in devices, browsers, lighting conditions, background noise, network environments, and user behavior, providing greater ecological validity than many laboratory-based datasets. Nevertheless, because all data were collected using a single platform and protocol, shared platform-specific characteristics may still influence feature distributions and model performance. Similar robustness concerns have been reported in movement disorder modeling, where small or heterogeneous datasets, class imbalance, and distributional artifacts can affect generalizability [93]. Although we used subject-independent test sets, consensus-derived labels, task-specific guidelines, and interpretable features, the present results establish internal validity within the PARK framework. Future work should evaluate NeuroSift on independently collected datasets and assess whether recalibration, domain adaptation, or platform-specific thresholds are needed to maintain performance across different recording environments. Beyond technical performance, the user experience of automated quality checks remains largely unexplored. Future work should examine both the benefits and burdens of rerecording requests through end-to-end studies that capture not only quantitative gains in data reliability but also participants’ qualitative perspectives on usability and trust.

Finally, this study did not empirically test whether NeuroSift improves downstream PD prediction performance, model calibration, or clinical decision-making. The present work focused on developing and validating the quality-assurance framework itself, rather than evaluating a complete diagnostic pipeline. A rigorous downstream analysis would require an independently evaluated PD assessment model, sufficient held-out data with complete task recordings and reliable clinical labels, and prespecified comparisons across quality-gating strategies. Future work should compare downstream model performance with no quality filtering, flexible filtering that retains borderline recordings, and restrictive filtering that accepts only good-quality recordings. Such studies should report not only predictive performance but also calibration, coverage, user rerecording burden, and impact on clinical workflow.

In summary, NeuroSift demonstrates how task-aware quality assurance can be built into remote multimedia-based PD assessment. The broader implication of this work is that reliable remote health AI requires more than accurate downstream prediction models; it also requires safeguards that determine whether user-recorded data are sufficiently clear, complete, and compliant for analysis. By combining expert-derived quality guidelines, interpretable features, lightweight classifiers, and explanation-based feedback, NeuroSift provides a framework for making quality assessment more transparent and user-correctable. At the same time, the findings should be interpreted as validation of a quality-classification framework, not as evidence that downstream clinical prediction or deployment outcomes have already improved. Future work should test NeuroSift in independent datasets and real-world workflows, evaluate its effect on downstream prediction and calibration, and study whether its feedback mechanisms are usable and fair across diverse populations. If validated in these settings, task-aware and explainable quality assurance could become an important design layer for remote health systems that depend on user-generated multimedia data.