The study included an in-clinic screening visit, a screening period of up to 3 weeks, and a 4-week observation period at each participant’s home. Clinical scores for characterizing the severity of disease in patients with AD were assessed during the in-clinic screening visit. The in-home portion of the study included a 28-day observation period during which scratching was assessed every night by the Philips and Emerald DHTs (Figure 1).

The study enrolled men and women, aged 18 years and older, with moderate to severe AD, mild AD, or healthy volunteers (HVs). Participants with AD were required to have an established diagnosis for at least 12 months and meet the following key inclusion criteria at screening: patients were eligible for the mild AD cohort if they had a Validated Investigator Global Assessment scale for Atopic Dermatitis (vIGA-AD) score of 2 and ≥3% body surface area of AD involvement; and patients were eligible for the moderate to severe AD cohort if they had a vIGA-AD score of ≥3, ≥10% body surface area of AD involvement, an Eczema Area and Severity Index score of ≥16, and an excoriation component score of >0 in at least 1 body region. While patients with moderate to severe AD based on vIGA-AD score were eligible, those enrolled in this cohort all had AD of moderate severity according to vIGA-AD (vIGA-AD score of 3). Therefore, this group of participants is referred to as a “Moderate AD cohort.” Participants with AD were allowed to continue treatment for AD as long as they were on a stable regimen by the start of the observation period, and topical medications were permitted throughout the study for the treatment of AD flares. HVs were eligible if they were healthy on the basis of medical history at screening, did not have chronic itch, and reported a numeric rating scale (NRS) Peak Nocturnal Itch score of ≤1 (over a 7-day recall period) at screening.

Video recordings during sleep were captured in each participant’s home environment using a bespoke IR video camera developed by Emerald Innovations Inc. The camera consisted of an IMX219 8-megapixel sensor and an IR illuminator setup capable of capturing images in 720-pixel resolution at 21 frames per second. The camera was mounted on the wall at the head of the bed about 1 m above the bed surface. Participants were asked to record video during sleep hours for 8 nights over the 4-week observation period (ie, 2 nights per week).

Videos were masked by blurring faces before subsequent processing and labeling. A computer vision–based motion-filtering algorithm trained on historical data was applied to the masked video before human labeling. This algorithm automatically removed video segments containing no motion to reduce the duration of video presented for labeling to the human raters. Random spot checks were performed on video data from each participant to confirm that the motion-filtering algorithm did not inadvertently filter out any participant motion.

Independent raters watched the videos and provided labels for multiple tasks.

1. Scratching bout labeling: Two independent raters (R1 and R2) watched a video segment and labeled the start and end times of all scratching behavior episodes (defined as “any repetitive and rhythmic skin contact movement—including through fabric—performed with any part of the body”). Consensus was reached in the first round if the intersection over union of R1 and R2 annotations exceeded a threshold of 0.765 (empirically chosen based on historical rating data). If consensus was not reached, the video segment was rated by 2 additional raters (R3 and R4) in round 2 and the same consensus threshold was used. If consensus was reached in either round 1 or 2, the intersection of the 2 raters’ labels within the round reaching consensus was taken as the final Reference. If consensus was still not reached after round 2, a fifth rater (R5) decided on the final scratch labels in round 3. Raters had access to view labels from all previous rounds, but labels were independent within a given round.

2. TSO labeling: Two independent raters (R1 and R2) watched the full night video and determined when the participant got into bed intending to sleep (TSO start) and when the participant got out of bed for the last time, intending to stay awake (TSO end). If the 2 raters in round 1 did not agree (disagreement defined as >1-minute difference in either TSO start or TSO end), a third rater with access to labels from round 1 determined the TSO start and end labels.

The Emerald system consists of a sensor using frequency-modulated continuous wave radar and antennae arrays to receive reflections from nearby people. The Emerald RF system uses a machine learning algorithm that operates on the data recorded by the Emerald sensor to derive sleep-related metrics [19,20]. It uses a second proprietary machine learning algorithm that operates on the recorded data to derive scratching-related metrics [24]. In this study, the RF sensor was installed on the bedroom wall 1.15 m from the floor and not more than 5 m from the participant’s bed. Participants in this study agreed to be the only person sleeping in the room with the RF sensor and agreed to keep pets out of the bedroom during the 28-day observation period.

The GENEActiv Original (ActivInsights) wristband was configured to collect accelerometer data at 20 Hz. Each participant received 2 wristbands on study day 1 and wore both for 14 days. Each participant received a new pair of wristbands around day 14 and shipped the first set back to study sites for data collection. Participants were instructed to wear 1 wristband on the nondominant wrist continuously and the other on the dominant wrist during sleep hours.

Philips Respironics Rapid Actigraphy Data Analyzer (RADA) algorithms were applied to the raw GENEActiv data to generate sleep and scratch metrics. The RADA scratch algorithm is based on a previously published study [12]. Scratches output by the RADA algorithm were characterized as left wrist, right wrist, or both wrists. The analysis of the data was wrist agnostic, and all scratching events were used for analysis.

TSO was defined for the Reference by the human-labeled TSO start and TSO end times from video. TSO was calculated separately for each DHT to emulate how the DHT would be used in a future study (ie, without information from the videos). TSO for the Philips DHT was selected as the major rest period on each night as determined by the RADA algorithms. TSO for the Emerald DHT was calculated as the merged sleep period where multiple sleep periods on the same night with <30 minutes gap were merged. Any TSO with a duration of <3.5 hours was excluded from the analysis as they were not considered representative of a nightly sleep period. A short, estimated TSO can occur when the true TSO is short, or the estimated TSO is capturing only a portion of the true TSO. The cutoff was set based on the observed data to minimize filtering out any true TSO. The intent of the filter was to automatically detect and remove cases when only a portion of the true TSO was detected (eg, the video was switched on only for a portion of the true TSO). Seven out of 232 Reference nights (3%) were excluded from all analyses due to Reference TSO of <3.5 hours. One additional night was excluded when comparing Reference to Philips, and 3 additional nights were excluded when comparing Reference to Emerald due to the respective DHT-derived TSO of <3.5 hours. These 11 nights impacted by the TSO filter on nights with Reference data were examined, and in 8 cases, the short TSO is a subset of a longer TSO detected by another method; in 2 cases, the short TSO does not overlap at all with the other methods, and in 1 case, all methods of TSO determination were short. Over all nights recorded by the DHTs, 23 out of 728 nights (3%) were removed for short TSO from Emerald, and 11 out of 635 (2%) nights were removed for short TSO from Philips.

The 2 primary scratch outcomes analyzed were the scratch duration and the scratch frequency. Before any downstream analysis, individual scratch events were merged and filtered. First, all scratch events with a <3-second gap were merged into a single scratching bout. Next, only scratch events of at least 2 seconds in duration (69% of all Reference scratch events) were retained. The same merging and filtering of scratch events were applied to both DHTs and the Reference. A minimum duration of 2 seconds was chosen both because the clinical relevance of very short scratch events is unclear and to allow for fair comparisons since 2 seconds was the minimum duration of scratch predicted by the Philips algorithm [12]. In all analyses, scratch duration and frequency were normalized by the duration of the TSO window so that changes in sleep time would not unduly impact the outcomes. The scratch duration is presented in units of seconds of scratch per hour of TSO, while the scratch frequency is presented in units of number of scratch bouts per hour of TSO. Both scratch duration and scratch frequency have right-skewed distributions and are therefore modeled (in intraclass correlation coefficient [ICC] analysis) on a logarithmic scale.

Atopic Dermatitis Itch Scale was completed in the AD cohort twice daily (morning and evening) to assess both nighttime and daytime itch. The characterization of scratch by itch level was performed using the morning Atopic Dermatitis Itch Scale administration, which asked about worst eczema-related itching last night on a 0‐10 numeric rating scale. PROs were administered electronically through a mobile phone app. In cases of technical issues with the phones, paper PROs were completed, and responses were entered by the site staff. Only responses with time stamps (electronic or handwritten) between 4 AM and 12 PM were considered valid for the morning administration. In rare cases where multiple responses were recorded on the same day, all responses on that day were excluded from analysis.

This clinical study was performed at 6 commercial dermatology clinic sites across the United States between 2022 and 2023. Institutional review board approval was received for the study (WCG institutional review board protocol no 20213992), and all study participants provided informed consent. This study adhered to ethical standards by obtaining informed consent from all participants and ensuring that they received fair compensation for their involvement. Participant privacy and confidentiality were protected as outlined in the informed consent process, and any participant-level data presented in this manuscript are pseudonymized to preserve participant privacy.

Thirty-one participants enrolled in the study, including 15 with moderate AD, 11 with mild AD, and 5 HVs. Of the 31 participants, 2 participants were excluded for not meeting the inclusion or exclusion criteria (one based on disease severity and another based on change in therapy during the screening period). One additional participant in the moderate AD group was excluded from the analysis due to capture of poor-quality videos that failed spot checks and could not be manually labeled. A schematic overview of the study design is shown in Figure 1.

A summary of demographics and disease severity characteristics at baseline for the 28 participants included in the analysis is shown in Table 1. Participants ranged in age from 19 to 65 (median 37.5) years. Participants were categorized as having mild AD or moderate AD based on the vIGA-AD. Mean Eczema Area and Severity Index scores for the moderate AD and mild AD groups were 17.54 and 4.63, respectively. Mean SCORing Atopic Dermatitis (SCORAD) values for the moderate AD and mild AD groups were approximately 50 and 36, respectively. Peak nocturnal itch reported on a 0‐10 NRS was 0 at screening for all HVs and at least 2 for all patients with AD, with a similar median peak nocturnal itch score for the groups with mild and moderate AD (mild AD: 6.0, moderate AD: 5.5).

To define a nocturnal scratch outcome, it was first necessary to determine the time window during which scratching was to be measured. Following the terminology used in previous research, we call this window of intended sleep the TSO [10]. The TSO windows for the Reference were manually annotated and defined to start when the participant got into the bed intending to sleep and defined to end when the participant got out of the bed intending to stay awake.

Nightly scratch measures (normalized scratch duration and normalized scratch frequency) as defined in the Reference were characterized by disease severity and patient-reported itch in Figure 2. The median frequency and duration of nighttime scratching increase with AD disease severity from HVs (2.4 bouts per hour, 13.1 seconds per hour) to mild AD (3.51 bouts per hour, 17.4 seconds per hour) to moderate AD (4.64 bouts per hour, 30.0 seconds per hour); however, the ranges are highly overlapping between all 3 cohorts (Figures 2A and B). Scratching was also characterized by SCORAD severity since this is a frequently used scoring system in AD. Similarly to cohort, there is a trend of increasing median nocturnal scratch frequency and duration with increasing severity of SCORAD but highly overlapping ranges (Figures 2C and D).

Finally, we investigated how the reference nocturnal scratch frequency and duration compared on nights when participants reported no itch, mild itch, moderate itch, or severe itch (Figures 2E and F). In this cross-sectional analysis, when AD study participants reported in the morning that the previous night had no itch, the scratching was dramatically lower (but not zero) than when any itch was reported. Median scratch frequency on nights with no itch was 2.37 bouts per hour versus 4.5 bouts per hour with mild itch and 4.64 bouts per hour with moderate itch. The trend did not hold for severe itch, which had the lowest number of nights and fell in between the levels of no itch and mild itch with a median of 3.68 bouts per hour. Similar trends were observed for nocturnal scratch duration with 10.2 seconds per hour in no itch, 26.7 seconds per hour in mild itch, 26.5 seconds per hour in moderate itch, and 17.8 seconds per hour in severe itch.

After characterizing the scratching behavior with the human-labeled reference, we began to assess the agreement of the DHT-derived measures with the Reference. First, we assessed the level of agreement between TSO derived from Philips versus Reference and from Emerald versus Reference with a Bland-Altman analysis (Figures 3A and B). The average bias in TSO in the home environment was less than half an hour for both DHTs with bias (lower and upper 95% limits of agreement) estimates of 0.42 hours (−2.18 hours and 3.02 hours) for Philips and 0.44 hours (−1.85 hours and 2.73 hours) for Emerald.

Next, we assessed the level of agreement in scratch classification between each DHT and the Reference for all 10-second windows. The time period considered for the evaluation was the intersection of the TSO between the Reference and the corresponding DHT to avoid ambiguous cases where scratch labels were missing. All time windows within the TSO intersections were considered, and no balancing of the scratch and nonscratch windows was performed. In the moderate AD group, an F₁-score of 0.64 was achieved for Emerald and 0.43 for Philips (Table 2). Overall, the performance of the Emerald DHT, as measured by F₁-score, was noticeably higher than that of the Philips DHT in each cohort. The difference in F₁-scores was mainly due to the lower precision (0.29‐0.36) of the Philips algorithm, indicating that many false-positive scratching events were identified.

To better understand the behaviors leading to false-positive scratch prediction with the Philips algorithm, we sampled 2 nights from each study participant and watched the videos during the false-positive event to identify which nonscratch movements were contributing to false-positive scratch predictions. We found that the main nonscratch movements incorrectly classified as scratch were body repositioning within the bed, mobile phone usage, and movement associated with entering or exiting the bed.

A more detailed listing of performance metrics by participant is shown in Multimedia Appendix 1. A 10-second window size was chosen for this analysis of within-night performance to mitigate any misalignment of scratch events resulting from incomplete clock synchronization between DHTs and video. A sensitivity analysis showing the performance metrics with 1-second windows is shown in Multimedia Appendix 2. Overall, the trends observed with 10-second windows are consistent with 1-second windows, but the performance metrics are generally lower.

Because of the wide range of participant-level F₁-scores observed (ranging from 0.13 to 0.85), we investigated whether any participant characteristics were associated with F₁-scores (Multimedia Appendix 3). We found that the strongest factor associated with F₁-score was the scratch prevalence that varied between 0.5% and 4.5% when pooling all epochs at the participant level. Scratch prevalence was strongly positively correlated with F₁-score for both Philips (r=0.82; P<.001) and Emerald (r=0.59; P=.001) DHTs. Ethnicity was the next most associated variable with F₁-score in this dataset, with non-Hispanic or non-Latino participants having an average F₁-score 0.17 lower than Hispanic or Latino participants (Philips: P=.004, Emerald: P=.01). However, upon closer inspection of the variables’ correlation structure, it was noted that ethnicity is associated with prevalence in this dataset, which is likely driving the observed association. We also identified a significant correlation between the participants’ geometric mean power of scratch events (in G) and the F₁-score for the Philips DHT (r=0.43; P=.02). Additionally, a negative correlation was observed between participant age and F₁-score (Philips: r=−0.39, P=.04; Emerald: r=−0.41, P=.03). Participant age was found to be negatively correlated with the power of scratching events, which may be partly mediating the association with age.

The level of agreement in nocturnal scratch measures between each DHT and Reference on a nightly level is shown in Figure 4. The points were largely distributed along the identity line, indicating that the predicted nocturnal scratch frequency and nocturnal scratch duration were similar to the reference. Philips scratch predictions had a higher false-positive rate, resulting in overestimation of both nocturnal scratch frequency and duration as observed in Figures 4A and 4C. Emerald scratch predictions fall more evenly about the identity line, indicating lower bias as observed in Figures 4B and 4D.

The primary hypothesis of this study was that the nocturnal scratch metrics estimated from DHTs would be comparable with nocturnal scratch metrics assessed by human-labeled IR video. However, to our knowledge, the level of human agreement expected between human raters for this task had not been characterized in the literature and thus was unknown to us at the start of the study. Therefore, we used a rigorous human labeling approach where every night of video was first split into motion and nonmotion periods, and then every motion period was independently labeled by at least 2 human raters for scratching. To quantify the level of agreement between methods, the ICC was used as a measure of interrater reliability.

Interrater reliability estimates comparing 2 human raters fell in the “good” to “excellent” range (95% CI lower bound >0.75) for HVs and the “moderate” to “good” range (95% CI lower bound 0.5‐0.9) for patients with AD [28] (Figure 5). In all cohorts, the 95% CIs for DHT versus human rater and human rater versus human rater ICCs overlapped, indicating noninferiority. The noninferior ICC estimates for both Philips and Emerald indicate that DHT estimates of normalized scratch duration and frequency are comparable with human raters.

We conducted an analytical validation study of both a wearable sensor and a touchless sensor, and corresponding algorithms, for the ability to correctly identify nocturnal scratching events in adults with AD, while participants were sleeping at home. We found that the intended sleep window (TSO) can be reliably estimated with both DHTs. The TSO determination for both the Philips DHT and the Emerald DHT was in the range previously reported in a sleep clinic (<30 minutes mean bias) [10]. The sensitivity of scratch classification performance was higher with the Philips DHT, whereas the precision was higher with the Emerald DHT. Overall, the F₁-score was higher with the Emerald DHT (ranging from 0.51 to 0.68) than with the Philips DHT (ranging from 0.47 to 0.56). The Philips algorithm was found to be oversensitive, falsely identifying movements as scratch events during nonscratch participant motion. Participant-level F₁-scores varied widely from 0.13 to 0.85, and this was highly associated with the scratch prevalence—participants having a higher scratch prevalence generally achieved a higher F₁-score for both Emerald and Philips DHT. Other weaker associations identified with the participant-level F₁-scores (accelerometer power, ethnicity, and age) should be interpreted cautiously due to the limited sample size.

While the F₁-scores in this study were lower than those previously reported for these DHTs [12,24], this was expected due to several factors. First, the dataset used was previously unseen by the algorithms tested, whereas previous studies tested an algorithm that was developed on the same dataset. Second, the study execution was made as “real-world” as possible with all data collected in the participants’ homes. Additionally, the comparison with reference to generate F₁-scores was performed on all windows, without first balancing scratch and nonscratch events, which was shown to have a large impact on F₁-score [11]. Finally, we have shown that F₁-score is influenced by scratch prevalence, which makes it difficult to compare results across studies without accounting for differences in the scratch prevalence between study populations.

Importantly, we quantified the interrater agreement for the nightly scratch summary measures and found that the performance of both DHTs and human raters was noninferior to how 2 independent human raters performed when compared with each other. The mean level of human interrater agreement for nocturnal scratch duration and nocturnal scratch frequency was approximately 0.75 for both mild and moderate AD groups. This finding provides a ceiling for the maximal level of agreement expected between DHT and human raters. Our results show that current DHTs are close to, but do not reach, this ceiling of agreement between 2 human raters. Future work should investigate whether improved nocturnal scratching algorithms trained on additional datasets could reach the level of agreement of 2 human raters. Algorithm development should seek to minimize false positives from nonscratch movements.

When characterizing human-labeled scratch measures, we noted that nocturnal scratching was also observed in HVs in this study. While previous studies have generally found little to no scratching among HVs as assessed by video monitoring [9,12,17], recent results from other studies of nocturnal scratching hint that even HVs may scratch during the night [29]. We also found that nocturnal scratch duration and frequency were similar between mild AD and moderate AD cohorts in this study. Previous work has also shown overlapping ranges between mild and moderate AD without finding a significant relationship in the total scratching time as a percentage of the total recording [9]. Converting the metric used in Figure 1 of Ebata et al [9] into normalized scratch duration as we define it would lead to a median estimate of normalized scratching duration of <36 seconds per hour in mild AD and 216 seconds per hour in moderate AD. Therefore, it is possible that our study tended to recruit people who scratch less than a typical patient with moderate AD. Notably, the similarity in mild AD and moderate AD scratch was also mirrored in the self-reported itch NRS at screening, which had a similar range and median value in both cohorts (Table 1). Future work may further elucidate whether there are differences in the characteristics (eg, intensity, distribution by sleep stage, and timing) of scratching between patients with AD of varying severities or between patients with AD as a whole and, otherwise, healthy individuals reporting no pruritus.

This study has a few limitations to be considered. The number of participants analyzed (n=28) was relatively small since it was chosen to be powered primarily for the analysis of ICC estimation in patients with moderate AD. Still, the sample size is comparable with other studies of nocturnal scratching [10-12], and this study was unique in that rather than 1 or 2 nights of collected IR video, participants collected, on average, 8 nights of IR video over the 4-week observation period. The study also had a limited sample size to investigate the performance of the scratch algorithms in subgroups (eg, by age or race). While associations with F₁-scores were performed, additional data are necessary to reach a conclusion about scratch performance in any subgroups. Future work should test these tools in a more diverse population to ensure that performance is acceptable in a representative sample of the targeted population. This study has another limitation tied to the collection of video data in the home. An intentional choice was made to give the study participants full control over the IR video recording, allowing them to turn it on and off at will. This could result in some degree of bias in the determination of the reference TSO window because it is possible that participants recorded video only for part of the true TSO window. Anecdotal evidence from DHT-derived sleep windows suggests that on some nights, the camera may have been turned off before the end of the sleep period, leading to a bias in the Reference TSO duration. To mitigate the impact of this bias, short estimated TSO windows (<3.5 hours) were excluded from the analysis. Overall, the net benefits of performing the study in the home (less participant burden and a natural sleeping environment) outweigh any potential drawbacks in capturing the TSO.

While our intent was to enroll patients with a broad range of disease severity, no patients with AD with severe disease according to vIGA-AD enrolled in our study. This is due, in part, to the requirements for having a stable regimen of systemic treatment for at least 4 weeks prior to study screening and no changes in treatment during the 4-week study, which made it difficult to recruit participants with severe disease. Notably, when classifying the study participants according to SCORAD, there are 9 participants considered severe (total score of >50). Future analytical validation studies may consider modifying the inclusion criteria to recruit participants with a broader range of AD severity.

Results from the Emerald DHT are promising; however, data in this study were collected with strict lifestyle restrictions. Namely, participants were required to sleep without any other people or pets in the bedroom due to the requirements of current algorithms for detecting scratching. Future work should focus on algorithmic solutions to the problem of detecting scratching in a single individual while not sleeping alone to enable scaling this technology for more widespread clinical trial use.

Our results support the analytical validity of both DHTs tested for continuous measurement of nocturnal scratching in the home environment. The most supportive finding is the noninferior performance of both the Emerald DHT and Philips DHT compared with human labelers. Opportunities remain for improving the performance of the DHTs tested, especially in the precision of wrist-worn accelerometer scratch detection, to reach human-level performance. Nevertheless, ultimately the use of measuring nocturnal scratching as an efficacy end point for clinical trials is dependent not on perfect agreement with the reference but rather on detecting clinically meaningful changes in scratching behavior in patients. Therefore, current tools may be fit-for-purpose to generate clinical trial end points depending on their intended use and the magnitude of the treatment effect on specific nocturnal scratch end points. Practitioners who would like to use these tools may consider collecting additional data on diverse samples to better understand scratching behavior in the context of natural history studies or interventional studies where treatment effects on scratching behavior can be assessed.