To train an ML model to recognize pain-related behaviors, we conducted a secondary data analysis of historic longitudinal sensor-based data and semistructured, holistic nursing assessment interviews containing descriptions of pain. Historic records afforded a convenient and purposive sample [47] of 17 independent, community-dwelling smart home participants aged ≥55 years with ≥2 chronic conditions, living alone, without pets. Of these, 11 participants met the inclusion criterion of having at least one recorded pain event during the longitudinal study. For each participant, approximately 60,000 sensor events per month were available, totaling 720,000 per year. All data from passive infrared and door use sensors throughout 1-2 years’ monitoring were included for training the model. Based on prior work developing techniques to train ML models using small sample sizes [35,36], we determined this amount of data exceeds ML training requirements for models using longitudinal data and that this amount of data would likely allow the model to capture aspects of pain. The health records of each participant contained data from approximately 50-100 nursing assessments (1-2 years of weekly visits). We extracted pain-related information from the nursing record, defining a “pain event” as any report of pain associated with a report of related behavior changes. We also included 1 week of sensor data surrounding the pain event for conducting qualitative analysis of the sensor data as well as several weeks of baseline activity data — these were weeks where the nurse noted no health changes had occurred. “Normal” weeks were compared to weeks containing health events to help illuminate the event in the sensor data. We included data showing short-term visitors, which is exhibited in the data as multiple sensors turning ON nearly simultaneously (within <0.01 seconds of each other and <3 meters apart).

We excluded personal health data unrelated to pain. When conducting qualitative analysis of the sensor-based data, we excluded sensor data outside of the week surrounding the pain event except the weeks chosen to represent baseline normal routines. We also excluded data showing extended stay visitors (ie, stays across multiple days and nights). More information on the nursing team’s analytic methods, including data exclusion processes, is available in the literature [45,46]. For training the ML model, we excluded data from light, temperature, and humidity sensors.

The same 11 smart homes were included for both qualitative and quantitative analyses. Only 2 sensor types (passive infrared motion, door use) were processed for the quantitative portion of the study. Pain events varied in length from 1 to 15 days (mean 7.85 days); for each pain event, an equivalent set of relatively pain-free data was included for each participant’s pain event dataset. All collected smart home sensor events had already been automatically labeled with corresponding normal daily activities using our activity recognition algorithm that had previously been trained on 30 homes (not analyzed as part of this study). In each of these prior homes, a non-clinical annotator provided ground truth labels for 2 months of sensor data. From activity-labeled sensor data for the 11 homes included in this study, 550 behavior markers were extracted. A set of activities that occur with sufficient regularity in the homes (at least once a day on average) was selected to inform the ML models. The activity categories were Bathe, Bed-Toilet Transition, Cook, Eat, Enter Home, Leave Home, Personal Hygiene, Relax, Sleep, Take Medicine, Wash Dishes, Work, and Other Activity. The behavior markers corresponded to statistical measures of mean, median, standard deviation, maximum, minimum, zero crossings, mean crossings, interquartile range, skewness, kurtosis, and signal energy. Each measure was applied to time series data indicating hourly distributions for overall activity level (measured as the number of motion sensor events), hourly distributions over home locations, and hourly distributions over activity classes. Additionally, behavior markers were computed that indicate daily schedule regularity and 24-hour circadian rhythm values. The regularity value calculates the normalized difference (in location and in activity) between the same hours across all of the days in the sample. The circadian rhythm strength was calculated using a periodogram, which is an estimate of the spectral density of a signal (in this case, activity level). The periodogram identifies the strength of the frequencies that explain variations in time series data. We quantified circadian rhythm as the normalized strength of the frequency that corresponds to 24 hours.

We designed several ML approaches for detecting pain events from sensor-derived behavior markers. First, we employed a random forest binary classifier with 150 trees to predict whether a day was part of a pain event (positive class) or pain-free (negative class). Second, we trained a regression tree on the data to determine correlation between a model of behavior markers and pain events. For comparison with our clinician-trained ML approaches, we employed an isolation forest (iForest) anomaly detection algorithm with 100 estimators to detect pain days. Unlike the random forest and regression trees, iForest did not use any clinician guidance in detecting anomalies that may indicate days for which the participant was experiencing pain. Finally, we trained a decision tree algorithm based on the positive and negative instances to determine which behavior markers provided the greatest distinction between pain days and those that were pain-free. We did not train the algorithm to differentiate between pain subclasses due to the small number of events in each group (acute, flare); however, our clinical team’s early work determining these subgroups using sensor-based data positions us to explore subgroup comparisons in the future with larger sample sizes.

We found 13 pain-related behaviors: no exit home, decreased time out of home, visitors, sleep location, time of sleep, length of sleep, night time sleep interruption, change in total sleep hours (increase or decrease), sleep quality (body movement during sleep — tossing and turning), grooming, walking speed, change in walking pattern (short bursts, long rests), overall activity in 24 hours. From the sensor activity patterns, 6 themes representing all 13 behaviors emerged (Figure 2): Sleep, Walking, Grooming, Time Spent Out of Home, Visitors, Overall Activity Level. Themes represented pain-related behaviors. Table 2 shows major and minor theme characteristics, the types of pain reported by participants (noted by nurses), and the type of sensor, location, and sensor combinations informing qualitative interpretations. Themes generally aligned with the 13 activity categories that the ML algorithm recognized, and each of the 6 themes was represented in the subset of 30 activities (from the prior homes) included for training PASH. For example, the qualitative theme of “Time Spent Out of Home” was represented in PASH’s ML model as “Enter Home” and “Leave Home.”

Sleep was the leading pain-related behavioral theme, accounting for 6 of the 13 described behaviors. Of the 11 participants, 8 (having 22 of 27 pain events [81%]) reported sleep changes that were observed in the sensor data. Leading observable sleep behaviors included location (more time in a recliner), timing (bedtime, wake time, napping), hours of sleep at night, and sleep quality. Six of 8 acute pain events (short-term pain not associated with underlying pain) affected sleep, resulting in 2 people spending more time in their recliner chair, 1 person experiencing decreased overall sleep (day and night) and an inability to sleep in bed, and 3 people with restless sleep (decreased sleep quality). Twelve of 19 flare pain events (exacerbation of underlying pain) affected sleep, observed as more time in bed during the day, earlier bedtimes, more time spent in recliners, and intermittent changes in sleep location across several months. Two walking characteristics were observed: Walking speed slowed, and the number or length of rest breaks increased. We observed this in the monthly Timed-Up-and-Go tests and by calculating the difference in the time it took to move between specific sensors (bed and toilet or recliner and toilet).

Grooming activities were observed by reviewing bathroom sink, bedroom, and bedroom closet sensor groupings. A lack of grooming recorded in the nursing record as “stayed in bed 3 days on Dilaudid” was seen in the sensor data as a decrease in overall time spent in the bathroom in the mornings and evenings. A lack of grooming appeared to be associated with pain intensity. For example, the nursing record reported that 1 participant said, “I just haven’t felt like leaving the house or even showering. I mostly lay in bed all day.” Other pain-related behavior modifications regarded participants’ overall activity level (in 24 hours) and time spent out of the home. All acute pain events resulted in decreased overall activity: 2 reduced their out-of-home activities, and 2 had more visitors. Pain-event interpretations, event start and stop times, ground truth annotations, and themes were communicated to the computer science team to support the training of PASH.

The random forest classifier that was used to distinguish pain from pain-free days yielded a 3-fold cross-validation classification accuracy of 0.70, sensitivity of 0.72, specificity of 0.69, area under the receiver operating characteristic curve value of 0.756, and area under the precision-recall curve of 0.777. We use the term pain-free to distinguish between pain events and routine days; participants may not have actually been pain-free. To check the impact of clinicians’ ground truth annotations and the expert-guided approach, we used standard anomaly detection techniques to determine classification accuracy without expert guidance. We used iForest to determine periods of time that were generally considered anomalous without clinician guidance. Using the detected anomalies (no clinician guidance) as indicators of pain events yielded a predictive accuracy of 0.16, a difference from the random forest (with clinician guidance) that was statistically significant (P<.001). Using the regression tree, a moderate correlation (r=0.415) was found between the behavior models and an indication of pain on the corresponding days.

The decision tree classifier created a greedy ordering of behavior features that best distinguish pain from pain-free days using information gain as the ordering criterion. These results highlighted 3 features that provided a high level of differentiation between the 2 classes: normalized overall activity level (lower for pain days), time spent in bed-toilet transition activities (higher for pain days), and time spent in a favorite chair (higher for pain days). Behaviors such as overall activity level, walking speed (or time spent in bed-toilet transition), and time spent in a favorite chair (including sleeping there) were supported by both qualitative and quantitative analytic findings and may prove important to understanding pain experiences.

Our findings show preliminarily that ambient sensor-based data can be used to train an ML model to recognize pain-associated behaviors. These findings align with previous studies that indicate ML algorithms are capable of detecting behaviors that indicate a clinically relevant change in health status [21,23,32,33,49,50]. Unlike these previous studies, which primarily focused on associating behaviors with cognitive and functional health, we focused on behaviors exhibited by persons experiencing pain. While there is significant overlap in pain-related behaviors and other known health-related behaviors that machines can recognize, we uniquely discovered that one behavior not typically included in pain interference scales (yet recognized as physical activity sequences by ML algorithms) point to the existence of increased pain: decreased grooming. Importantly, almost all the qualitative themes (emerging from sensor-based data) align with behaviors that are already well-established with validated pain interference scales [26]. The exception is “Grooming.” Grooming is not specifically identified as a factor in some of the most commonly used pain interference measurement scales (eg, the Brief Pain Inventory); yet, it could become an important target to assess pain populations when using sensor monitoring. This is an example of how smart home monitoring can generate new evidence-based information to support pain management.

Given that the standard anomaly detection techniques used to determine classification accuracy without expert guidance yielded a 16% classification accuracy compared with the expert-guided approach (predictive accuracy 70%), our findings support the ideas that (1) clinicians, such as nurses with frontline pain management experience, add value to the efficacy of the ML model and (2) PASH offers possibilities as a clinical tool for identifying pain-related behaviors. Though the model demonstrated a pragmatically low predictive accuracy (70%) for clinical applications and we did not ask it to discriminate between pain subclasses (acute, flare), it performed quite well given the small participant sample size (N=11) and small number of captured pain events (N=27). PASH’s accuracy could improve given a larger participant sample size and greater number of pain-related training events interpreted by clinicians.

The question of whether all pain leads to the same pain-related behaviors could be raised. Our smart home approach to pain management cannot yet determine the type of pain. Further, this approach to pain assessment cannot determine the source, severity, or location of a person’s pain (eg, abdominal versus head). However, given that ML models are capable of recognizing clinically relevant behavior changes [21,29], it is reasonable to consider that ML models could alert when anomalous pain-related behaviors occur regardless of which behavior the model chooses to prioritize or the pain characteristics. Based on our preliminary findings, it is also plausible that, with larger samples, ML models could be trained to alert on unrealized pain-related behaviors. Such alerts, based on naturalistic real-time data of persons experiencing and attempting to manage pain, would be of significant value to clinicians seeking to perform early interventions using minimal pharmacologics. ML affords this opportunity while also individualizing pain context and offering the potential to discover new perspectives on pain. Randomized controlled trials supporting current interventions do not account for individual differences in pain experiences. Rather, they focus on average pain responses, leaving persons who are outliers without optimal care. Learning how individuals with pain uniquely express their pain moment-by-moment could lead to novel understandings of pain and afford the opportunity to provide effective, precise interventions. To achieve such precision, PASH would benefit from adding ecological momentary assessment, an in-the-moment data capture technique for naturalistic settings [51] now regarded as the most accurate method for capturing real-world pain [52,53].

PASH offers clinicians a more objective, data-driven way of knowing about pain. PASH could be of benefit to nonverbal or cognitively impaired individuals [54]. Developing consistent, reliable, objective pain measurements; detecting patterns in behaviors and activities that exacerbate or relieve pain; and accurately capturing responses to medications and other pain treatments are potential scientific discoveries that could be realized using an AI system like PASH. In addition, PASH could add new sensor-based evidence of biopsychosocial pain components and facilitate the combining of traditional and new data to augment and support clinical assessments (with reduced bias) and clinical decision making. We recognize that it is impossible to know when an acute pain condition may transition into chronic pain; yet, part of our enthusiasm about this work is the future possibility of detecting minor and important changes in this regard using sensor data. This would be an important contribution to pain science.

Privacy and data security have been identified as primary concerns for older adults considering the use of a smart home. As ubiquity of data collection expands to the home environment and is integrated into the delivery of health care, considerations for data security are needed so risks for data breaches are mitigated and identities of vulnerable persons, such as persons in pain, cannot be stolen or easily reconstructed.

Clinicians and software developers can potentially introduce bias into ML models [5,14,55]. Assumptions and generalized perspectives regarding subpopulations and disease progression tendencies incorporated in a clinician’s belief structure (overtly or inadvertently) over time could impact the reporting of ground truth. Likewise, software developers’ assumptions (recognized or unrecognized) have potential to impact design approaches and perceived end-user wants and needs. To avoid algorithm bias, clinicians and computer scientists as well as study participants need to be a diverse group of humans [56]. To accomplish this, intentionality toward diversity in all aspects of the design loop is required.

This project was limited by the larger study’s design and the purposive sample size (number of participants, number and type of pain events; both convenience samples). Using historic datasets not specifically collected for discovering pain-related behaviors as well as the small numbers of participants and pain events limit generalizability and make the model susceptible to overfit. Reliance on participants’ weekly recall of health changes, including pain, limits accuracy of sensor data interpretations. Not all qualitative and quantitative components of this multiple methods, secondary analysis aligned. We do not make an attempt to differentiate between depression and pain-related behaviors in this study and acknowledge the synergy between these conditions. Significant time and effort are required for the expert-guided approach to ML, which potentially limits scalability. Larger and more diverse samples, a longitudinal design, and use of an ecological momentary assessment for data collection are needed. Using ML to identify and discriminate between pain phenotypes would be of benefit to providers and patients. PASH should be tested in a prospective study to identify true and false positive ratios. Additionally, PASH needs to be trained to accommodate a multiperson household.

Innovative monitoring and treatment options are needed to support persons experiencing chronic pain, their caregivers, and the health care professionals working alongside them to improve their quality of life and health outcomes. Our findings suggest that smart homes using AI monitoring tools are well-positioned to become pragmatically useful at detecting clinically relevant pain-related behaviors or relevant changes in those behaviors. Using smart homes to provide automated pain assessment and intervention could alleviate some of the pressure on patients and clinicians working in the pain management health care arena. Concrete, objective data demonstrating how people with pain are affected and how they self-manage painful conditions at home could be used to deepen understandings and innovate solutions. Leveraging such technologies for health care delivery should be done intentionally, and clinicians should participate in technology development studies to interpret data, provide meaningful context, and illuminate meaningful use possibilities in all phases of development.