Several health risks such as cardiovascular disease and immune deficiencies, which can diminish the quality of life and shorten life expectancy [1], are linked to repetitive daily acute stress (ie, short-term response to a perceived threat or challenge [2,3]). Stress can also have a profound impact on cognitive and emotional well-being. Advances in wearable technology enable affective and cognitive state measurements. Still, wearable devices can suffer from high attrition and low adoption, in general, and getting stable stress measurements “in the wild” (eg, heart rate variability [HRV] and electrodermal activity [EDA]) remains challenging.

Previous lab studies have shown that data from interactions with everyday devices, such as computer mouse movements, keyboard pressure, smartphone touch, and key swipes, or even the steering wheel of a car, can be used in an unobtrusive and scalable way to detect the presence of acute stress [4-8]. To the best of our knowledge, this paper presents the first work showing the feasibility of detecting acute stress from interactions with laptop computer trackpads.

There are several billion personal computing devices (PCDs) deployed in the world, and this number increases by around 300 million every year [9]. Laptops, or notebooks, represent the largest (42%) and the fastest-growing segment [9] among PCDs. Most laptop users prefer the trackpad over an external mouse, for improved mobility or due to space limitations. Investigating biomechanics and motor control of finger dynamics during trackpad use can lead to reliable detection of acute stress, especially for long-term unobtrusive continuous monitoring. We propose two research questions (RQs):

We conducted a within-subjects study (N=18) counterbalancing relaxed and stressed conditions. Subjects performed 40 trials each of three canonical computer tasks using a trackpad, namely, clicking, steering, and drag and drop (see Table 1 and Figure 1), similar to what Sun et al did with a computer mouse [4]. We focused our analyses on metrics derived from finger dynamics. Our results confirm a significantly higher mean and SD of the contact area under the finger for the tasks performed under stress compared to relaxed conditions, even among the initial 10% of the data (ie, 4 out of 40 clicks) from clicking trials. To the best of our knowledge, this is the first systematic study that links acute stress to finger strokes on the trackpad of a laptop device.

We recruited 22 subjects (10 males [45%] and 12 females [55%]), with ages ranging from 19 to 68 (mean 37, SD 16), not screened for preferences in using a computer mouse or trackpad. We eliminated 4 subjects for whom our stressor did not have any effect on SRS, leaving a total of 18 subjects (8 males [44%] and 10 females [56%]). Of the remaining 18 subjects, 11 (61%) reported using a computer mouse and 9 (50%) reported using a laptop trackpad daily. The mean self-reported daily usage duration of the laptop trackpad (mean 2.6, SD 3.1 hours) was significantly lower (t₁₇=2.1, P=.03) than that of the computer mouse (mean 5.2, SD 3.1 hours). Subjects provided written informed consent before participation and were given US $20 gift cards for their participation. The study was approved by the Institutional Review Board of Stanford University.

The experiment was performed in an office setting without any distractions. We used a 15-inch 2015 MacBook Pro (Apple Inc) (see Figure 3) equipped with a 140×70-mm Force Touch trackpad with sensitivity preset to a default value of 1.0. An overhead camera captured hand movement, and our logging software captured trackpad and cursor activity. The logger was implemented in C and Swift (Apple Inc) using Apple’s MultiTouchSupport framework. It recorded coordinates rounded to the nearest hundredth of a millimeter with an average sampling rate of 8.17 milliseconds (SD 3.75), multiple finger touches marked with ID numbers, contact shape under the finger (ie, major and minor axes of the ellipse), interaction type (ie, touching, dragging, etc), and pressed state (ie, active or inactive).

The stimuli consisted of Relaxation and Stressor phases replicated from prior studies [4,5]. During the Relaxation phase, we instructed subjects to breathe deeply while viewing a soothing video, which is recommended over doing nothing [31]. The acute Stressor phase consisted of an arithmetic cognitive stressor combined with social evaluation (ie, the presence of an experimenter continuously observing and correcting if the subject made a mistake), derived from the Trier Social Stress Test [32], and enhanced with a biased financial stimulus. The test required subjects to perform a series of subtractions out loud (eg, subtract 13 from 2017, and so forth) under the scrutiny of the experimenter. If the subject made a mistake, or if he or she took more than four seconds to respond, the experimenter asked the subject to start again from 2017. The subject was offered an additional US $1 for every 10 consecutive arithmetic operations performed correctly, but only US $1 was to be deducted for each mistake. All subjects made mistakes that made them “lose” money, but after debriefing, they all received full compensation. The financial part of the stimulus based on performance could have been more frustrating than stressful for some subjects. However, it was added, as was done in our previous study [4], since frustration can further exacerbate acute stress.

All subjects performed three types of tasks—click, steer, and drag and drop (see Figure 1)—as in our previous study using the computer mouse [4]. Five different distance (D) and four different width (W) configurations were used for each of the three task types, resulting in a total of 20 configurations per task (see Table 1) [4]. Users performed two trials per configuration within each task type (see Figure 4). Tasks of the same type were grouped to prevent performance differences due to subjects needing to adjust to different task types. In total, each experimental condition, tStressed (stressed task) and tRelaxed (relaxed task), had 120 trials (3 task types × 40 trials).

First, we evaluated a Mixed Task omnibus model independent of specific task types—click, drag and drop, and steer—and task configurations (W × D). Values for each task type were obtained by averaging across all configurations and repetitions (ie, 40 trials) for each of the tStressed and tRelaxed phases for each subject. Then, a single value for each tStressed and tRelaxed phase was obtained after averaging across all three task types for each subject. Variables with significant differences across the tStressed and tRelaxed phases in the Mixed Task model were further examined using Task-Specific models. To avoid inflation, we applied Bonferroni correction (0.05/3 = 0.017) for Task-Specific models. Finally, for the type of task with the most significant difference, we applied a sensitivity analysis comparing the initial 10% (ie, 4 trials out of 40) of data across the tStressed and tRelaxed phases. We also carried out post hoc power (β) analyses on all finger dynamics measures that were found to be significant. Additionally, to check if the order of stress-relax condition has any effect, we investigated the Order (ie, A vs B; see Figure 5) × Condition (ie, stress vs relax) interaction effect using analyses of variance on all finger dynamics measures that were found to be significant.

Enabling precision health and effective stress management regimes are dependent on the evolution of ubiquitous real-time stress sensors and algorithms. An ideal scenario would be to passively obtain data from everyday use devices, rather than having people wear sensors or answer survey questions. In this study, we aim at assessing the feasibility of using passively obtained laptop trackpad data to detect acute stress. We present evidence that binary levels of acute stress (ie, whether it is present or not) can be detected from finger strokes alone.

In response to RQ1, we found out that our approximation to an MSD model, previously used with data from the computer mouse [4], cannot be used directly on trackpad data. Our analyses provide new insights into the complexity of the use of finger stroke data from trackpad operations for applying an MSD model. Unlike a computer mouse, handling a laptop trackpad is subject dependent. Some people may employ multiple hands and/or multiple fingers to perform the same tasks. This creates challenges to process the data due to the gaps present in the strokes and the difficulty of obtaining a clean intentional segment of data, with a clear beginning and end, required to get meaningful MSD results. These limitations leave little useful data for analysis, and we did not observe a difference in either of the two MSD parameters (ω and Γ) between the tRelaxed versus tStressed phases. However, our analyses were restricted to the largest stroke per trial. It is possible that in the wild, where we can collect larger amounts of data that would allow us to extract valid displacement segments, an MSD approach may still be useful to assess changes due to muscle tension under acute stress.

Our RQ2’s results are encouraging, showing the robustness of the mean and SD of the finger contact area as a stress sensor. Both metrics were higher during the stressed tasks as compared to the relaxed tasks. We show that as few as 4 click trials per subject are sufficient to see a significant difference, which is promising for longitudinal studies. Overall, we present evidence of the use of the finger contact area from laptop trackpad measurements to detect acute stress while performing common computer tasks.

This is the first study showing that the mean and SD of the contact area under the finger on a trackpad are higher under stress compared to a relaxed state (ie, Mixed Task model). Theoretically, the mean value is related to mean force production [19] and higher variability [17] under stressful situations in the wrist and forearm muscles. There may be other underlying mechanisms (eg, changes in hand or body posture [40]) through which stress may influence finger dynamics, and future studies should monitor those variables as well. In the Task-Specific model, differences in mean and SD of the contact area under the finger were different for click trials. If combined, mean or SD, we could detect up to 94% of acute stress events. This is encouraging because clicking tasks represent nearly 70% of all computer mouse events in a typical day [41].

With just the initial 10% of clicking trials (ie, 4 clicks out of 40), we could still see small and medium effect size differences in mean and SD of the contact area under the finger. Since click trials were not always carried out right after the stressor, the difference is not due to ordering effects; here we are referring to ordering of click, steer, and drag and drop, and not the ordering of stress-relax conditions. We attribute the strength of using so little data to the proximity to the stressor, as there is a clear decay of stress over time, which would clearly affect values averaged over longer periods. Furthermore, we simulated a scenario where clicks are produced at a different rate (ie, going from 1 click produced every ~3.5 seconds to every ~45 seconds). We obtained significant results even for the lowest click productivity we could simulate with our data (1 click/45 s). In a separate pilot study, collecting computer mouse data “in the wild,” we have seen that a typical user may generate a click every 10-30 seconds, and at least 1000 clicks are generated in any given working day. Thus, we feel confident we can use our findings as the basis for a passive, continuous, scalable, inexpensive, and unobtrusive stress sensor. In the wild, of course, changes in muscle tension would not only be due to affective (ie, ANS) processes but also due to cognitive processes (see Potential Interaction Effects With Task Performance section). Ultimately, a reliable “in the wild” stress detection system would have to combine multiple modalities, such as biomechanical, behavioral, and physiological sensors, as well as contextual information.

Mean and SD of finger velocity and acceleration were similar across the two phases, as well as trial completion time. This reconfirms that our stressor did not change the cognitive performance of the subjects but only their affective state. However, the relationships between stress, velocity, and acceleration may not be simple. For example, one may say that under a stressful situation, a user may want to move the fingers with higher velocity and/or acceleration; however, after a certain point, it will also start affecting the accuracy of this fine motor control task [42]. Thus, the user may not use higher velocities and/or acceleration under stress. Further studies with different elicitations to modify the performance rewards must be performed to observe interaction effects between these variables.

At least two studies have shown higher finger pressure under stress [28] and under frustrating [7] conditions while using touchscreen devices. However, the usage and handling of touchscreen devices are not the same as that of a trackpad, in terms of fine and precise interaction [43]. The capacitive technologies used in touchscreens and trackpads are different in terms of how much pressure needs to be applied: typically, the trackpad requires stronger pressure [43]. Additional research would be needed to determine if our finger dynamics from contact area (ie, mean and SD) can be translated to other types of touch-sensitive devices.

Our study has four limitations related to sample size, stressor effect size, apparatus, and assumptions for modeling. First, because of the small sample size (N=18) we could not investigate the effects of age, gender, length of experience with trackpads, and frequency of day-to-day usage of trackpads. We also could not build reliable predictive models (eg, using different ML techniques), which could parse out the importance of various features in predicting acute stress. Second, given the mild nature of the stressor, not all users (18/22, 82%) responded to our manipulation. Nevertheless, this level of efficacy is expected for a mild stressor and in line with our prior study [6]. Third, although we do not expect the behavior of users to be fundamentally different across different laptops, our experiment should be revalidated on other models of trackpads. Fourth, using finger contact area as a proxy to pressure assumes a linear relationship between the two. However, factors like the angle of the finger can also influence this relationship. Since the time of the data collection for this study, Apple has updated the MultiTouchSupport framework, and it now also provides information about pressure and capacitance density underneath the fingers. Additional models collecting this new data would be quite useful to strengthen the precision of our detection method further.

We plan to expand current results by running lab experiments with larger sample sizes; by logging other actions performed using trackpads, such as scrolling and multiple finger gestures; and by collecting other behavioral metrics, such as click production rate, the timing between clicks, etc. Once we collect more data, both in the laboratory settings and “in the wild,” we aim at building predictive models that include both biomechanical and behavioral metrics to increase the accuracy and specificity of stress prediction in real time.

As we move out of the lab, we will explore ways to carry out longitudinal studies collecting data from our passive loggers across devices (ie, mice, trackpads, touchpads), contextual data, and stress “labels” obtained from empirical sampling methods or physiological measurements. As we accumulate richer and bigger datasets, we plan to investigate data-intensive unsupervised and supervised ML methods to optimize data collection, preprocessing, and prediction. For example, we plan to investigate algorithms to parse types of events and movements (eg, clicks, drag and drop, etc) or to select segments that may have the right morphology to derive biomechanical features (eg, MSD parameters, contact area, etc). Ultimately, we hope that our lab and field findings can help select the appropriate behavioral, biomechanical, contextual, and perceptual features (ie, feature engineering) to train ML systems to be able to detect stress levels throughout the day.

The current methods for sensing stress are often inconvenient, expensive, or suffer from limited adherence. In this lab study, we show the efficacy of repurposing signals from a laptop trackpad to detect acute mental stress. We showed that with a handful of click events, we could use the mean and SD of the finger contact area to obtain a binary estimation of acute stress (ie, whether it is present or not). We validated our sensing models against well-known stress metrics such as self-reports and EDA. Complementarily, we validated that we did not elicit performance or cognitive confounds that may affect our affective stress elicitation. These results provide a firm baseline toward our future goal of deploying our algorithms in the wild, leveraging the trackpad as a potential unobtrusive, scalable, continuous, and inexpensive stress sensor.