^{1}

^{2}

^{3}

^{4}

^{5}

^{6}

This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.

Adaptive behavioral interventions are individualized interventions that vary support based on a person's evolving needs. Digital technologies enable these adaptive interventions to function at scale. Adaptive interventions show great promise for producing better results compared with static interventions related to health outcomes. Our central thesis is that adaptive interventions are more likely to succeed at helping individuals meet and maintain behavioral targets if its elements can be iteratively improved via data-driven testing (ie, optimization). Control systems engineering is a discipline focused on decision making in systems that change over time and has a wealth of methods that could be useful for optimizing adaptive interventions.

The purpose of this paper was to provide an introductory tutorial on when and what to do when using control systems engineering for designing and optimizing adaptive mobile health (mHealth) behavioral interventions.

We start with a review of the need for optimization, building on the multiphase optimization strategy (MOST). We then provide an overview of control systems engineering, followed by attributes of problems that are well matched to control engineering. Key steps in the development and optimization of an adaptive intervention from a control engineering perspective are then summarized, with a focus on why, what, and when to do subtasks in each step.

Control engineering offers exciting opportunities for optimizing individualization and adaptation elements of adaptive interventions. Arguably, the time is now for control systems engineers and behavioral and health scientists to partner to advance interventions that can be individualized, adaptive, and scalable. This tutorial should aid in creating the bridge between these communities.

Overwhelming evidence suggests health behaviors such as smoking, physical activity (PA), and diet are key to preventing noncommunicable diseases such as many forms of cancer, heart disease, and diabetes [

Adaptive interventions are complex interventions [

For example, we have been developing an adaptive PA intervention,

As this example illustrates, there are many elements within this seemingly simple adaptive intervention. The central thesis of this work is that adaptive interventions are more likely to succeed at helping individuals meet and maintain behavioral targets if its elements can be iteratively improved via data-driven testing of the elements. The classic evaluation strategy for behavioral interventions is the randomized controlled trial (RCT). An RCT provides information about whether an intervention package can produce an effect relative to a meaningful comparator (eg, current clinical best practice) but limited information about how, when, where, and for whom each element functions to produce the desired effect. As such, an RCT does not provide sufficient insights for supporting data-driven improvement (also called optimization) of the elements of an adaptive intervention such as

Control systems engineering is a field that focuses on decision making in systems that change over time. Control engineering is pervasive (eg, pacemakers, climate control, and robotics) but often goes unnoticed as a hidden technology [

The purpose of this paper was to provide an introductory tutorial on when and what to do when using control systems engineering for designing and optimizing adaptive mHealth behavioral interventions. We start with a review of the need for optimization, building on the multiphase optimization strategy (MOST) [

In classic RCT’s, all elements are combined into a unified package relative to another package. On the basis of this, limited information about each element, such as the tailoring variables to use for individualization or the decision rules to use for adaptation, is available. If an intervention package produces suboptimal results, it will be difficult, empirically, to localize what elements or interaction between elements could be further improved upon to produce a more potent intervention within RCTs.

Collins et al have been pioneering MOST, which provides structure for thinking about optimization of complex interventions [

The most common optimization trial used in MOST (indeed, sometimes inappropriately labeled a MOST trial) is the use of a factorial or fractional factorial design [

One approach for optimizing adaptive interventions is the sequential multiple assignment randomized trial (SMART) [

There is another emerging method for optimizing adaptive interventions called the microrandomization trial (MRT) [

As demonstrated by our publication record [

Control engineering has a long history dating back nearly a century and is pervasive (eg, pacemakers, artificial pancreas systems, and supply management) [

One can think of this like accounting. The controller keeps a ledger of measurements. In all controllers, including nonmodel-based controllers such as Proportional- Integral-Derivative (PID) controllers [

In more advanced controllers that include a dynamical model, such as model-predictive controllers [

Both model and nonmodel-based controllers conduct a series of simulations to predict responses to intervention options in the near and distant future, either based on deviations between the desired state and the state of the system alone (nonmodel-based controllers) or via dynamical models (model-based controllers). These forecasts are used to make decisions. The intervention option predicted to most likely foster movement toward the desired state within prespecified constraints (eg, only small daily changes to step goals allowed) is selected. In contrast, if-then rules require knowledge of the match between tailoring variables and intervention options before specification [

Control engineering includes a wealth of methods for optimizing adaptive interventions by managing and mitigating lack of knowledge related to intervention elements. Lack of knowledge can take various forms from sensor noise (eg, measurement noise when inferring steps [

Key concepts related to testing controllers are performance and robustness. Performance involves how well the controller can produce the desired effect as efficiently as possible. Robustness involves how well the controller can produce desired performance when issues such as poor measurement, models, or interventions or changing responsivity to interventions arise [

A closed loop experiment can be used to test the controller in relation to optimization targets, such as meeting and maintaining PA guidelines. It is closed loop because, like in closed loop system identification, a person’s response to each selected intervention option provided is documented and then taken into account when selecting the next intervention option, thus closing the loop. This type of experiment can include a variety of strategies to test the controller. For example, one could systematically vary providing the predicted optimal vs nonoptimal intervention option to test the controller, if appropriate for the research and intervention. This sort of strategy maps on to the computer science concept of exploring vs exploiting [

Within control systems engineering, it is common to use all of these methods (ie, system identification, closed loop experimentation, robustness testing) within a single system or individual. In particular, system identification experiments (ie, theory testing) and closed loop experiments (ie, testing with controller actions present to support, among other things, testing of the controller) can be offered sequentially to a single person and, indeed, decision rules can be defined on when to switch from one method to the next. For example, a closed loop experiment might be used to test a controller striving toward helping a person to meet PA guidelines. If the person meets the behavioral target for a prespecified time (eg, 2 weeks), this could trigger the switch to different optimization criteria, such as targeting maintenance of steps and minimization of interactions between the intervention and person (ie, a second controller optimization algorithm for maintenance). The combined study that includes system identification, closed loop experimentation, and robustness testing is what we call a

In this section, we describe attributes of problems that are well-matched to control engineering. We ground our discussion within the concrete case study of

Convincing evidence indicates PA is valuable for reducing risk of certain types of cancer [

One reason meeting and maintaining PA is hard may be because of the idiosyncratic and dynamic nature of steps. Specifically, taking steps occurs in both incidental and purposeful ways [

We have been developing

Screenshots of the Just Walk App. The image on the left is the view inside the app, which includes the suggested step goal for the day (in the red box), available points (in gold medal in the middle) and current steps (in green box). Below is the person’s step history. The image on the right is the app’s “widget,” which enables a person to receive feedback relative to their goal without opening the app.

We turn to a discussion on the types of problems that are well suited for control engineering (see

First, the problem is dynamic, meaning the input and output variables interact over time. Within

Second, interventions are available to foster movement from a less desirable to a more desirable state. As part of this, there are concrete decisions that can be made for each decision point. Note that these decisions can include providing or not providing an intervention or more continuous intervention options (suggested daily step goal).

Within

Third, the target outcome can be measured with sufficient temporal density over an extended period. In the

Finally, there is a need for definable desirable states for the target outcome(s), which are called

Within

Dynamic, input-outputs

Intervention options are available

Outcome variables are measurable (or inferable) intensively

A meaningful target or “state” exist

Frequent decision points

Previous theory available to guide model development

Other feasibly important variables can be intensively measured

Theorized dynamic interrelationships between inputs or outputs (eg, feedback)

Beyond these requirements, there are several desirable attributes. First, it is advantageous to have frequent decision points, such as every hour, day, or week. Within

Second, it is desirable if previous knowledge about the phenomenon is available. Within our example, we used the SCT to inform measurement selection, a model structure for defining our dynamical model and the interrelationships between variables, intervention selection, and the design of our study, discussed below [

Finally, it is desirable that other variables that could impact the outcome can be measured. Within

Finally, if there are strong theoretical reasons to hypothesize feedback loops and lagged effects [

In this section, we highlight suggested steps that could be used when using control engineering methods to optimize adaptive interventions. A full review on exactly how to do each step is beyond the scope of this introductory tutorial. Instead, for each step, we define

Although the use of the word “step” may imply a linear process, it often is not. For example, it can be highly advantageous to select a general theoretical model (a task within step 1) and to then define optimization criteria (a task in step 4) before moving on to creating or selecting intervention options (step 2) or to even start with optimization criteria as a definition of success, which is advocated for in agile science [

An important prestep is to make an initial decision on the type of controller one is targeting. Although there are many considerations involved in the selection of the appropriate controller, at a high level, selection of one controller over another largely hinges on the anticipated complexity of the dynamical system, the degree to which a model can be generated that is actually predictive or useful for making decisions based on forecasted responses, and the degree to which the dynamics can be inferred from the dynamics of one (or a relatively few number) of variables (nonmodel-based), as opposed to the response of multiple interrelated variables (model-based). If the guiding theoretical model implies a complex dynamic system that would not be well represented by monitoring only intervention options and outcomes, then a model-driven controller would likely be most appropriate. If, however, the dynamics can be picked up adequately with intervention options and outcomes, such as the direction a boat is pointed as measured via a compass as used within a boat autopilot (a classic PID controller), then a nonmodel-based controller is appropriate. There is a lot more subtlety involved in selecting the right controller (eg, the possibility of model-based PID controllers), and interested readers can gain more insights on control options here [

Suggested steps include the following: (1) derive a preliminary dynamical model; (2) select intervention options (ie, type, frequency, and dosages) and outcomes; (3) conduct system identification (ie, theory testing); (4) design the controller; and (5) conduct a control optimization trial (ie, intervention element testing).

This step is important for establishing a well-specified framework for understanding the eventual adaptive intervention and guiding all subsequent work. The tasks involve first specifying a general theoretical model for guiding the work, then translating that into a dynamical model, and finally, the option of vetting this dynamical model either via simulation studies, secondary data analyses, or both.

Like in MOST, a theoretical model is used to provide structure and specification about key intervention options, outcome measures, and other variables that impact the outcome measures. It is strongly advised to almost always engage in this step as it provides the foundation for understanding predictions and decisions made within the eventual adaptive intervention. The one caveat is when an adaptive intervention is being generated when very little is known about the phenomenon, except that it is highly dynamic. When this is the case, it is often more appropriate to do noninterventional work such as conducting more naturalistic studies such as ecological momentary assessment or human-centered design work [

Although there is not a single way to develop a theoretical model, we suggest thinking clearly through three reference points and using each to triangulate toward, first, a theoretical model and then a dynamical model. These three references are (1) prior theories, particularly those that have been well-validated in the literature among the target group; (2) prior empirical work about what works in general and other key variables to be aware of for the target group; and (3) hands-on experience and interactions with the target group in the form of human-centered design methods such as interviews, observation, codesign, or prototyping, to gain insights about your target group that may not be well understood or encapsulated in prior theories or evidence. For details on exactly how to create or select an appropriate theoretical model, we suggested the following references (see [

Translation of a theoretical model into a dynamical model requires far clearer specification of the prediction. For model-based controllers, this task is required. Creating a dynamical model involves clear specification of a variety of issues such as model structure, anticipated directionality and strength of relations between variables, and anticipated dynamics of the interrelationships [

The final optional task within this step is simulation studies or analyses with secondary data to vet a dynamical model. As the previous step highlights, dynamical models often require a high degree of mathematical specification on predictions. The use of simulations, such as changing one variable to see how the other variables might respond within the system, is valuable to gauge if the dynamical model is producing the sorts of effects that would be anticipated. If the simulated changes in one variable produce effects that are not anticipated, this can be used to either check the math or check the assumptions about the problem. Either way, it improves precision and understanding on what is being hypothesized dynamically. Secondary analyses can also be valuable as data can be used to ground the predictions of the dynamical model, again, to see if the dynamical model is working according to both expectations and available evidence. For more details on how to create a dynamical model and do preliminary vetting via simulation studies and secondary analyses, see [

Within our

Defining target intervention options and outcome metrics are the defining features of an adaptive intervention and, thus, this step is essential. The key tasks of this step include defining the outcome metric(s) being targeted (which will be translated into optimization criteria in step 4), defining the intervention options and then, optionally, also specifying clear dynamic hypotheses on how these intervention options will dynamically interact with the person to produce desired changes to the outcome(s).

Clearly defining the outcomes is a logical follow-up step from the theory and dynamical modeling work. This is because, within the prior step, it is technically possible to do most of step 1, save the secondary analyses, without any concrete outcome measure defined (eg, steps/day or hours of sleep per night). Defining outcome metrics to target is important as it establishes a grounding on the purpose of the specific adaptive intervention. As discussed in the previous section, outcome metrics are best when they can be measured repeatedly over time to establish the current state of the target person relative to the desired final state. The intervention options can then be defined to impact the outcome metrics dynamically. These intervention options could be thought of as the essential levers the adaptive intervention can use to make adjustments and thus, facilitate movement from a less desirable to a more desirable state.

Although not required, it can be valuable to generate a dynamic hypothesis about the interrelationship between an intervention and an outcome to further ground thinking about the intervention. Although there are many ways to think about dynamic hypotheses, one way is to think in terms of outcome responsivity to the intervention options when a person is in a different state or context, including changing disease state or changes in their readiness for change (eg, stages of change). The Transtheoretical Model (TTM) establishes a basic (albeit slow) dynamic hypothesis in that different processes of change are hypothesized to be needed for different stages of change [

Simplified dynamical model version of Social Cognitive Theory.

Similarly, one might theorize that a person will respond differently to an intervention if it is provided to them when they are stressed vs not, or at home vs at work. These variations, which are further described in detail elsewhere using the modeling logic of state-spaces [

These dynamic hypotheses can take various forms. A relatively simple dynamic hypothesis could be to specify if-then statements for different states or contexts of the individual, which is the implicit structure used in the TTM (eg, if stage of change=X then process of change=Y). This could be useful for stepped care interventions (eg, see [

We chose to focus on individualized step goal suggestions and provision of points as our two dynamic intervention components (grounded in self-monitoring as the third, but we assumed that to be a constant intervention component). Our key outcome measure in

Dynamic hypothesis.

For example, if a goal of 9000 steps or greater is suggested for our

This step is most distinct from other forms of testing and optimization within behavioral interventions. As such, we include a great deal more information here to highlight the logic and overall approach.

From a control systems perspective, the primary goal of this step is to estimate and validate dynamical models. This is valuable in and of itself, regardless of any subsequent controller, because it is explicitly focused on understanding a “system” or phenomena, such as, in this case, an individual human. To put it in more behavioral terms, system identification is a form of dynamic theory testing. System identification is also important for later steps, if there is a desire to use model-based controllers. Although not commonly the focus in control engineering, this type of study can also be used to select tailoring variables for individuals (ie, data-driven individualization), test dynamic hypotheses, or develop a benchmark comparator for optimization criteria when prior work provides limited insights on a meaningful benchmark. It is also technically possible to conduct a rigorous system identification experiment while also pilot testing aspects of the intervention and other protocols to test feasibility issues, if needed, because system identification is an inherently n-of-1 or idiographic approach, though this last strategy is not necessarily recommended.

As a reminder, within a

Suggested tasks that could be included in this step include the following: (1) design of the system identification experiment and analytic plan and (2) data analyses. If pilot testing of the technology is also needed, other optional tasks could include the following: (1) human-centered design work [

In terms of system identification, there is a rich literature, including toolkits within MATLAB (MathWorks), on procedures and best practices for the design of a system identification experiment and analytic plans [

With a sense of the amplitude defined, the next task involves designing for adequate excitation over time, which minimizes error for model estimation. This involves the length of a “cycle” and the number of cycles needed in a study to achieve sufficient minimization of error in estimation and validation. A cycle is a deterministic, repeatable pattern that defines provision of intervention options to an individual. Intervention options can be provisioned to mimic randomness via pseudorandom signals that can achieve the valuable properties of randomness for causal inference, while still being deterministic and, thus, repeatable (for more details see [

Once data are collected, the process of data analyses takes place. A central logic of dynamical systems modeling, as with other aspects of control systems engineering, is triangulation. In particular, system identification toolkits (eg, those available via MATLAB) include a wide range of strategies to examine time series data produced from system identification experiments, such as different visualizations, step-response curves (ie, the unique influence of each variable on the outcome, much like a partial r2), or model fits for both estimation and validation. Each one of these provides a different understanding on the overall reliability and validity of the dynamical models produced. As such, they are all used with the goal of defining dynamical models that work according to expectations across these tests.

Beyond the criteria used to evaluate the models, there are also different analytic techniques that can be used as part of dynamical systems modeling. For the sake of simplicity, we describe black-box dynamical modeling vs semiphysical or grey-box modeling. Central to these different modeling efforts is the degree to which prior theory and evidence is taken into account when defining a dynamical model structure. On one end are black box models from methods such as Auto-regressive model with eXogenous Input (ARX) modeling, which are much like generalized linear models. These models include no model structure to define the interrelationships between structures beyond ordinary linear regression accounting for repeated measures. Semiphysical modeling, on the other hand, includes theorized model structures, predicted dynamics, and other factors that are either known or theorized to be true in terms of the interrelationship between variables. One could think of semiphysical modeling as a dynamical version of structural equation modeling [

The final task is to define

With these tasks defined in abstract, we turn to the

To support the eventual controller, we chose to run an open loop system identification experiment. This was because, although we did conduct secondary data analyses to vet our dynamical model [

In

A full description on the design of the study is beyond the scope of this paper but has been described elsewhere, which includes concrete strategies for achieving the equivalent of “power” calculations for an open loop system identification experiment [

A full discussion on the analyses and results are beyond the scope of this paper, but interested readers can find more information at [

Using percentage model fit as a benchmark, we conducted data-driven analyses to support optimization of the dynamical model that conceptually maps on roughly to reliability and validity. In this case, reliability and validity are estimated for our dynamical models for predicting human behavior (as indicated by model fit) and, by extension, the selection of tailoring variables. For every individual, we conducted an exhaustive search of potential variations of predictors (eg, only our manipulated inputs or up to four additional endogenous or exogenous variables as plausible tailoring variables) using an ARX approach. In line with the leave-one-out approach commonly used when cross-validating models such as PA estimation via accelerometers [

For selection of the model and, thus, the tailoring variables to use for each person, we chose to use multiple criteria with the first three reflective of issues of reliability [

We then turned to good enough validity. As these analyses are a variation of multiple linear regression, and the model fit estimate is analogous to r^{2}, we chose Cohen’s conventions of explaining 3% of variance as a small effect, 13% as a medium effect, and 26% as a large effect [

System identification open loop experiment for Just Walk. These two signals were designed a priori using a pseudorandom signal design strategy. This strategy enabled specification of repeated 16-day cycles (delineated as different colors), which allows for robust data for estimation and validation of dynamical models.

Visualization from one participant from Auto-Regressive Dynamical Modeling.

Note, however, that further validity testing related to individualization is possible and a core target of the more definitive optimization trial, the closed loop experiment (see below). Furthermore, we also fully acknowledge that our approach is only one of many (see Discussion). The overall average model fit (estimation and validation data) for all participants combined was 19.2% (SD 9.25). The range was 6.3% to 46%. Using Cohen’s conventions, 20 out of 20 participants met the small effect threshold of explaining 3% of variance, 16 out of 20 met the 13% medium effect level, and 2 out of 20 met the 26% large effect level. On the basis of this, we achieved our good enough target of explained variance for individuals, thus justifying subsequent steps. Although it is unclear what the minimal levels are needed for establishing robust individualization based solely on this, it does provide a preliminary indication of the ability to make distinctions between people in terms of tailoring variables. For example, using the medium effect as a minimal threshold, our approach produced meaningful individualized models for 80% of our sample [

In terms of tailoring variable selection, different tailoring variables were identified for different people [

As of this writing, the team is conducting semiphysical modeling [

As this example illustrates, a great deal of valuable insights about human behavior and outcomes can be gleaned from system identification experiments. As this example also illustrates, this step can generate meaningful scientific insights as a mechanism for doing rigorous theory testing that is grounded in an individual first and then can be generalized if similar model structures are found, what we previously called data-driven case studies [

The next step is to design the controller. This step is essential as it is the mechanism whereby prior insights can be translated into actionable dynamic decision rules (ie, the controller) for guiding an adaptive intervention. The key steps in this process include defining optimization criteria (eg, set-points), constraints of the controller (eg, clinical constraints of the intervention), and, for more complex controllers, alternative strategies the controller could use to maintain robustness to factors such as a person’s changing responsivity to an intervention. Creation of these is often supported via all of the prior work done (eg, dynamical model, intervention and outcome specification, and system identification experiment), as well as additional simulation studies specifically focused on the robustness of the controller. These steps are done in any type of controller, including those that are not model-based.

In terms of controller design, the central focus of controller design is to define the targeted right balance between performance and robustness. Within a controller, strategies for supporting performance largely revolve around the quality of the previous steps. In particular, performance is improved when potent interventions and predictive models are available to be used by the controller. The prior work provides a foundation for anticipated performance of the intervention options and value of the dynamical model for making predictions. Strategies for maintaining robustness can be devised to help manage and mitigate these risks, which tends to be the larger focus of the controller design for this step. For interested readers, see our more detailed papers formulating our controller [

In terms of key tasks, the type of controller being targeted must be defined (eg, model-based or nonmodel based); several parameters for the controller must be defined, including the optimization criteria, constraints, and strategies for achieving robustness; and finally, simulation studies can be conducted to examine anticipated issues of robustness. The optimization criteria can be thought of as a definition of success that can be operationalized based on a measurable outcome variable. Constraints are the parameters that define what is feasible or appropriate within a given domain, such as what is safe, appropriate, or clinically viable. Finally, there are a wide range of strategies for supporting robustness. These assertions can be examined via simulation studies [

Within the

Model-predictive controller “Receding Horizon” strategy. The model predictive controller visualized here is simplified to include only one controlled variable (desired daily steps), one input (ie, goals), and one disturbance (ie, environmental context). Controller moves (ie, goals) are calculated over a horizon, and only the first control move calculated is implemented. The entire procedure is repeated at the next assessment period and continues until the end of intervention.

In terms of how our controller works, the model-predictive controller forecasts changes in outcomes (ie, steps, intervention adherence) over time to determine an error projection that reflects current and expected deviations from the optimization criterion of 10,000 steps/day or +3000 steps/day from baseline. On the basis of this error projection, a real-time optimization algorithm chooses the sequence of future control actions (eg, adjusts step goal, points, and other factors) that minimizes the difference between the set-point (eg, 10,000 steps) and current steps.

The optimization problem is solved for each day considering a prediction time to obtain a predicted optimal step goal suggestion for each decision point. The first recommendation is provided, and the process repeats at each decision point. The model-predictive control strategy continually reevaluates the quality of the previous day’s predictions on what was actually observed. The information can be incorporated into the model-predictive control algorithm, particularly if there are alternative strategies the controller might take based on changing observations for maintaining robustness.

We have conducted simulation studies to stress test the design of our controller.

This is the key step for unpacking complex adaptive interventions via control systems engineering methods. This step can provide insights about

The key tasks of this step include clear definition of the elements of the adaptive intervention, the design of subexperiments (eg, open loop system identification and closed loop experimentation) and data analysis plan to test the elements, and conducting the trial and the analyses.

As already highlighted, the key elements of an adaptive intervention include the decision points, tailoring variables (or, in this case, dynamical models), decision rules (or, in this case, the controllers), and, of course, the intervention options themselves and the meaningful proximal outcomes the intervention options target. In terms of decision points, these are often defined based on clinical intuition, such as the case in

As highlighted in the introduction, it is common in control systems engineering to include multiple experiments provided sequentially, over time, to the same system (ie, person in this context). The key, from a design standpoint of the subexperiments, is to think through what is clinically appropriate or feasible and also what the logical progression is in terms of the likely changes that will occur within the target individual. In terms of the data analytic plan, as with system identification, there is a wealth of analytic strategies that are available, largely within MATLAB, for conducting the analyses. Much of the testing of controllers is actually built into the controllers themselves as, ultimately, they are mathematical equations seeking to minimize error while accounting for noise and other unknown issues. Controllers, thus, engage in self-testing relative to optimization criteria. The key advantage here of self-correction is also arguably a weakness, as this work hinges on the quality of the optimization criteria (a point we return to in the Discussion). A full description of the type of analyses that can be done and the many ways in which to design effective exploration or exploitation is beyond the scope of this introductory text, but interested readers should examine here [

Controller simulation.

Control optimization trial for Just Walk.

Returning to our

With this experiment completed, we will be able to systematically test and optimize core elements of our adaptive intervention. In particular, our open loop system identification portion enables data-driven optimization for individualized dynamical models and selecting individualized tailoring variables as described above. Unlike the above work, the final definition of success, which is a person maintaining targeted step levels, will be available and, thus, can be used to define percentage model fits that are, indeed, good enough for individualization purposes. Our closed loop subexperiments allow us to optimize our controller’s ability to achieve set-point targets for each individual for each state, including initiation, maintenance, and possible relapses. We can judge success or failure relative to our optimization criteria (eg, 10,000 steps/day).

Furthermore, we can also produce aggregate (also called nomothetic) information across the sample of participants. Specifically, another optimization check involves comparison of the percentage of our sample that achieves our maintenance targets relative to current best practice PA interventions that appear to produce maintenance targets for approximately 50% of their samples [

Control systems engineering is a rich discipline that has strategies mHealth researchers and practitioners can use for optimizing elements of adaptive interventions. It is particularly well matched to problems that (1) are dynamic, (2) have useful dynamic interventions available, (3) have an outcome measure that can be measured with sufficient temporal density over an extended time period, and (4) have desirable states for the target outcomes that can be defined as the optimization criteria on a case-by-case basis. There are five suggested (though not necessarily all required) steps for optimizing an adaptive intervention via control engineering: (1) derive a preliminary dynamical model, (2) select intervention options, (3) conduct a system identification open loop experiment, (4) design the controller and optimization criteria, and (5) conduct a control systems optimization trial. This approach holds great promise for expanding the potential of adaptive interventions. This is because control engineering provides a wide range of approaches to systematically unpack and test or optimize the various elements of adaptive interventions both on a case-by-case or idiographic and an aggregate or nomothetic level.

These steps map on to the MOST framework [

Beyond the steps, there is also synergy between MOST and control engineering principles. A central focus of MOST is efficiency, including the use of efficient experimental designs and grounding research in real-world constraints related to implementation with the long-term goal of facilitating more efficient and robust knowledge accumulation across studies. Continuous optimization is the second common principle that emphasizes the logic of a continual, iterative process related to further improving and refining behavioral interventions. Control engineering shares these principles of efficiency and continuous optimization. Overall, our work fits well with MOST and current trends in mHealth and the science of behavior change [

As highlighted already, control engineering practices include the principle of triangulation [

These methods are designed to understand and support better prediction and decision making for a given individual. Although we have highlighted the strengths of this approach, there are inherent weaknesses. For example, one potential trade-off exists related to the optimization criteria. If the optimization criteria that are chosen are not meaningful, then even if the controller achieves success (ie, optimization criteria are met), then nothing clinically meaningful has been achieved. This can be mitigated, of course, with optimization criteria that are grounded in clear evidence showing that they are clinically meaningful as is the case within

A second major trade-off of control engineering and, indeed, any idiographic approach, is the undervaluing of generalizability to other individuals and contexts. This establishes the need for other methods that are better at balancing this idiographic emphasis with more of a nomothetic emphasis, such as RCTs. With that said, generalization to other individuals and contexts can feasibly occur via a different pathway toward generalizability knowledge, namely, the generalizability concept of causal explanatory models [

Returning to the concept of triangulation, an RCT can balance out the weaknesses of control engineering methods. As illustrated in the introduction and optimization section, RCTs compromise on providing insights about how, when, where, and for whom a given intervention element works, in the pursuit of stronger internal validity at the intervention package level and also increased external validity in terms of statistical claims of generalizability to the population the sample is conceptually drawn from. As the control optimization trial is an inherently n-of-1 study design, it enables the possibility of it being embedded within an RCT as the intervention arm. This possibility enables a highly efficient way of conducting multiple tests within a single trial that is squarely grounded in the philosophical logic of triangulation, as one trial can test intervention elements and also compare the package to another package. Indeed, including a control condition as a comparison with a control systems optimization trial is, arguably, a highly efficiently rigorous approach to test an adaptive intervention [

We emphasize triangulation as we see this as well matched to the complexity of adaptive interventions and possibly behavioral interventions more generally, even outside of the domain of control systems engineering. It is the cornerstone of our key thesis that adaptive interventions are more likely to succeed if its elements can be iteratively improved via optimization. In particular, the fact that there are so many elements within an adaptive intervention (eg, intervention components, decision points, tailoring variables, and decision rules) establishes the need for triangulation. This fits with discussions in psychology, such as the need for a pluralistic approach to causality [

As articulated elsewhere, advancements in digital technologies are rapidly converging to enable a new era in the understanding of human behavior [

In terms of limitations and future work, more work is needed to clearly evaluate the utility of this approach relative to other methods. For example, the current method used in MOST for optimizing a static intervention is a factorial trial, and SMART and MRT are proposed for adaptive interventions. One valuable test to be conducted is comparison of an optimized intervention to an intervention that was not optimized using these methods. This comparison can be made using an RCT. As the control optimization trial is an inherently n-of-1 method, it is possible to compare the control optimization trial, as a proxy of an optimized intervention, with a control condition that lacks control engineering features. A trial such as this would provide insights on the plausible added value optimization via control engineering may produce relative to more traditional approaches for intervention development whereby the elements are not optimized but, instead, the elements of the intervention are defined based on prior aggregate evidence, user-centered research, and theory.

Building on this point, future work should focus on providing greater clarity on when to use which method for optimizing static and adaptive interventions. As one possible formulation on this, SMART appears useful when the goal is the selection of a progression of decisions to make with relatively infrequent adaptation (eg, once every few months) and with well-specified if-then decision rules. As such, SMART might be particularly valuable within clinical practice. MRT appears particularly valuable for just-in-time adaptive interventions. We argue that control systems engineering methods are likely particularly valuable when the goal is to facilitate a more long-term trajectory of change, such as gradually increasing a target behavior whereby achievement of a desired state cannot happen immediately (eg from 6000 steps/day to 10,000 steps/day or a 5% reduction in weight) but, instead, requires slow progression and building up of skills. Similarly, control engineering methods can also be valuable for facilitating maintenance of a targeted set-point by facilitating small adjustments and provision of support in sort of stepped-care framework. Although we think these general principles are correct conceptually, future empirical work is needed to explore the strengths and limitations of these approaches and the assertions made on when to use which method.

More work related to establishing meaning optimization criteria is needed. This work hinges on well-specified definitions of success that are clinically and practically meaningful but that is not necessarily always available for all elements of an adaptive intervention. For example, we established our model fit estimates as good enough for individualization based on Cohen’s work [

Finally, future work should further explore if and, if so, how to integrate the logic of triangulation more actively within the development of mHealth interventions. As highlighted before, there is already research starting in this domain but, future research that provides scaffolding for health and behavioral scientists to work through this more complex approach to the design, optimization, and evaluation of interventions could be valuable. We have started this process through the articulation of agile science [

In sum, mHealth is well poised to take advantage of control engineering methods for the optimization of adaptive interventions. The time is now for health and behavioral scientists to more closely examine control engineering methods. If the approach proves valuable for health problems, new partnerships should be forged between health and behavioral sciences and control systems engineers in the design, optimization, and evaluation of adaptive interventions.

Auto-regressive model with eXogenous Input

mobile health

multiphase optimization strategy

microrandomization trial

physical activity

Proportional-Integral-Derivative

randomized controlled trial

reinforcement learning

Transtheoretical Model

Social Cognitive Theory

sequential multiple assignment randomized trial

This work was supported by a grant from the National Science Foundation (IIS: 1449751, PI: Hekler).

EBH was the principal investigator for the collective research efforts and primarily drafted the manuscript. All other authors reviewed and provided support to sections of the paper and also engaged in the design, implementation, and evaluation of the

None declared.