To meet our first objective, we partnered with VA clinical and health research psychologists with pain experience and researchers with extensive pain expertise (DMH, MSM, and AMM) who had previously developed messages to support pain self-management. We adapted these messages to focus specifically on 3 nonpharmacological psychological and behavioral treatments for chronic pain, which included cognitive behavioral therapy, mindfulness-based stress reduction, and acceptance and commitment therapy. Nonpharmacological approaches are recommended as the first line of treatment for chronic pain in the VA; yet, these evidence-based treatments have historically had low rates of patient engagement [22,23]. We also conducted a literature review to ensure that the psychological and behavioral pain treatment messages reflected constructs from SDT. Finally, we developed message prototypes to visually represent common technology modalities, where patients could view the messages.

Prior to this project, the VA Pain Management, Opioid Safety, and Prescription Drug Monitoring Program Office tasked the aforementioned VA subject matter experts with developing short SMS text messages about pain management approaches, including topics on sleep hygiene, mindfulness, and self-care. These texts were formatted for deployment via Annie, a VA SMS text messaging service that sends automated messages to veteran patients [24]. Messages were iteratively refined by both subject matter experts and an Annie expert (Figure 1). The messages developed in step 1 served as the foundation for the messages in step 2.

In this step, we refined messages to focus on evidence-based psychological and behavioral treatments, as opposed to general pain management strategies. ACG and SJJ first conducted a literature review of health interventions enacting components of SDT, including autonomy, relatedness, and competence. Teixeira et al [25] developed a list of 21 motivation and behavior change techniques (MBCTs) linked to SDT via an iterative expert panel consensus study. These techniques include specific ways to bolster components of SDT, including autonomy (eg, use noncontrolling, informational language), relatedness (eg, acknowledge and respect perspectives and feelings), and competence (eg, help develop a clear and concrete plan of action). For example, a technique to bolster autonomy support is using nonjudgmental language that conveys choice, such as using “might” or “could” instead of “should.” Following the literature review, ACG and SJJ engaged in iterative development and refinement of messages, with concurrent review by a team of subject matter experts in chronic pain, psychology, and behavioral health (DMH, MSM, and AMM) during each iteration (see Multimedia Appendix 1 for the final set of messages [25]).

Prototypes are tangible examples of abstract ideas that can be used to generate knowledge and feedback [26]. It was important to develop prototypes that visually represented common technological modalities to give engagement boards realistic examples to react to. As such, we created digital prototypes for messages that would appear on various patient-facing platforms: email, patient portal message, and SMS text message (Figure 2). We used free internet resources (eg, iFake Text Message) and replicated email and patient portal interfaces on Microsoft PowerPoint to develop prototypes.

To meet our second objective, we sequentially met with patients and caregivers on 3 patient engagement boards and used rapid prototyping after each meeting to improve the content and understanding of the messages between iterations. We found it beneficial to involve engagement boards both early in the design process and often thereafter, as it allowed us to create a more targeted set of messages informed by end users.

Patients, families, and caregivers participating on engagement boards should be representative of the target population and include diverse backgrounds, expertise, and experiences. We sought to have a variety of chronic pain experiences represented, including patients with opioid use disorder, as it is common among patients with chronic pain [27]. As such, we identified 3 separate engagement boards of varying demographics: Veteran and Family Advisory Committee [28], Pain/Opioid Consortium of Research Veteran Engagement Panel [29], and Substance Addiction and Recovery Veteran Engagement Board (formerly Opioid Addiction and Recovery Veteran Engagement Board) [30]. Each board is comprised of approximately 13-18 veterans or family members of diverse demographics and backgrounds. The latter 2 boards focus on the input of patients from across the United States with a lived experience of chronic pain, opioid medication use, or opioid use disorder.

We found that patients’ input varied based on their backgrounds and experiences with pain and prior treatments. Some words and phrases that seemed to be beneficial in initial drafts deterred some patients. As one example, some patients pointed out that phrases including “relax” may not be effective for someone who is experiencing anxiety because of their pain. Our team had to weigh these differing opinions and feedback, which is a common issue during the design process. We sought to align contrasting feedback with SDT constructs and the MBCTs (see step 6).

To refine the messages, we incorporated feedback from each engagement board prior to meeting with a subsequent board using rapid prototyping (Figure 3). Before each meeting, our team was required to submit information and materials that we planned to present and specify the types of feedback requested from patients. These premeeting materials were crafted using lay terms so that all engagement board members would have the opportunity to provide feedback regardless of education levels. Researchers should account for the time required to prepare such premeeting information as well as be intentional and specific about the types of feedback desired. We also had to be flexible in terms of scheduling, as each engagement board had a set day each month dedicated to meeting with researchers.

During each engagement board meeting, patients were asked to reflect on what they liked and disliked about the prototypes, changes, and additions and what would be the most effective way to communicate this information to patients with chronic pain. It is important to note that each meeting was facilitated by a representative of the board who was often a veteran or patient. The presence of a meeting facilitator allowed the researchers (ACG and SJJ) to focus on feedback and engagement with board members while the facilitator took notes. After each meeting, ACG and SJJ refined prototypes through team-based discussions and consensus. We reviewed meeting transcripts, corroborated notes, and consulted with our larger team’s subject matter experts to align feedback with prototype edits.

Overall, there was a lot of variation in patient preferences regarding the content and language of the messages, which differed both within and across engagement boards. Our decision process to compile input from each engagement board, synthesize feedback, and align decisions with the most related MBCT is depicted in Figure 4. For example, some patients recommended phrases such as “There’s good news!” while others believed that a message that started with this phrase would sound like an infomercial and turn people away. Leveraging MBCT 11 (Demonstrate and show interest in the person) and rearranging the message resulted in the following: “If you sometimes feel frustrated or sad because of your chronic pain, the VA offers Acceptance and Commitment Therapy which can help.”

Patients also provided valuable feedback on setting expectations about the duration, frequency, and sender of the messages. Some recommended specifying the period of messages (“Over the next two weeks you’ll receive messages about pain treatments”), and others were interested in knowing the number of messages (“Message 1 of 10”), as receiving too many messages might discourage patients from participating. Patients also commented on the importance of the first sentence in a message; some preferred to begin with “We’re e-mailing you about your chronic pain,” while a few desired a statement that resonated with them such as “Pain is complicated.” To be inclusive of varying views, we framed the introductory messages based on the MBCTs for clarifying expectations (MBCT 16) and providing statements that acknowledge and respect perspectives and feelings (MBCT 8). For instance, our final refined introductory message began with: “Hi, it’s your VA care team. We’re e-mailing you about chronic pain. Pain can be complicated and affect many parts of your life ... Over the next few weeks, we will send you some information about the following treatments.” Figure 5 provides a comparison of the initial prototype and final prototype message for acceptance and commitment therapy after synthesizing engagement board input and alignment with MBCTs (see Multimedia Appendix 2 for additional message prototypes).

Based on the input received from the engagement boards and our teams’ reflections, we identified strategies to collaborate with patient partners to enhance health messaging content. Our recommendations draw upon guiding principles and lessons learned from patient engagement literature [18,41,42], co-design approaches [43-45], and the field of human-centered AI [46]. We provide examples based on our messaging approach for patients with chronic pain, which has considerations for other patient populations and emerging AI-based messaging. By reflecting on this process, we hope these recommendations can be used to enhance the comprehension and clarity of patient-facing messages to improve uptake of evidence-based treatments.

We sought feedback on the messages to ensure that the content reflected appropriate framing, language, and accessible modalities for patients to learn about evidence-based pain treatments. At an early stage, we found it valuable to gather input on the clarity, readability, relevance, and tone of the messages. For instance, some patients preferred messages with a warmer tone, such as “your care team is thinking of you and wants to work with you,” whereas others recommended using language that grabs attention by providing the success rates of the treatment for alleviating pain. Patients also generally agreed that messages should be available through multiple modalities, including SMS text messaging, email, and patient portal, which have important early design considerations for the structure, brevity, character limit, subject line, and timing of messages.

For AI-based messaging, little work has been done to involve patients in their content or framing, but early engagement can potentially mitigate known AI concerns related to privacy, fairness, and transparency. A 2023 national survey of US adults found that most individuals have low trust in health systems to responsibly use AI [47]. This is to be expected with novel and unfamiliar technologies, as most US adults have a limited understanding of AI and are largely unaware of common uses of AI [48]. Prior work eliciting input from patients on AI health app development found that educating patients on AI foundations is vital to facilitate meaningful engagement in technology designs [49]. Any engagement board designing AI-related messages should receive training on AI.

We incorporated end users’ feedback throughout the message refinement process, emphasizing diversity across an array of characteristics throughout the engagement boards. Sociodemographic factors as well as education, rurality, clinical needs, and technology aspects (eg, digital literacy, accessibility, and manual dexterity) may affect the use of messaging interventions. For clinical needs, in addition to meeting with patients with chronic pain, we sought input from patients with histories of opioid or other substance use disorders. We found that these patients’ perspectives were especially helpful for crafting messages that would resonate with patients with chronic pain who were prescribed opioids or other patients with opioid use disorder. To promote digital inclusion and avoid widening the digital divide, thoughtful attention is needed to make engagement board participation as easy and appealing as possible to patients who may be less familiar with using technology or the research process. For instance, research teams should think about their work in a simplistic way and be able to translate it to lay terms when meeting with engagement boards, opening the ability for everyone to participate.

Diverse patient perspectives are especially valuable for AI-based messaging approaches, given the well-known algorithmic biases that lead to different outputs across patient populations [50]. This often results in fewer benefits for specific individuals or groups. For example, a study examining LLMs found that the models can recommend different medical treatments for the same condition based on a patient’s sociodemographic characteristics, particularly for race, housing status, and gender [51]. For the design of messages intended to promote uptake of evidence-based treatments, having appropriate representation is needed to avoid perpetuating existing biases that might deter some patients from trying a new treatment.

It is essential to recognize that each patient has a unique experience and background and that their contributions are important for designing messaging interventions. Based on these distinct experiences, expectations should be set with engagement boards on how their feedback will be prioritized into messaging designs, such as on a certain criterion like alignment with theory, practicality, or accessibility. While it was important to have a final set of messages that reflected the rich feedback we received, we also wanted the final message set to align with SDT, given that interventions informed by this theory have been shown to have positive effects on health behavior change [9]. Other researchers and design teams will need to decide whether patient feedback supersedes theoretical alignment in their unique studies. At a minimum, teams should be intentional about how they are going to handle patient input when it does not align with the selected theory (see recommendation 4).

Researchers and design teams should also consider how curated messages that incorporate patient input and theory can serve as training data for AI-based messaging approaches. As this work focused on developing messages for 3 types of nonpharmacological pain treatments, future work could explore how generative AI could leverage our existing corpus to expand the messages to other evidence-based treatments for chronic pain or other conditions. Integrating patient input and feedback on such a hybrid set of messages would be a key step in evaluating the feasibility of this approach.

For certain messages, we prioritized feedback that most closely aligned with MBCTs due to the empirical evidence of interventions based on SDT in altering health behaviors. We also prioritized feedback from the board whose members most closely resembled the patients for whom the intervention was designed (ie, patients with chronic pain). We received diverse feedback across the 3 engagement boards and had to make systematic decisions to finalize the messages. In an ideal world, we would design a system that could generate unique messages based on distinct patient characteristics, such as clinical, social, and technical factors. Some patients mentioned how rather than the message displaying “the treatment can help you do more of what you would like to do,” it should be personalized based on their unique goals, such as “the treatment can help you spend more time with your friends and family.”

With the advancement of LLMs, such data have the potential to be extracted from electronic health records and clinical notes to facilitate the personalization of interventions. However, a recent systematic review found that only 5% of studies are using LLMs with real patient data [52]. It is evident that these emerging tools warrant further study to understand patients’ perceptions and the implications of integration with their health data. This would also involve understanding patients’ concerns about using their data to train an algorithm, such as handling sensitive health information, unintended consequences, or ethical issues.

Gathering iterative input and continual feedback from engagement boards can build bridges for future collaborative work with engagement boards [53]. Design teams should be flexible and open to suggestions for improvement on each iteration. For instance, we had the opportunity to have a follow-up meeting with 2 of the engagement boards. During this time, we presented the final messages and prototypes based partially on their feedback. We described how their insights were helpful for creating the messages and prototypes and to ensure they would resonate with patients with similar experiences. Patients on each board were positive about the changes we made and felt encouraged that their input and ideas were included. They also offered further suggestions to improve digital communication, outreach through caregivers, and health tools geared toward patients in recovery.

Prior work proposed a framework for integrating continual perspectives into AI language tools using a community-based participatory research approach [54], referred to as “community-based natural language processing.” The framework outlines ways to involve communities in developing AI health tools spanning the entire development pipeline, ranging from data curation to model deployment and validation. Such approaches could be used to routinely engage with patients over time to foster collaborative, meaningful AI messaging content.

In addition to the message content, several patients emphasized the importance of a human component in messages they receive. This is aligned with the SDT construct of relatedness, as it pertains to feeling understood and cared for by others. Moreover, patients felt that sometimes messages can lead to frustration if they are not interactive. This finding is concordant with the computers are social actors theorem, which describes how people communicate with computers as if they are people [55]. Prior work on this topic has also shown that frustrating interactions with computers can lead to strong negative emotions toward the system and its developers [56]. Some patients recommended that it would be more beneficial to use AI to answer certain types of questions rather than the system resending information. At a minimum, it was suggested to provide a way to get support from another person, including a phone number. This may be especially important for some groups of patients, such as those with opioid use disorder, who may need more frequent interactions with their care team.

Ensuring patient-care team relationships are not harmed by AI tools is critical, as there are concerns about the loss of the human element from these new technologies [48]. The human-centered AI field recently emerged to embrace a human-centered perspective on building AI tools to better serve human needs [46]. This approach builds on typical design methods of stakeholder engagement, user observation, testing, and iterative refinement for systems that use AI. Actively involving patients throughout the design and development process of AI tools emphasizes using AI to improve communication and empower patients and care teams, rather than replacing human interactions. Future efforts should examine effective combinations of hybrid AI-human messaging interventions, including when it is suitable for a human versus AI to respond to certain questions and when it is appropriate to connect the patient to the care team.