Background: A critical unmet need for underserved patients with diabetes is regular access to sufficient support for diabetes self-management. Although advances in digital technologies have made way for eHealth applications that provide a scalable path for tailored interventions for self-management of chronic conditions, health and digital literacy has remained an obstacle to leveraging these technologies for effective diabetes self-management education. Studies have shown that the availability of coaches helps to maintain engagement in internet-based studies and improves self-efficacy for behavior change. However, little is known about the substances involved in these interactions.
Objective: This study aims to compare the content of conversations between patient–coach pairs that achieved their self-management goals and those that did not. The context is a clinical implementation study of diabetes self-management behavior change using Health360x within the practices of the Morehouse Choice Accountable Care Organization in the Atlanta metro area. Health360x is a coach-assisted consumer health information technology designed to support self-management skills acquisition and behavior among underserved, high-risk patients with diabetes.
Methods: We provide a novel analysis of the discursive emphasis on patients and coaches. We examined transcripts of visits using a structural topic model to estimate topic content and prevalence as a function of patient and coach characteristics. We compared topics between patient–coach pairs that achieved diabetes-related self-management goals and those who did not. We also estimated a regression in which utterances are the units, the dependent variable is the proportion of an utterance that is about a given topic, and the independent variables are speaker types and explored other themes.
Results: Transcripts from 50 patients who were recruited and consented, starting in February 2015, were analyzed. A total of 44 topics were estimated for patient–coach pairs that achieved their intended health goals and 50 topics for those who did not. Analysis of the structural topic model results indicated that coaches in patient–coach pairs that were able to achieve self-management goals provided more contextual feedback and probed into patients’ experience with technology and trust in consumer information technologies. We also found that discussions around problem areas and stress, support (βCoach=.015; P<.001), initial visits (βCoach=.02; P<.001), problems with technology (βCoach=.01; P<.001), health eating goals (βCoach=.01; P=.04), diabetes knowledge (βCoach=.02; P<.001), managing blood sugar (βCoach=.03; P<.001), and using Health360x (βCoach=.003; P=.03) were dominated by coaches.
Conclusions: Coach-facilitated, technology-based diabetes self-management education can help underserved patients with diabetes. Our use of topic modeling in this application sheds light on the actual dynamics in conversations between patients and coaches. Knowledge of the key elements for successful coach–patient interactions based on the analysis of transcripts could be applied to understanding everyday patient–provider encounters, given the recent paradigm shift around the use of telehealth.
e-Patients are a relatively new breed of informed health consumers comprising 61% of all adults in the United States using technology to equip, enable, empower, and engage with relevant health-related information . Despite increasing access to wireless and mobile technology among minority and vulnerable populations, there is still a higher trend in the adoption of digital health technologies among disproportionately advantaged groups, such as young, White males [ - ]. It is known and well-accepted in the disparities research community that health and digital literacy are barriers to the increased use of eHealth technologies among minority and vulnerable populations. These obstacles include lack of perceived value, technologies creating more work, materials not in appropriate reading levels to make informed health decisions, lack of cultural relevance (ie, accessing sources in their respective language), and privacy and trust concerns [ ]. These barriers are a function of individual characteristics, including socioeconomic status, education, and less access to change agents, among other factors that are associated with higher rates of attrition in internet-based studies [ - ]. Higher rates of attrition are even more noticeable among minority and underserved populations, who are also disproportionately challenged by intervention factors, including usability problems, complexity, and ease of discontinuance [ ]. The net result is a lack of diffusion of innovations among vulnerable populations and a worsening of disparities in health outcomes [ ], that is, the uneven adoption of digital health innovations by minority and other vulnerable populations because of health and digital literacy exacerbates existing disparities in health.
An area where there is an agreement in the literature is that health and digital literacy are obstacles to the use of eHealth technologies for implementing tailored health interventions that support self-management of chronic conditions, particularly diabetes . A typical example for diabetes is the challenge with the delivery of appropriate and effective diabetes self-management education (DSME) to vulnerable populations. Multiple studies have found that diabetes DSME is associated with improved diabetes knowledge and self-care behavior [ ], improved clinical outcomes such as lower hemoglobin A1c [ ], lower self-reported weight [ , ], improved quality of life [ ], healthy coping [ , ], and lower costs [ ]. Diabetes education is associated with increased use of primary and preventive services and lower use of acute, inpatient hospital services [ ], and patients who participate in DSME are more likely to follow best practice treatment recommendations, particularly among the Medicare population. A critical issue and unmet need for underserved patients living with diabetes is the regular access to sufficient support for effective diabetes self-management. Better outcomes were reported for DSME interventions that were longer and included follow-up support, that were culturally appropriate and age appropriate, and that were tailored to the individual psychosocial needs and behavioral preferences [ , ].
Prior studies indicate that the availability of a trusted partner (coach) helps to improve user experience with health technology for DSME, including perceived relevance and ease of use , among vulnerable populations who face health and digital literacy barriers and higher rates of attrition. The support of a coach improves engagement with technology, and accountability improves adherence to and self-efficacy for behavior change [ ]. In this patient–coach relationship, coaches are not intermediaries between patients and health information but serve as an apomediary, that is, a guide to relevant information and services for the patient [ ]. Much has been theorized about what leads to successful patient–coach relationships; however, what is less understood is the content of patient–coach interactions.
In this paper, we aim to evaluate the discursive emphasis on patients and coaches engaged in DSME. To do so, we will leverage self-reported patient data and >100 hours of recorded transcripts from a clinical implementation study of Health360x, a coach-assisted consumer health information technology (CHIT) designed to support diabetes self-management skills acquisition and behavior for underserved, high-risk patients living with diabetes . The platform is available as a web application and mobile app and includes functionality for improving health literacy and self-efficacy through access to a health coach, a social network of peers, a curriculum of DSME materials, and a health tracker that can record blood pressure, BMI, physical activity, and self-management goals.
In what follows, we provide some background on the study from which the data were generated. We then provide a novel analysis of the discursive emphasis on patients and coaches from transcripts of visits using a structural topic model (STM) to estimate topic content (ie, characteristic words in a topic) and prevalence (ie, proportion of an utterance that is related to a topic) as a function of patient and coach characteristics. We conclude with a discussion of our results, limitations, and directions for future work on the use of coach-assisted CHIT for DSME.
Transdisciplinary Collaborative Center Project
Longitudinal data and transcripts of patient–coach interactions were taken from a clinical implementation study of Health360x, a coach-assisted CHIT designed to assist with chronic illness care through behavior change [, ]. It is important to note that as an implementation study, there was no intervention or control arm as the goal of the study is not to make generalizations to a broader population [ ]. The research design of the study focused almost exclusively on external validity. As such, countless hours were spent working with clinical practices through focus groups and plan-do-study-act cycles to ensure clinical readiness. The goal of the transdisciplinary collaborative center was to study the implementation across several clinics that are part of the Morehouse Choice Accountable Care Organization (MCACO).
Health360x was developed at the Morehouse School of Medicine and evaluated within the MCACO. MCACO is a physician-led integrated delivery model participating in the Medicare Shared Savings Program offered by the Centers for Medicare and Medicaid Services Innovation Center . The MCACO partner practices care for a disproportionate share of high-need, complex populations and endure extraordinary challenges in managing the use, with comparatively limited resources. The Health360x implementation at MCACO was nested within the MCACO’s Centralized Care Coordination model. This synergistic approach represents a health system where an African American majority patient population spends time and receives primary health care.
To determine health practice readiness for inclusion in the study, key individuals within the practice were identified for a focus group. For a clinic to participate, their front office manager, practice manager, lead physician, and (optionally) patients living with diabetes were required to participate in this focus group. The goal of this focus group was to delineate patient experiences and practice workflow for patients living with diabetes. Findings from focus group discussions were then used to identify and address issues around recruitment, signup, and visits through a plan-do-study-act cycle from select practices. In total, we recruited between 3 and 5 individuals across 5 sites for a total of 25 individuals for these focus groups.
Health coaches from within and outside of clinical practices were also recruited. The preferred characteristics included the following: (1) certified by Americans With Disabilities Act with the tool Fundamentals of Diabetes Care; (2) previous experience with living with diabetes either directly or through providing care for a family member or patient; (3) health care professionals with previous experience in educating patients living with diabetes; and (4) soft skills, including being considerate, responsible, dependable, and understanding within their community. A total of 4 health coaches were recruited by participating in health practices. This included 3 foreign-trained physicians, including 1 man in his late 30s whose second language was English, and 2 female physicians in their late 30s who were native English speakers. The final coach was a licensed practical nurse in her late 50s. All coaches identified as Black, with 50% (2/4) of coaches identifying as African and the others as African American.
Health coaches interacted with a slate of patients identified through the MCACO practices and established communication with the practice to facilitate patient self-management support. Approximately 20-30 patients across 4 practices were selected, obtained consent from, and trained on the use of Health360x to report outcomes of self-efficacy; overall health, including blood pressure, blood glucose, exercise, sleep, and quality of life; and satisfaction with care. A total of 200 patients were screened for being at high risk of living with diabetes. A total of 100 patients were excluded as they did not meet the eligibility criteria for the study or were part of a clinical site that was ultimately deemed not practice ready, leaving 100 patients to be allocated the intervention.
This study was approved by the Morehouse School of Medicine Institutional Review Board (approval number 674).
Included in the study were adults living with diabetes at high risk of complications and mortality, as defined by the presence of obesity or overweight status (BMI>25 kg/m2); tobacco use; history of depression; systolic blood pressure >140 mm Hg and diastolic blood pressure >90 mm Hg; hemoglobin A1c >7%; recent hospitalization or emergency room visit for uncontrolled diabetes or hypoglycemia; history of renal disease defined as estimated glomerular filtration rate <60 mg/mmol; and history of heart attack, angina, claudication, or cerebrovascular disease.
Health coaches were required to complete 4 sessions of training. These sessions included training on diabetes knowledge, diabetes management and prevention, cultural competencies of the target population, working with low-literacy populations, patient recruitment and retention, confidentiality, and familiarity with data collection tools. Coaches were also required to complete web training and certification by the American Association of Diabetes Educators titled Fundamentals of Diabetes Care. Finally, the coach training sessions also included hands-on training on Health360x.
A total of 6 hours of training sessions were required to be completed by patients before the start of participation. The training was conducted at the Morehouse School of Medicine and covered the role of health coaches as navigators; the use of Health360x, glucose meter, blood pressure monitor, and pedometer; and process for downloading data. Patients with access to a computer or personal smartphone compatible with Health360x were instructed on how to access or download the app. Those patients who did not have access to a computer or personal smartphone that is compatible with the mobile app relied on a kiosk at the practice that was accessible when they visited their coach. The Health360x application access and training were completed through a combination of YouTube videos, and a handbook detailing steps for using the application was also provided to patients.
Patient and Coach Visits
The health coach met with patients at the practice during scheduled visits. The purpose of each visit was to advance the patients’ self-efficacy for self-management of their diabetes. During the initial visit, patients were asked to select individual diabetes-related self-management health goals around being active, healthy coping and eating, monitoring, problem solving, reducing risky behaviors, and taking medication. At each subsequent visit, coaches ascertained the barriers the patient experienced through the use of motivational interviewing. Once these barriers were identified, incremental steps to address them, which the patient suggested or agreed with, were made. Techniques such as behavioral contracting were incorporated into the process. All interactions between patients and coaches were between 20 and 40 minutes. All meetings were digitally recorded and transcribed using a third-party transcription service. As a rule, initial schedules for interactions were every 2 weeks for the first 2 months and then monthly for 4 months.
Primary Outcome in Transdisciplinary Collaborative Center Study
The primary outcome of interest in the study was self-efficacy for behavior change, which is a predictor of intention and behavior. Self-efficacy was measured using a self-reported 10-point General Self-Efficacy Scale at each follow-up visit. A patient was determined to have achieved their self-efficacy goals if they self-reported a score ≥7.
Our sample size calculation was based on a systematic review and meta-analysis that included 85 high-quality published studies of internet-based behavioral health interventions with a total sample size of 43,236 patients . On average, interventions with more extensive use of theory were associated with increases in effect size (P=.049). Interventions based on the theory of planned behavior tended to have substantial effects on behavior (Cohen d=0.36, 95% CI 0.15-0.56). Health360x was designed with particular attention to the incorporation of theory-driven behavioral change techniques such as an enriched information environment (current diabetes knowledge and curriculum), self-monitoring with color-coded feedback, goal setting, identifying barriers, and problem solving. Our a priori power analysis indicated that a minimum sample size of 97 was required to detect an anticipated effect size of 0.36, assuming a desired power and significance level of 0.8 and 0.05, respectively. Therefore, we recruited 100 patients for this study.
To explore the substance of conversations between patients and coaches, we used a topic model. This statistical machine learning method identifies common themes in a corpus of documents, or in our case transcripts, and is based on the intuition that particular words are associated with particular topics. Formally, each utterance in a conversation is represented as a vocabulary multiset with corresponding word counts (also referred to as a document-term matrix). Each word in this vocabulary has some associated probability that it belongs to a given topic. If we think of a conversation as representing a random mixture of topics that are in turn defined by a set of characteristic, high-probability words, we can infer latent topics.
Topic modeling has been shown to reliably uncover conversational topics from transcripts between patients and providers in clinical settings [, ]. provides a visual example of how this works from a snippet of the conversation between a patient and a health coach. Characteristic words are highlighted in yellow, green, and blue and correspond to eating, anxiety, and finances, respectively. Taken together, we can see that this conversation is about anxiety and stress, leading to overeating. However, the snippet can be further broken down, and a topic model estimates the proportion of the conversation that corresponds to each of these topics based on the observed vocabulary.
We use an STM  for 2 main reasons: first, the STM extends traditional probabilistic models such as the Blei et al [ ] latent Dirichlet allocation and the Blei and Lafferty [ ] Correlated Topic model to incorporate document metadata into the analysis of latent topics. This was done by defining a generative process that produces documents, that is, utterances, with associated data, given document-topic and topic-word distributions, and using a variational optimization algorithm to estimate topic model parameters. Each conversation and utterance was made up of different proportions of the estimated topics. Incorporating speaker metadata into our topic modeling allowed us to explicitly model the association among patients, coaches, and topics of conversation, that is, the way the characteristics of patients and coaches were associated with the proportion of a specific topic in a given utterance. In our topic model, we included the speaker’s age, gender, and race and ethnicity, as well as that of the person they are speaking with and whether the speaker was a coach. We estimated a regression in which utterances were the units, the dependent variable was the proportion of an utterance that is about a given topic, and the independent variables were speaker types.
The second reason we used the STM relates to what is a common subjective decision made when using topic models: the number of topics to estimate. This decision was generally arbitrary. To avoid any human-injected bias in our analysis, we leveraged a data-driven approach to select the number of topics based on the identification of anchor words that exist in a document only if the document is about a specific topic. The Mimno and Lee  algorithm automatically selects distinctive and probable anchor words from which a number of topics can be reliably estimated. As the authors of the STM indicate in their documentation of their software in R (R Foundation for Statistical Computing), the primary advantage of this approach is that the number of topics to be estimated is automatically selected by the algorithm.
Patient Sample Characteristics
A total of 100 high-risk patients with diabetes were recruited, and they consented to participate in this study from February 2015 over 5 years. Of the 100 patients, 18 (18%) patients were lost to follow-up before starting any coaching sessions, leaving 82 (82%) patients with at least 1 and as many as 11 follow-up visits. A total of 195 patient–coach follow-up visits were completed throughout the study, with an average of 2.35 (SD 1.98) follow-up visits per patient and 4.79 (SD 5.77) goals per patient. Of the 82 patients, 3 (4%) patients specified 1 diabetes-related self-management health goals, 18 (22%) patients specified 2 goals, 15 (18%) patients specified 3 goals, 8 (10%) patients specified 5 goals, 4 (5%) patients specified 7 goals, 1 (1%) patient specified 8 goals and another 15 goals, and 1 (1%) remaining patient specified 24 goals. Of the 82 patients who participated in the study, transcripts were analyzed for 51 (62%) patients. Digital recordings for the remaining 38% (31/82) of patients were of bad quality and, therefore, were not uploaded or corrupted during transcription. Most patients were Black and African American (45/51, 88%) and between the ages of 45 and 65 years (40/51, 78%).provides a complete breakdown of patient demographic characteristics. provides a sample of diabetes-related self-management goals.
|Characteristics||Values, n (%)|
|Age range (years)|
|No response||2 (4)|
|Race and ethnic background|
|Asian or Pacific Islander||1 (2)|
|Black or African American||45 (88)|
|Hispanic or Latino||1 (2)|
|Indian or Alaskan Native||1 (2)|
|No response||1 (2)|
|No insurance||17 (33)|
|Dual eligible||5 (10)|
|No response||2 (4)|
Frequency of the number of health subgoals.
- Exercise longer
- Cope with diabetes
- Get support from my medical team
- Get support from my family and friends
- Get preventive help
- Stop smoking
- Check my feet
- Lose weight
- Get blood pressure under control
- Learn to have a safe pregnancy
- Follow my eating schedule better
- Eat better food
- Overeat less often
- Check my blood sugar more often
- Miss fewer blood sugar checks
- Do my blood sugar checks on time more often
- Prevent high blood sugars
- Treat high blood sugars
- Prevent low blood sugars
- Treat low blood sugars
- Manage diabetes when sick
- Miss fewer medications
Structural Topic Model
In total, our corpus included >17,000 talk turns, that is, an utterance or spoken statement in a conversation. To explore differences between topics of conversation between patient–coach pairs that achieved their health goals and those that did not, we split the corpus into successful patient–coach pairs (achieved) and patient–coach pairs that did not achieve the intended health goals (not achieved). This left us with 7196 talk turns in the achieved corpus and 9644 talk turns in the not achieved corpus. We then preprocessed each corpus using the textProcessor function in the STM R package to remove punctuation, stop words, numbers, and stemming. This produced a final document-term matrix with 6737 talk turns and a vocabulary of 1444 terms for the achieved group and 8663 talk turns and 1952 terms for the not achieved group. Patients represented more talk turns, that is, spoke more, in both the achieved and the not achieved groups, 60% (4318/7196) and 63% (6076/9644), respectively.
A total of 44 topics were estimated for patient–coach pairs that achieved their intended health goals and 50 topics for those who did not. In line with common practice, our approach to assigning labels to topics began by reviewing topic model outputs to identify potential topics based on the coherence of characteristic word forms. We then reviewed the high-scoring conversations for each topic to determine the topic labels. Where there was a clear topical theme in the first N talk turns, we validated the topic label. In general, a clear theme emerged within the first 30 talk turns. A review of high-scoring talk turns was completed collaboratively by 4 reviewers until a consensus on topic labels was reached. Of the 44 topics estimated for the achieved group, 13 (29%) coherent topics emerged, as presented in. provides the results for the second STM of the not achieved group, which produced 10 coherent topics.
Structural topic model 1 for patient–coach pairs that achieved health goals.
Topic 3: healthy eating
Topic 5: monitoring blood pressure and blood sugar
Topic 6: healthy coping
Topic 13: use of consumer health information technology
Topic 18: healthy snacking
Topic 22: goal setting and habits
Topic 25: Health Insurance Portability and Accountability Act and diabetes knowledge (initial visit)
Topic 26: sleep
- Did not
Topic 30: monitoring for diabetics
Topic 31: follow-up visit
- You will
Topic 38: diabetes knowledge test
Topic 43: health coping and strategies (coaching)
Topic 44: goal assessment and progress (coaching)
- Were not
Structural topic model for patient–coach pairs that did not achieve health goals.
Topic 1: medication
- Have not
- Does not
Topic 2: problem areas, and stress and support
Topic 9: Health Insurance Portability and Accountability Act and diabetes knowledge
Topic 14: problems with technology
Topic 15: healthy eating goals
Topic 20: diabetes knowledge test
Topic 21: social determinants
Topic 22: sleep
Topic 44: managing blood sugar (coaching)
Topic 49: using Health360x
Coaching and Contextual Feedback
We found quite a bit of overlap between topics of discussion between the 2 groups (eg, healthy eating, diabetes knowledge, and sleep). However, we found more coherent, that is, meaningful, topics that capture coaching in the achieved group versus in the not achieved group. Further investigation of high-scoring talk turns and the broader conversation that they were a part of provides many examples of coaches providing contextual feedback and incorporating this feedback into the patient’s strategy.provides 2 examples of such conversations. In the first example, which is taken from a high-scoring conversation on healthy coping and strategies, we see that after the coach first probes the source of anxiety and stress that is leading to overeating, acknowledges the patient’s feelings, and contextualizes how their anxiety and stress lead to overeating, only then does the coach start discussing a plan to reduce overeating by focusing on the identified stressors. The next sample conversation was around healthy eating. The coach begins the conversation by recentering the patient around the goal of getting their blood sugar down through exercise and healthier eating habits. The conversation quickly zeroes in a specific challenge in implementing this strategy: not eating on a regular schedule as the patient was not hungry at regular mealtimes. Only then does the coach provide contextualized feedback around what, when, and how much to eat.
User Experiences With CHIT
Another clear theme in our analysis of the topic model results was conversations around user experiences with CHIT. We found that CHIT use in the not achieved group focused almost exclusively on technical issues and how to use Health360x.provides 2 examples from the not achieved group. In the first example, the coach expresses frustration with technical issues that require troubleshooting Health360x during a demonstration with a patient. The second example is from a conversation in which the patient and coach work together to reset the password (a common theme throughout the study was remembering passwords) and then updating the patient’s health record on Health360x.
Conversely, the topic on the use of CHIT in the achieved group contained multiple examples of coaches probing user experiences with technology.provides 3 examples of coaches asking probing questions on CHIT use in the achieved group. These questions ranged from familiarity with technology and use of the internet to how trusting the patients are of consumer information technologies, including sharing their health and other information on those tools. What we found to be particularly interesting is that we did not find a topic in the achieved group that resembled issues with using Health360. This suggests that understanding a patient’s experiences with technology can lead to better coaching around the use of technology. This, in turn, can lead to better engagement with CHIT, self-management behavior changes, and health outcomes.
Who Dominates Topics of Conversation: Patients or Coaches?
In the achieved group, we found that healthy eating was much more likely to be discussed by patients. The average proportion of conversations about healthy eating across all patient and coach interactions, that is, E (topic prevalence), was 0.026 or 2.6%. Being a coach was associated with an increase of 4.9% (βCoach=.049; P=.02) in the healthy eating topic proportion of a given utterance. The average proportion of conversations around follow-up visits and coaching in the form of goal assessment and progress across all visits was 3.4% and 0.8%, respectively. Being a coach was associated with an increase of 3% (βCoach=.03; P=.007) and 0.3% (βCoach=.003; P=.01) in the proportion of utterances related to follow-up visits and coaching in the form of goal assessment and progress. All other coherent topics in the achieved group were equally as likely to be discussed by either coaches or patients.
Conversations in the not achieved group were dominated by coaches. This included a 1.5% increase in the proportion of an utterance on problem areas, and stress and support (βCoach=.015; P<.001; E [topic prevalence]=0.012), a 2.4% increase related to initial visits (βCoach=.024; P<.001; E [topic prevalence]=0.012), a 1.3% increase around problems with technology (βCoach=.013; P<.001; E [topic prevalence]=0.012), a 0.8% increase in health eating goals (βCoach=.008; P=.04; E [topic prevalence]=0.03), a 1.7% increase in the proportion of diabetes knowledge (βCoach=.017; P<.001; E [topic prevalence]=0.02), a 3% increase in speaking about managing blood sugar (βCoach=.03; P<.001; E [topic prevalence]=0.02), and a 0.3% increase in speaking about using Health360x (βCoach=.003; P=.03; E [topic prevalence]=0.03). A total of 2 exceptions were the proportion of an utterance on medication (βCoach=−0.008; P=.009; E [topic prevalence]=0.02) and social determinants (βCoach=−0.02; P<.001; E [topic prevalence]=0.012), which were positively associated with patients.
and present the mean difference in topic proportions as a function of whether the speaker was a coach or a patient, that is, which topics were more likely to be discussed by coaches versus patients. Also included in the plots are CIs indicating a statistically significant difference between the 2 groups. Although we found quite a bit of overlap between topics of discussion between the 2 groups, plotting the mean difference in topic proportions across all coherent topics indicated that conversations in the achieved group were equally distributed between patients and coaches. However, topics of conversation between patients and coaches in the not achieved group did not include 0 in their 95% CIs, indicating that these topics were dominated, and therefore more likely to be driven, by coaches.
In this paper, we present the results of a novel analysis of the discursive emphasis on patients and coaches. The dialog is derived from an implementation of Health360x, a coach-assisted CHIT designed to support diabetes self-management skills acquisition and behavior for underserved, high-risk diabetic patients. Thematic analysis of conversations clustered using STM results of patient–coach interactions indicates that in patient–coach pairs that were able to achieve self-management goals, coaches provided more contextual feedback and probed into patients’ familiarity with technology and use and trust of consumer information technologies. We also found that topics of conversation between successful patient–coach pairs were equally distributed, whereas discussions with patients who were unable to achieve enhanced self-efficacy for diabetes-related self-management were dominated by coaches.
There are a few limitations of the analysis presented in this paper. First is the small corpus of transcribed conversations. As with all machine learning algorithms, the more training data, the better the models perform. Typically, topic models are used on corpora with tens of thousands of talk turns; however, we analyzed significantly less (approximately 17,000) talk turns. With that said, the methodological novelty of this paper is the incorporation of covariates into our estimation of topic models using the STM. Although we had a smaller than expected corpus, we found that we could reliably identify latent topics in comparatively smaller corpora by incorporating metadata into topic modeling. Transcription of the sessions was also outsourced and included errors, which may have further limited the overall information content of the transcripts.
A second potential limitation is that there are subjective decisions that go into creating a topic model that can influence topic outputs. The number of topics estimated as well as decisions around preprocessing (eg, removing stop words, stemming, and lemmatization) can influence topic model coherence and stability [- ]. We addressed the subjectivity of arbitrarily selecting a number of topics by automating the number of topics to be estimated; however, another approach is to estimate and compare topic models with different topic sizes for stability across models. Finally, evaluations of preprocessing on topic model outputs are needed within the context of the STM used in this paper.
Another limitation relates to the lack of power in our analysis. The sample size calculation of 97 participants for this study was based on an effect size with a P value that is effectively .05. Moreover, our analysis of discursive analysis included transcripts from only half of the estimated sample size. To assess whether patient characteristics influence retention and transcript quality, we ran a logistic model regressing whether we had at least one session recording of high enough quality for analysis of patient characteristics. The results are presented in. Patient characteristics, including gender, age, insurance status, and willingness to use the internet, were not significantly different between patients whose session recordings were retained and those whose session recordings were not.
A final limitation was related to the training of coaches, which was naturally limited. Coaches did not receive extensive training in cognitive behavioral therapy or motivational interviewing. Our training built the coaches’ existing skills and provided additional tools to help with an appreciative inquiry. In the future, we will use these learnings to provide more uniform training for health coaches.
Comparison With Prior Work
A need exists to better understand how DSME might be delivered within primary care, the outcomes that can be achieved, and the training and system changes needed as a result . Effective face-to-face multicomponent interventions require skilled primary care workers and activated patients coupled with tailored training and self-help systems, resources, and materials [ ]. These interventions for diabetes self-management are complex, and their effectiveness is in large part determined by dynamics within the patient–provider relationship [ , ]. Where these relationships are 1-sided, that is, focus on what the patient should be doing, and do not account for a patient’s circumstance, strategies for encouraging diabetes-related self-management behaviors will be ineffective [ ].
Patients stand the best chance of developing skills to actively participate and take responsibility in the management of their chronic condition when they receive personalized coaching. Polonsky and Fisher  provided a conceptual model of how the frequency of feedback, personal meaningfulness, clarity, guidance and support, and patient characteristics influence diabetes-related feedback and affect patient engagement and behavior change. The way each of these elements should be personalized for diabetes-related coaching to be effective is beyond the scope of this comparison with prior work. What is relevant is that our analysis of the discursive emphasis on patients and coaches provides a novel approach for quantifying the substance of these relationships. Examples of coaches providing contextual feedback, probing patient experiences with technology, and engaging in back-and-forth conversations are all indicators of tailored coaching. However, what remains less clear is how we can relate these conversations to clinical outcomes and how we can quantify the additional benefit that technology provides in addition to tailored coaching. These 2 questions are the focus of subsequent projects and manuscripts.
Our study also contributes to prior work by providing new and important insights into the usability and effectiveness of mobile health interventions targeting older African American patients living with diabetes. This underserved population has traditionally been considered not amenable to the use of technology to manage their health . In addition, we present novel methods that can be used to understand (and predict) what is happening in patient–coach interactions. These methods enable the analysis of themes in text at scale using techniques that minimize the amount of human-injected bias into qualitative studies.
Coach-facilitated, technology-based DSME can help address critical issues and unmet needs for underserved patients with diabetes around accessing sufficient support regularly. A variety of factors influence the efficacy of coaching; however, cultural expectations must be further explored, given the growing diversity among patients and the health care workforce prevalent today. Our use of structural topic modeling in this application is novel, and it creates an opportunity to introduce this technique into everyday patient–provider encounters, particularly in a post–COVID-19 world in which there has been a paradigm shift around the use of telehealth. The opportunity to create outputs that guide further physician action and patient action could drive better patient engagement and overall patient health outcomes.
PP and E Ofili conceptualized the project. PP and MYI contributed to the development of the project. E Olorundare, MM, and MB led the investigation. EAM conducted the data curation. MYI conducted the formal analysis. PP and E Ofili led the investigation. MYI contributed to writing the manuscript and preparing the original draft. PP and E Ofili completed the finding acquisition. All authors have read and agreed to the published version of the manuscript.
Conflicts of Interest
E Ofili is the Founder and Chief Science Officer for AccuHealth Technologies, a technology startup which holds the patent for the Health360x platform. AccuHealth Technologies was not involved in the design and analysis of the study.
Example of contextual feedback in health coaching around health coping and strategies.DOCX File , 23 KB
Example of consumer health information technology problems and use in the not achieved subgroup.DOCX File , 20 KB
Example of probing questions around consumer health information technology use in the achieved subgroup.DOCX File , 20 KB
Logistic regression of at least one recording on patient characteristics.DOCX File , 13 KB
- The social life of health information. Pew Internet & American Life Project. 2009. URL: https://web.archive.org/web/20091224000545/http://www.pewinternet.org/~/media//Files/Reports/2009/PIP_Health_2009.pdf [accessed 2022-02-03]
- Health topics: 80% of internet users look for health information online. The Communication Initiative Network. URL: https://www.comminit.com/content/health-topics-80-internet-users-look-health-information-online [accessed 2022-02-03]
- Mobile Health 2010. Pew Research Center. URL: https://www.pewresearch.org/internet/2010/10/19/mobile-health-2010/ [accessed 2022-02-03]
- Christopher Gibbons M. Use of health information technology among racial and ethnic underserved communities. Perspect Health Inf Manag 2011 Jan 01;8:1f [FREE Full text] [Medline]
- Institute of Medicine, Board on Neuroscience and Behavioral Health, Committee on Health Literacy. In: Nielsen-Bohlman L, Panzer AM, Kindig DA, editors. Health Literacy: A Prescription to End Confusion. Washington, DC: The National Academies Press; 2004:41-42.
- Jensen J, King A, Davis L, Guntzviller L. Utilization of internet technology by low-income adults: the role of health literacy, health numeracy, and computer assistance. J Aging Health 2010 Sep;22(6):804-826. [CrossRef] [Medline]
- Sarkar U, Karter A, Liu J, Adler N, Nguyen R, Lopez A, et al. The literacy divide: health literacy and the use of an internet-based patient portal in an integrated health system-results from the diabetes study of northern California (DISTANCE). J Health Commun 2010;15 Suppl 2:183-196 [FREE Full text] [CrossRef] [Medline]
- Levine R, Rust G, Pisu M, Agboto V, Baltrus P, Briggs N, et al. Increased Black-White disparities in mortality after the introduction of lifesaving innovations: a possible consequence of US federal laws. Am J Public Health 2010 Nov;100(11):2176-2184 [FREE Full text] [CrossRef] [Medline]
- Toobert D, Strycker L, King D, Barrera M, Osuna D, Glasgow R. Long-term outcomes from a multiple-risk-factor diabetes trial for Latinas: ¡Viva Bien!. Transl Behav Med 2011 Sep;1(3):416-426 [FREE Full text] [CrossRef] [Medline]
- Steinsbekk A, Rygg L, Lisulo M, Rise MB, Fretheim A. Group based diabetes self-management education compared to routine treatment for people with type 2 diabetes mellitus. A systematic review with meta-analysis. BMC Health Serv Res 2012 Jul 23;12:213 [FREE Full text] [CrossRef] [Medline]
- Deakin T, McShane CE, Cade JE, Williams RD. Group based training for self-management strategies in people with type 2 diabetes mellitus. Cochrane Database Syst Rev 2005 Apr 18(2):CD003417. [CrossRef] [Medline]
- Gary TL, Genkinger JM, Guallar E, Peyrot M, Brancati FL. Meta-analysis of randomized educational and behavioral interventions in type 2 diabetes. Diabetes Educ 2003;29(3):488-501. [CrossRef] [Medline]
- Cochran J, Conn VS. Meta-analysis of quality of life outcomes following diabetes self-management training. Diabetes Educ 2008;34(5):815-823 [FREE Full text] [CrossRef] [Medline]
- Sturt J, Whitlock S, Fox C, Hearnshaw H, Farmer A, Wakelin M, et al. Effects of the Diabetes Manual 1:1 structured education in primary care. Diabet Med 2008 Jun;25(6):722-731. [CrossRef] [Medline]
- Duncan I, Birkmeyer C, Coughlin S, Li Q, Sherr D, Boren S. Assessing the value of diabetes education. Diabetes Educ 2009;35(5):752-760. [CrossRef] [Medline]
- Powers M, Bardsley J, Cypress M, Duker P, Funnell M, Fischl AH, et al. Diabetes self-management education and support in type 2 diabetes: a joint position statement of the american diabetes association, the american association of diabetes educators, and the academy of nutrition and dietetics. J Acad Nutr Diet 2015 Aug;115(8):1323-1334. [CrossRef] [Medline]
- Couper M, Alexander G, Zhang N, Little R, Maddy N, Nowak M, et al. Engagement and retention: measuring breadth and depth of participant use of an online intervention. J Med Internet Res 2010 Nov 18;12(4):e52 [FREE Full text] [CrossRef] [Medline]
- Tang T, Funnell M, Oh M. Lasting effects of a 2-year diabetes self-management support intervention: outcomes at 1-year follow-up. Prev Chronic Dis 2012;9:E109 [FREE Full text] [CrossRef] [Medline]
- Pemu P, Josiah Willock R, Alema-Mensa E, Rollins L, Brown M, Saint Clair B, et al. Achieving health equity with e-Healthystrides©: patient perspectives of a consumer health information technology application. Ethn Dis 2019;29(Suppl 2):393-404 [FREE Full text] [CrossRef] [Medline]
- Heisler M, Choi H, Mase R, Long JA, Reeves PJ. Effectiveness of technologically enhanced peer support in improving glycemic management among predominantly African American, low-income adults with diabetes. Diabetes Educ 2019 Jun;45(3):260-271 [FREE Full text] [CrossRef] [Medline]
- Ofili E, Pemu P, Quarshie A, Johnson P, Linn L, Kaur D, et al. System and method for chronic illness care. Science On. URL: https://scienceon.kisti.re.kr/srch/selectPORSrchPatent.do?cn=USP2012078234131 [accessed 2022-02-03]
- Brown CH, Curran G, Palinkas LA, Aarons GA, Wells KB, Jones L, et al. An overview of research and evaluation designs for dissemination and implementation. Annu Rev Public Health 2017 Mar 20;38:1-22 [FREE Full text] [CrossRef] [Medline]
- Brown M, Ofili EO, Okirie D, Pemu P, Franklin C, Suk Y, et al. Morehouse Choice Accountable Care Organization and Education System (MCACO-ES): integrated model delivering equitable quality care. Int J Environ Res Public Health 2019 Aug 25;16(17):3084 [FREE Full text] [CrossRef] [Medline]
- Webb T, Joseph J, Yardley L, Michie S. Using the internet to promote health behavior change: a systematic review and meta-analysis of the impact of theoretical basis, use of behavior change techniques, and mode of delivery on efficacy. J Med Internet Res 2010 Feb 17;12(1):e4 [FREE Full text] [CrossRef] [Medline]
- Howes C, Purver M, McCabe R. Using conversation topics for predicting therapy outcomes in schizophrenia. Biomed Inform Insights 2013 Jul 15;6(Suppl 1):39-50 [FREE Full text] [CrossRef] [Medline]
- Park J, Kotzias D, Kuo P, Logan Iv RL, Merced K, Singh S, et al. Detecting conversation topics in primary care office visits from transcripts of patient-provider interactions. J Am Med Inform Assoc 2019 Dec 01;26(12):1493-1504 [FREE Full text] [CrossRef] [Medline]
- Roberts ME, Stewart BM, Tingley D, Lucas C, Leder-Luis J, Gadarian SK, et al. Structural topic models for open‐ended survey responses. Am J Polit Sci 2014 Mar 06;58(4):1064-1082. [CrossRef]
- Blei D, Ng A, Jordam MI. Latent dirichlet allocation. J Mach Learn Res 2003 Jan 3;3:993-1022. [CrossRef]
- Blei DM, Lafferty JD. A correlated topic model of science. Annals Appl Stat 2007 Jun 1;1(1):17-35 [FREE Full text] [CrossRef]
- Mimno D, Lee M. Low-dimensional embeddings for interpretable anchor-based topic inference. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP). 2014 Presented at: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP); Oct 25-29, 2014; Doha, Qatar. [CrossRef]
- Denny M, Spirling A. Text Preprocessing For Unsupervised Learning: Why It Matters, When It Misleads, And What To Do About It. Political Analysis 2018;26(2):168-189. [CrossRef]
- Schofield A, Mimno D. Comparing apples to apple: the effects of stemmers on topic models. Transact Assoc Comput Linguistics 2016 Dec;4(2):287-300. [CrossRef]
- Understanding text pre-processing for latent dirichlet allocation. Cornell. URL: https://www.cs.cornell.edu/~xanda/winlp2017.pdf [accessed 2022-02-03]
- Dineen-Griffin S, Garcia-Cardenas V, Williams K, Benrimoj S. Helping patients help themselves: a systematic review of self-management support strategies in primary health care practice. PLoS One 2019;14(8):e0220116 [FREE Full text] [CrossRef] [Medline]
- Jonkman NH, Schuurmans MJ, Jaarsma T, Shortridge-Baggett LM, Hoes AW, Trappenburg JC. Self-management interventions: proposal and validation of a new operational definition. J Clin Epidemiol 2016 Dec;80:34-42. [CrossRef] [Medline]
- Schillinger D, Piette J, Grumbach K, Wang F, Wilson C, Daher C, et al. Closing the loop: physician communication with diabetic patients who have low health literacy. Arch Intern Med 2003 Jan 13;163(1):83-90. [CrossRef] [Medline]
- O'Connor PJ, Sperl-Hillen J, Johnson PE, Rush WA, Crain AL. Customized feedback to patients and providers failed to improve safety or quality of diabetes care: a randomized trial. Diabetes Care 2009 Jul;32(7):1158-1163 [FREE Full text] [CrossRef] [Medline]
- Hood KK, Hilliard M, Piatt G, Ievers-Landis CE. Effective strategies for encouraging behavior change in people with diabetes. Diabetes Manag (Lond) 2015;5(6):499-510 [FREE Full text] [Medline]
- Polonsky WH, Fisher L. When does personalized feedback make a difference? A narrative review of recent findings and their implications for promoting better diabetes self-care. Curr Diab Rep 2015 Aug;15(8):50. [CrossRef] [Medline]
|CHIT: consumer health information technology|
|DSME: diabetes self-management education|
|MCACO: Morehouse Choice Accountable Care Organization|
|STM: structural topic model|
Edited by C Richardson; submitted 16.08.20; peer-reviewed by J Luo, Z Ren, A Alasmari, I Shubina, R Krukowski; comments to author 18.11.20; revised version received 13.01.21; accepted 19.11.21; published 18.03.22Copyright
©Muhammed Y Idris, Ernest Alema-Mensah, Elizabeth Olorundare, Mohammad Mohammad, Michelle Brown, Elizabeth Ofili, Priscilla Pemu. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 18.03.2022.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.