Background: Acquired brain injuries (ABIs), such as stroke and traumatic brain injury, commonly cause cognitive-communication disorders, in which underlying cognitive difficulties also impair communication. As communication is an exchange with others, close others such as family and friends also experience the impact of cognitive-communication impairment. It is therefore an internationally recommended best practice for speech-language pathologists to provide communication support to both people with ABI and the people who communicate with them. Current research also identifies a need for neurorehabilitation professionals to support digital communication, such as social media use, after ABI. However, with >135 million people worldwide affected by ABI, alternate and supplementary service delivery models are needed to meet these communication needs. The “Social Brain Toolkit” is a novel suite of 3 interventions to deliver communication rehabilitation via the internet. However, digital health implementation is complex, and minimal guidance exists for ABI.
Objective: This study aimed to support the implementation of the Social Brain Toolkit by coproducing implementation knowledge with people with ABI, people who communicate with people with ABI, clinicians, and leaders in digital health implementation.
Methods: A maximum variation sample (N=35) of individuals with living experience of ABI, close others, clinicians, and digital health implementation leaders participated in an explanatory sequential mixed methods design. Stakeholders quantitatively prioritized 4 of the 7 theoretical domains of the Nonadoption, Abandonment, Scale-up, Spread, and Sustainability (NASSS) framework as being the most important for Social Brain Toolkit implementation. Qualitative interview and focus group data collection focused on these 4 domains. Data were deductively analyzed against the NASSS framework with stakeholder coauthors to determine implementation considerations and strategies. A collaborative autoethnography of the research was conducted. Interrelationships between considerations and strategies were identified through a post hoc network analysis.
Results: Across the 4 prioritized domains of “condition,” “technology,” “value proposition,” and “adopters,” 48 digital health implementation considerations and 52 tailored developer and clinician implementation strategies were generated. Benefits and challenges of coproduction were identified. The post hoc network analysis revealed 172 unique relationships between the identified implementation considerations and strategies, with user and persona testing and responsive design identified as the potentially most impactful strategies.
Conclusions: People with ABI, close others, clinicians, and digital health leaders coproduced new knowledge of digital health implementation considerations for adults with ABI and the people who communicate with them, as well as tailored implementation strategies. Complexity-informed network analyses offered a data-driven method to identify the 2 most potentially impactful strategies. Although the study was limited by a focus on 4 NASSS domains and the underrepresentation of certain demographics, the wealth of actionable implementation knowledge produced supports future coproduction of implementation research with mutually beneficial outcomes for stakeholders and researchers.
International Registered Report Identifier (IRRID): RR2-10.2196/35080
Acquired brain injury (ABI), including stroke and traumatic brain injury (TBI), is a globally prevalent condition affecting >135 million people . ABI represents a significant and growing global burden of disease [ , ] with a high need for rehabilitation service support [ ].
Cognitive-communication disorders are common sequelae of ABI [, ]. Cognitive-communication disorders arise when underlying impairments in cognitive skills, such as memory and organization, manifest in a person’s communication [ ]. Evidence-based treatment for these communication difficulties is provided by speech-language pathologists [ ]. Author RM recalls the following from her living experience:
During my months in hospital, I discovered the critical role speech therapists have in helping people who acquire TBI get back some semblance of normality. As well as losing all my power of speech, my cognition was truly scrambled. So not only did the speech therapist help me regain the ability to speak, but also to reason.
As communication is a bilateral and multilateral exchange between “communication partners,” close others such as friends and family can also be affected by communication changes after ABI. For example, if a person with ABI experiences difficulty staying on topic or remembering details to include during conversations, their communication partners have been found to adjust their own communication to compensate for these communication behaviors . In doing so, communication partners can positively or detrimentally affect the communication rehabilitation of the person with ABI [ - ]. The protocol for this study included a firsthand account of potentially negative impacts of a close other’s sense of disempowerment during communication after ABI [ ]. Conversely, author MTM recalls positive experiences of being able to support a friend:
The diminishment of Rosey’s speech was hard on her—she explains and expresses her world through speech. I saw Rosey at least weekly and like Rosey I have a background in special education, so I felt I had some knowledge of supporting others. I think the best support I gave Rosey was feedback—not when we were talking as friends do, but after. I understand the value of data and by being able to tell her explicitly the gains I could see she was making in between each visit, I could affirm her and support her in a really positive way.
Communication changes after ABI can have pervasive and long-term psychosocial effects on both the person with ABI and their close others. As a result of communication changes after ABI, people with ABI can experience negative impacts on their social participation [, ], relationships [ , ], employment [ , ], and mental health [ ]. Inversely, close others can experience altered and challenging relationships with the person with ABI [ ], increased burden, and reduced quality of life [ , ]. Wider communities are also economically affected by the burden of informal care, including reduced workforce participation by caregivers [ ].
The internationally recommended best practice approach to managing communication impairment after ABI is to provide communication partner training (CPT) to both the person with ABI and the people who communicate with them after ABI [, ]. However, the substantial and growing burden of ABI makes it challenging to meet this recommendation at a global scale, for both health care systems generally [ , ] and the speech-language pathology profession specifically [ ]. For example, in 2 recent national surveys of speech-language pathologists supporting people with ABI, a minority of 122 Australian speech-language pathologists self-reported using published, evidence-based programs to train familiar (11/110, 10%) and unfamiliar (10/75, 13%) communication partners of adults with stroke [ ], with comparable proportions of 169 speech-language pathologists surveyed in the United Kingdom reportedly using published programs to train familiar (27/136, 19.9%) and unfamiliar (12/87, 14%) communication partners of adults with TBI [ ].
Current research also indicates a need for neurorehabilitation professionals to support people with ABI in the effective use of social media , as communication difficulties can extend beyond face-to-face conversations to digital interactions. Potential neurorehabilitation support therefore extends to cyber safety training for people with ABI, with exploratory research indicating that more than half (n=54, 53.5%) of 101 surveyed Australian and New Zealand neurorehabilitation providers had clients who had been affected by cyberscams [ ]. However, there is currently a lack of effective interventions supporting digital interactions through social media after ABI [ , ].
To address these needs, the “Social Brain Toolkit”  was developed as a novel suite of 3 targeted communication interventions delivered via the internet. It includes a self-directed web-based training program, “social-ABI-lity,” specifically developed to address an identified need [ , ] to support people with ABI to use social media, and 2 stand-alone yet complementary CPT programs, “interact-ABI-lity” and “convers-ABI-lity.” interact-ABI-lity and convers-ABI-lity were derived from previously published, evidence-based [ , ], face-to-face and telehealth-delivered CPT programs targeting cognitive-communication disorders, “TBI Express” [ ] and “TBIconneCT” [ ]. A key differentiation from these previous programs is that convers-ABI-lity has a fully web-based service delivery model incorporating self-directed learning modules and weekly telehealth sessions between a person with ABI, communication partner, and speech-language pathologist. interact-ABI-lity is a short, self-directed learning program solely for the communication partner and features advice relating to aphasia (language impairment), dysarthria (weakness in speech muscles), augmentative and alternative communication (communication via methods other than natural speech, such as facial expression or a voice output device), and cognitive-communication difficulties (communication difficulties due to cognitive impairments). A comprehensive comparison of each intervention is presented in .
Digital health interventions and applications targeting neurological conditions continue to proliferate with modern technologies [, ]. However, the demonstration of their clinical effectiveness is insufficient to guarantee their successful implementation [ ]. Digital health implementation can be highly complex [ ], with growing concern in relation to complex challenges such as equity of access [ ]. Despite a need for guidance, there is currently a paucity of empirical evidence to guide the process of tailoring implementation strategies [ - ]. This gap is compounded by misalignment between general implementation frameworks and the specific challenges of implementing digital health [ ]. Therefore, there is currently an urgent need for guidance to inform the development of tailored, evidence-based implementation strategies that align with the burgeoning need for digital health.
It has been recommended that researchers consult stakeholders to identify implementation barriers and tailored strategies . Therefore, this study aimed to coproduce this implementation knowledge with stakeholders [ ] to support the deployment of implementation strategies: (1) at intervention launch, (2) iteratively during initial pilot (convers-ABI-lity) and 12-month implementation-effectiveness hybrid studies (interact-ABI-lity and social-ABI-lity) [ ], and (3) in future versions. The Australian development team includes speech-language pathologist researchers LT, RR, and MB from the University of Sydney and EP and MM from the University of Technology Sydney, with advisory and steering committees comprising people with living experience of ABI and clinicians supporting them. Partner organizations include technology vendor Changineers, community partner Brain Injury Australia, and funding partner icare New South Wales. The coproduction of implementation knowledge was conceptualized and led by author MM, whose relevant experience and qualifications include a clinical background in speech-language pathology with specialized training in accessibility, prior experience of coauthorship with stakeholders, qualitative research and peer-review experience, and membership of a high-risk human research ethics committee.
Given the aforementioned medical and psychosocial complexity of ABI [, , - ]; the complex nature of health care systems as a complex adaptive system (CAS) [ , ] facing the growing global burden of ABI [ - ]; and the complexity of implementation itself [ , ], specifically digital health implementation [ , - ], researchers should anticipate that the implementation, scale-up, and sustainability of the Social Brain Toolkit might be highly complex. It has been suggested that the field of complexity science can offer essential insights to support health [ ] and implementation [ ] research contending with real-world complexity. However, given that researchers have used a wide range of conceptualizations of complexity [ ], with interchangeable use of related terms and concepts [ - ], a clear definition of complexity was required for this study [ , ]. In this study, digital health implementation was conceptualized as a CAS in its own right, in which multiple “agents” (eg, implementation considerations and strategies) followed local rules that adapted to each other (eg, an implementation strategy reducing the impact of an implementation consideration). System “attractors” (eg, a new implementation strategy or consideration) could vary according to parameters such as the number, strength, and diversity of their connections, with some stability gained from the constraints of other agents but also lost as a result of the evolving, emergent nature of local rules. Despite conceptualizing implementation as a CAS, it must also be acknowledged that this study adopts the “least radical” conceptualization of complexity [ ], in which complexity is conceptualized as varying between the “simple” (straightforward, predictable, and with few components), “complicated” (with multiple interacting components or issues), or “complex” (dynamic, unpredictable, and not easily disaggregated into constituent components) [ ], rather than more radically viewed as something omnipresent at all times. This least radical conceptualization of complexity served as a logic pathway for the use of tailored implementation strategies [ ] as recommended in implementation science, whereby implementation strategies could facilitate implementation by theoretically managing or reducing the complexity of implementation considerations from “complex” to merely “complicated” or “simple.”
As described in the published protocol , the broad aim of this study was to support the implementation of the Social Brain Toolkit by coproducing implementation knowledge with people with ABI, close others, clinicians, and leaders in digital health implementation. The specific aims of the study were to (1) prioritize theoretically informed implementation targets for the Social Brain Toolkit; (2) understand the nature of these priorities; (3) coproduce targeted implementation strategies; and (4) explore potential interrelationships between implementation considerations and strategies as a CAS, as a post hoc aim formulated in response to emergent complexity within the data set.
This study was epistemologically, ontologically, methodologically, and axiologically oriented from a critical realist research paradigm . Critical realism would (1) ontologically support the examination of underlying mechanisms beyond empirical observation; (2) enable an epistemologically pluralist stance in which researchers can value complexity science alongside implementation science and experiential knowledge alongside academic knowledge; (3) allow for pragmatic and pluralist methodologies, including both implementation science and complexity science approaches; and (4) axiologically support the wider emancipatory values of coproducing Open Science [ ], implementing evidence-based practice, and improving health outcomes for people with ABI and their communities.
Therefore, this study was theoretically guided by the Nonadoption, Abandonment, Scale-up, Spread, and Sustainability (NASSS) framework . The NASSS framework was selected for its comprehensive inclusion of 7 interacting domains of digital health implementation, scalability, and sustainability; systematic itemization of the degree of complexity in each of these 7 domains; specific focus on digital health; and alignment with author MM’s living experience of using technology to implement and scale speech-language pathology services.
As shown in, participants were a purposive, maximum variation sample (N=35) of adults with living experience of ABI, communication partners, speech-language pathologists supporting people with ABI, and individuals with experience implementing digital health. Out of 10 individuals who had living experience of ABI, 2 (20%) also had either clinical experience of ABI or digital health implementation experience. In addition, 2 (40%) out of 5 clinicians and 1 (10%) out of the 10 people with ABI had prior experience with TBI Express [ ] and TBIconneCT [ ] and were thus able to directly compare their delivery and content with those of the potential or prototype versions of convers-ABI-lity and interact-ABI-lity ( ).
People with living experience of ABI (10/35, 29%) were required to be discharged from hospital, participating at least 6 months after sustaining an ABI, and with adequate capacity to consent to study participation. An adapted consenting process , published as supplementary materials of the study protocol [ ], was used screen capacity to consent. Participants with living experience of ABI were required to be based in Australia, where the Social Brain Toolkit was developed and where they could be financially reimbursed for their living experience expertise. People with ABI were recruited through tailored electronic flyers posted with alternative text on social media, as well as organizational websites, email distribution lists, and snowball recruitment.
Communication partners (11/35, 31%) were self-identified as individuals who interacted with a person with ABI at least once a week and who had not sustained an ABI themselves. Communication partners were also required to be based in Australia, where the Social Brain Toolkit was developed and where they could be financially reimbursed for their living experience expertise. They were also recruited through tailored electronic flyers, organizational websites, email distribution lists, and snowball recruitment.
Clinicians (5/35, 14%) were self-identified as qualified professionals practicing with a caseload of which at least 20% included people with ABI. No other restrictions were applied, including restriction on international participation. Clinicians were recruited through tailored electronic flyers, organizational websites, and email distribution lists.
Individuals experienced in digital health implementation (9/35, 26%) were required to have a research or industry track record in digital health implementation and were able to participate internationally. Eligible individuals were recruited via direct email to publicly listed contact details on university website profiles, researcher networks, and snowball recruitment.
Participant demographic information is described in detail inand was previously published in the protocol [ ].
|Adults with experience of ABIb (n=10, 29%)||Communication partners (n=11, 31%)||Clinicians (n=5, 14%)||Individuals with experience of digital health implementation (n=9, 26%)|
|Living experience||9 (90%) with living experience of ABI and 1 (10%) with living experience of both ABI and clinically supporting patients with ABI||11 (100%) with living experience of being a communication partner of someone with ABI||5 (100%) with clinical experience supporting people with ABI as speech-language pathologists||8 (89%) with living experience implementing digital health interventions for any condition and 1 (11%) with living experience of both ABI and digital health implementation|
|Sex, n (%)|
|Male||7 (70)||4 (36)||1 (20)||6 (67)|
|Female||3 (30)||7 (64)||4 (80)||3 (33)|
|Location||Australia (inclusion criteria)||Australia (inclusion criteria)||1 (20%) from the United Kingdom, 1 (20%) from the Netherlands, and 3 (60%) from Australia||1 (11%) from Denmark and 8 (89%) from Australia|
|Education||1 (10%) with graduate diploma or certificate, 4 (40%) with bachelor’s degree, 4 (40%) with certificates or diplomas, and 1 (10%) with high school diploma||1 (9%) with a PhDc, 1 (9%) with a master’s degree, 2 (18%) with a graduate diploma or certificate, 3 (27%) with a bachelor’s degree, 3 (27%) with certificates or diplomas, and 1 (9%) with a high school diploma||2 (40%) with a master’s degree, 1 (20%) with a graduate diploma or certificate, and 2 (40%) with a bachelor’s degree||9 (100%) with a PhD|
|Age (years), n (%)|
|18-24||0 (0)||0 (0)||1 (20)||N/Ad|
|25-34||2 (20)||1 (9)||2 (40)||N/A|
|35-44||4 (40)||2 (18)||1 (20)||N/A|
|45-54||2 (20)||2 (18)||1 (20)||N/A|
|55-64||1 (10)||1 (9)||0 (0)||N/A|
|>65||1 (10)||5 (45)||0 (0)||N/A|
|Time postinjury, n (%)|
|>12 mo||9 (90)||N/A||N/A||N/A|
|<12 mo||1 (10)||N/A||N/A||N/A|
|Clinical experience||N/A||N/A||3 (60%) working for >10 years with 80% to 100% of their caseload involving people with ABI, 1 (20%) working for <5 years with 80% to 100% of their caseload involving people with ABI, and 1 (20%) working for <5 years with 20% of their caseload involving people with ABI||N/A|
|Relationship with the person with ABI||N/A||3 (27%) friends, 3 (27%) spouses, and 5 (45%) family members||N/A||N/A|
aThis table is republished from the study protocol  under Creative Commons Attribution License BY 4.0.
bABI: acquired brain injury.
cPhD: Doctor of Philosophy.
dN/A: not applicable.
This study includes a (1) mixed methods coproduction of implementation knowledge with (2) collaborative autoethnography of the coproduction process and (3) post hoc analysis of the complexity of implementation. The full methodological details and original materials used in the coproduction were previously published in an Open Access study protocol . An additional step in the analysis and the methodological details of collaborative autoethnography and network analysis are also described below.
Coproduction of Implementation Knowledge
The coproduction of implementation knowledge was completed as described in the published protocol , using an explanatory sequential mixed methods design [ ] as follows:
1. Author MM used the NASSS Complexity Assessment Tool  to script and edit a series of videos explaining each domain of the NASSS framework [ ] in relation to the Social Brain Toolkit. The videos were short, slowly paced, captioned in plain English, and included both visual and auditory accessibility. Downloadable video files were published Open Access as supplementary materials of the study protocol [ ].
2. Videos were embedded with large sans serif transcripts within an electronic prioritization survey created using Qualtrics software (Qualtrics International Inc) . To identify the 4 most highly ranked domains, respondents rated, commented on, and ranked the perceived significance of each domain of the NASSS framework for the implementation of the Social Brain Toolkit. The survey design, layout, and questions, refined through 3 rounds of piloting from researcher, caregiver, and clinician perspectives, are downloadable as supplementary materials of the Open Access study protocol [ ]. People with expert living experience of ABI (10/35, 29%) and people with digital health implementation experience (9/35, 26%) accessed the videos and questions via screenshare during a 1- to 2.5-hour video call interview with a speech-language pathologist researcher (MM), whereas clinicians (5/35, 14%) and communication partners (11/35, 31%) completed the electronic survey independently in their own time. Leaders in digital health implementation (9/35, 26%) viewed the video transcripts rather than the accessible videos. People with living experience expertise in ABI (10/35, 29%) viewed and member checked each of their responses in real time via screenshare on Teams (Microsoft Corporation) [ ] before proceeding to the next question.
3. Over a series of 7 focus groups containing a diversity of 3 to 6 people with living experience of ABI, close others, or clinicians, the 4 most highly ranked domains of the NASSS framework were verbally discussed . These were the (1) condition of ABI, (2) technology used to deliver the interventions in the Social Brain Toolkit, (3) supply- and demand-side value propositions of the Social Brain Toolkit, and (4) adopters of the Social Brain Toolkit. Qualitative survey and interview data pertaining to these 4 domains were anonymously and visually presented via a screen-shared slideshow summary, in combination with preliminary systematic review findings [ ], followed by 1 to 3 prompt questions for group discussion. Plain English questions are available in the study protocol, with a detailed outline of time allocations and procedures provided in supplementary materials [ ]. In addition, participant-generated discussion points were sequentially incorporated over the span of the 7 focus group discussions to promote dialogue across all participants. Each 3-hour focus group was facilitated using Teams (Microsoft Corporation) [ ] video calls, with at least 1 qualified speech-language pathologist researcher present in each group. Interviews and focus groups were audio recorded and video recorded for verbatim transcription. Data were collected from April 13, 2021, to November 18, 2021.
4. Data were deductively analyzed  using the NASSS framework [ ]. Meaning was parsed and condensed from the verbatim transcripts in Excel (Microsoft Corporation) [ ]. Domains and subdomains were then coded with reference to the original published domain and subdomain definitions [ ] which had been reprinted as a codebook in large, sans serif font with color codes. As described in the protocol [ ], only the top 4 domains were included in the final narrative report.
An additional step not specified in the protocol was a deductive content analysis  of the implementation considerations and strategies within each domain and subdomain from the verbatim transcripts. Author MM also deductively identified the key implementer (ie, clinician, developer, or both) of each strategy.
Collaborative Autoethnography of the Research
Collaborative autoethnography may help implementation scientists reflexively document and learn from experiences of intersectionality during research coproduction  and “shed light on the challenges, solutions and processes of producing or co-producing knowledge” [ ]. Collaborative autoethnography was also used to qualitatively evaluate the prioritization process and outcomes through stakeholder reflection. All participants with living experience of ABI, both those with direct living experience of ABI (10/35, 29%) and those with indirect living experience of ABI as a close other (11/35, 31%) or clinician (5/35, 14%), were eligible to participate in the collaborative autoethnography. Of the 26 eligible participants, 9 (35%) consented to be coauthors, including a close other (BT) who participated in the authorship of the protocol [ ] and a close other (MTM) who participated in the authorship of the results. An additional person with living experience expertise in ABI (CR) was invited to participate in the authorship of the study protocol at the suggestion of a study participant. The methodology followed that outlined by Ratnapalan and Haldane [ ], in which (1) the primary author (MM) solicited first-person narratives from the coauthors through their preferred communication mode, (2) the coauthors refined and approved their contributions, and (3) all authors approved the final narrative. A notable deviation from the published method was that all authors were given the opportunity to have their narrative contributions attributed to them by name, if desired.
Complexity of Implementation
Initially, the documentation of potential interrelationships between the identified implementation considerations and strategies was necessary during primary analysis to avoid duplicate findings and to ensure that only unique, mutually exclusive considerations and strategies were identified and counted (ie, strategies 1.3, 1.6, and 2.5 repeat but are not recounted toward the total). However, relationships were ultimately formally and systematically documented across the entire qualitative data set () [ ], as participants both explicitly and implicitly described multiple additional interrelationships. These included strategies that targeted multiple considerations, considerations that interacted with each other, and strategies that may benefit other strategies.
Interrelationships were identified through either explicit or implicit reference in the verbatim qualitative focus group or interview data; examples are provided in. For example, although suggestions to “leverage technology to reduce user memory requirements” (strategy 1.7) originally targeted “memory impairments” (consideration 1D), an explicit reference to “short term memory loss” in the stakeholder suggestion to “provide printable reference materials” (strategy 4.7) explicitly identified print materials as also potentially benefiting individuals with memory impairments. As an example of an implicit relationship, although the strategy to “create bite-sized modules that allow for breaks” (strategy 1.3) was identified as supporting individuals with “concentration impairments” (consideration 1B), it can be inferred that short modules could also support someone with “fatigue” (consideration 1G) and “pain” (consideration 1H) because of the statement that these comorbidities make it difficult to sit for “long periods of time.”
Visual Network Analysis
Following the identification of these relationships, author MM completed a network analysis . Specifically, the visual network analysis (VNA) method was selected to provide a thick description and comprehensive visualization of qualitative relational data [ ]. Reflecting the study’s underlying conceptual and theoretical focus on complexity, VNA was used to examine the mechanisms that emerge from a network’s composition [ ]. VNA was thus selected to understand and explore the relational distribution of the implementation of the Social Brain Toolkit without having to invoke holistic or individualistic explanations [ , ]. It was also selected to enable presentation, rather than representation, of how the Toolkit’s implementation is constantly affected by relationships in a CAS. VNA was also able to accommodate a pragmatic, a priori cutoff of the network at domains 1 to 4 [ ].
Although VNA cannot be proceduralized into neatly delineated substeps , it can be broadly outlined as progressing from qualitative data collection and coding to visualization and narrative analysis and interpretation [ ]. Visualization was completed in Gephi (Gephi Consortium) [ ] software. In addition to its ability to represent each consideration and strategy as a circular “node” and each directional relationship as a curved “edge,” Gephi was selected for its capacity to visually emphasize particular network properties through proportionality [ ]. That is, the size of each node could reflect its total outgoing relationships, with larger nodes indicating a larger number of outgoing relationships, and vice versa. Its ForceAtlas algorithm was selected as adequate to account for node sizes in a layout of 100 nodes. Each domain was indicated by 1 of 4 colors, with considerations and strategies indicated by a lighter and darker shade of that color, respectively. Edges were colored according to their source node. Narrative analysis and interpretation focused on the number of outgoing connections from each node to semi-quantitatively identify the potentially most impactful “leverage points” within the identified networks.
Coproduction of Implementation Knowledge
As noted in the Study Design section of the same name, interviewees member checked the transcription and interpretation of their survey and interview responses in real time via screen-shared video calls. All interview and focus group transcriptions were also verified against the original audio and video recordings. All participants were provided with the opportunity to further member check the interpretation of their qualitative data, with any changes or clarifications incorporated and reported as original data. Of the 35 participants, 15 (43%; including 7/10, 70% of people with ABI; 1/11, 9% of close others; 4/5, 80% of speech-language pathologists; and 3/9, 33% of leaders in digital health implementation) confirmed the interpretation of their data, with either no changes or minor clarifications made. Furthermore, coding for 5 (71%) of the 7 focus groups was verified by 1 or 2 authors who directly participated in those focus groups (RM, AS, BM, J-CE, MRW, MTM, MW, or LW), and 25% of the coding of the first individual interview was verified by a second author (DD). In addition, direct stakeholders in the implementation research are coauthors who have contributed to the analysis, interpretation, and write-up of the results.
Collaborative Autoethnography of the Research
The following three key criteria for robust collaborative autoethnography  were adhered to: (1) firsthand accounts of the implementation research process from all the coauthors were included across the 2 publications of the study protocol and results, (2) the accounts focused on research challenges and experiences rather than retelling the study results, and (3) the accounts included broader reflections on Open Science [ ] and the systems and culture of academia [ ]. Finally, both the research protocol and results were comprehensively reported according to the reporting guideline for priority setting of health research [ ] to address aspects pertinent to coproduction more comprehensively than required by standard mixed methods guidelines.
Complexity of Implementation
As complexity science approaches follow different and often contrasting standards of rigor from those followed in traditional health research , the rigor of the network analysis depended on uniformity throughout the study in the theoretical mapping of relationships [ ] rather than the positivist “triangulation” of data between multiple researchers. Therefore, network analysis was completed by the author who was most familiar with the theoretical framework and immersed across the full data set (MM).
The study received human research ethics approval with commendation from the University of Technology Sydney (ETH20-5466).
All participants provided informed written consent. In recognition of their living experience expertise, people with ABI and their close others were reimbursed at the rate stipulated by Health Consumers New South Wales . All data were stored and transferred securely. Identifying audio and video data were deidentified as much as possible and stored separately from identifying information. Collaborative autoethnographic data were attributed to authors by name only when discussed and agreed upon in writing during the authorship process.
Across the top four prioritized domains of (1) condition, (2) technology, (3) value proposition, and (4) adopters of the Social Brain Toolkit, the coproduction of implementation knowledge yielded a total of 48 unique digital health implementation considerations and 52 unique, tailored implementation strategies for both developers and clinicians. Collaborative autoethnography identified both the benefits and challenges of coproducing implementation research within traditional academic systems. Post hoc network analysis yielded 172 unique relationships between the 100 implementation considerations and strategies and identified the potentially most impactful implementation strategies within the network.
Coproduction of Implementation Knowledge
First Priority Domain: Condition
Stakeholders coproduced a total of 14 implementation considerations and 17 tailored strategies within their top priority domain of the “condition” of ABI (). These spanned all 3 subdomains, including the nature of the condition, its comorbidities, and sociocultural factors.
|Subdomain and consideration||Strategy tailored to the consideration||Implementer|
|Nature of ABIa|
|Clinician and developer|
|Clinician and developer|
|Clinician and developer|
aABI: acquired brain injury.
Although developers could deploy all 17 strategies in this domain by making various design, accessibility, and language adjustments, speech-language pathologists could also deliver 4 of these strategies within clinical settings. First, when considering the subdomain concerning of the nature of a person’s ABI, stakeholders highlighted the importance of the “stage of recovery” of a person with ABI (consideration 1A). Communication partners expressed a desire to receive the Social Brain Toolkit “as early as possible” (Communication partner 6, focus group 6), whereas people with ABI may not yet be ready: “I was just focusing on getting through another day, so giving me any work or anything, I just know in hospital it was beyond me at that time” (Person with living experience of ABI 2, focus group 3). Therefore, stakeholders suggested disseminating the Social Brain Toolkit to close others while the person with ABI is in the acute stage of their recovery, and to people with ABI between discharge and rehabilitation (strategy 1.1). Stakeholders also expressed a desire for speech-language pathologists to help them determine whether a tool would be suitable for a person with ABI and their communication partners (strategy 1.2). This approach was perhaps best summarized in the words of a person with ABI who reflected on the best time for dissemination as follows:
Probably not straight after I’d had the accident [for me], but my parents, my friends, my best friend, who came and saw me every day at the rehab[ilitation] center, every day at the hospital, he could have done this straight up, so the support that came from it, he’d have been able to tell my parents, and my parents would be able to tell my sisters, and my sisters would be able to effectively understand what’s going on with me.
[Person with living experience of ABI 3, focus group 2]
In the theoretical subdomain concerning comorbidities, speech-language pathologists were identified as potentially being able to assist the management of “fatigue” (consideration 1G) and “pain” (consideration 1H) by suggesting that people with ABI complete the intervention in the morning (strategy 1.13). Finally, in the subdomain concerning sociocultural factors, speech-language pathologists were identified as being able to seek multidisciplinary support if needed (strategy 1.16):
When speech pathologists are using these types of tools with clients, especially when you’re working on communication with a conversation partner, it can sort of reveal sort of pre-injury relationship or marital or friendship issues. And sometimes I find I need to get people to see a social worker, psychologist, relationship counsellor, before we then can address communication, because I feel like sometimes that’s just worth as a clinician, being mindful of.
[Speech-language pathologist 1, focus group 6]
Second Priority Domain: Technology
Stakeholders ranked the “technology” used to deliver the Social Brain Toolkit interventions as the second most important implementation target. They identified a total of 14 considerations and 8 unique tailored strategies across all 4 subdomains, including the key features of the technology, the knowledge made visible by and required to use it, and supply models ().
Technological considerations could primarily be addressed by developers, including ensuring a responsive web design across all device types (strategy 2.1), providing appropriate software training and video guides (strategies 2.4-2.7), and creating clinician dashboards (strategy 2.3). However, speech-language pathologists could also deliver 2 strategies. First, assistance with hardware setup (strategy 2.5) could only be provided to people with ABI who might want “someone to help set up the computer” (Person with living experience of ABI 5, focus group 3) by a clinician as opposed to a developer. Second, clinicians could reinforce the recommendation that telehealth video calls be conducted via a tablet or computer (strategy 2.2) to make nonverbal communication visible:
...observations about nonverbal communications, so how the couple are sitting, whether they’re facing each other, looking at gestures and things like that. Often with smartphones you can see the person’s face but you’re not getting much else apart from that. So, from a clinician’s perspective, all of that other non-verbal stuff is really important and good to observe, which you don’t always get with the smartphone [with] just people often holding it up.
[Speech-language pathologist 1, focus group 6]
|Subdomain and consideration||Strategy tailored to the consideration||Implementer|
|Key features of the technology|
|Knowledge made visible by the technology|
|Clinician and developer|
|Knowledge required to use the technology|
aABI: acquired brain injury.
Third Priority Domain: Value Proposition
A total of 8 considerations and 13 tailored strategies were identified in relation to the “value proposition” of the Social Brain Toolkit. These spanned both the supply- and demand-side value subdomains ().
On the demand side, developers of the Social Brain Toolkit could direct efforts toward communicating the Toolkit’s value in improving communication after ABI (consideration 3A), supporting communication partners after ABI (consideration 3B), addressing social media use after ABI (3C), and improving access to support (consideration 3D). On the supply side, developers could offer improvements to service efficiency (consideration 3E) and the provision of person-centered care (consideration 3F). Developers could also address the financial sustainability of the interventions (consideration 3G) by adopting a hierarchical pricing structure, with least to no payment from people with ABI, and the most funding from organizations, government, or insurance (strategy 3.12). It was also recommended that developers address the scalability (consideration 3H) of the Social Brain Toolkit in a strategic, stepwise manner (strategy 3.13).
|Subdomain and consideration||Strategy tailored to the consideration||Implementer|
|Clinician and developer|
aABI: acquired brain injury.
There were 2 ways in which clinicians could also be instrumental in the dissemination of the Social Brain Toolkit. First, although stakeholders emphasized the importance of disseminating the Social Brain Toolkit through as many channels as possible, including through professional groups and organizations, social media, and a website, clinicians were the preferred referral source for people with ABI and their communication partners (strategy 3.3): “I would take their word because they are professionals, and the advice directly from them is better than Google or the Internet” (Communication partner 7, focus group 2). Second, leaders in digital health implementation highlighted the value of the Social Brain Toolkit gaining momentum through clinician champions (strategy 3.4):
It’s really critical from your aspect that you find some of those people—and there’s always a few of them, so there will always be a small percentage of people that look at it and go ‘absolutely, this is a great idea, let’s do it.’ Then the more it gets out there, the more people will start to hear about it and become onboard with it.
[Digital health implementation leader 8]
Although people with ABI, communication partners, and clinicians shared similar views on the demand-side value proposition of the Social Brain Toolkit, including common desires for “improving communication after ABI” (consideration 3A), “supporting communication partners after ABI” (consideration 3B) and “improving access to support” (consideration 3D), implementation strategies could be tailored to each stakeholder type, as outlined in. Some demand-side strategies might also apply to the supply-side proposition for health care systems. Maximizing user autonomy (strategy 3.11) as a strategy to support the supply-side value of providing person-centered care could offer demand-side value for users. Likewise, generating and updating supporting evidence (strategy 3.1) could be important for both supply- and demand-side value propositions.
People with acquired brain injury (ABI)
- 3.2: provide video stories from people with ABI
- 3.3: suggest that speech-language pathologists can refer people with ABI and their communication partners to the tools
- 3.11: maximize user autonomy
- See also 3.3: suggest that speech-language pathologists can refer people with ABI and their communication partners to the tools
- 3.5: ensure that communication partner training is described in an empowering rather than judgmental manner
- 3.7: disseminate the Social Brain Toolkit through as many channels as possible
- 3.4: gain momentum through clinician champions
- 3.6: create clear, simple messages to explain and differentiate between the 3 different tools in the Social Brain Toolkit
Health care systems
- 3.1: generate and update evidence of benefit
- 3.8: use the language of the funder to communicate the Social Brain Toolkit’s benefits
- 3.9: communicate that the Social Brain Toolkit complements or adds value to existing services, rather than replacing them
- 3.10: demonstrate to funders that there is a financial benefit to providing the tools
Fourth Priority Domain: Adopters
A total of 12 considerations and 14 strategies were identified in relation to the fourth priority: the “adopters” of the Social Brain Toolkit. These included people with ABI, their close others, and clinicians ().
Although stakeholders identified ways in which developers could provide support across all 3 subdomains, clinicians could also support implementation for people with ABI and their close others. For close others, this included facilitating buddy or peer support (strategy 4.10) and providing reminders (strategy 4.11). For people with ABI, this included providing the setup checklist created by developers (strategy 4.2), reminders to complete the intervention at a regularly scheduled time (strategy 4.4), positive feedback of progress and achievement (strategy 4.5), upfront communication of the estimated time requirement (strategy 4.6), and printable reference materials (strategy 4.7). It also included the aforementioned strategy in the technology domain of providing hardware setup support for people with ABI (strategy 2.5).
In addition, it was identified that people with ABI may benefit from being accountable to a clinician (strategy 4.8):
From our experience, the other big thing that helped: [spouse] got a lot of homework when she did speech [therapy] and that would always be done...At least by the night before. Yeah, because she didn’t want to go to her speech therapist not having done the homework.
[Communication partner 2, focus group 3]
Similarly, people with ABI may enjoy being accountable to their peers with ABI (strategy 4.9):
Working in groups works for me. Like if [peer with living experience of ABI] and I were working as a team, right, and we were meeting at say 12 o’clock every Tuesday and we had a certain deliverable that we had to meet, then he could motivate me to get my bit done and I can motivate him. Working as a team is often a good motivator for me. [It would be ideal if] at the end of the completion of the course we’d all go out and have a big cup of coffee together or something like that.
[Person with living experience of ABI 4, focus group 5]
Other people with ABI could also be a mentor:
I’m wondering if it would be useful when someone had gone through the course and had already graduated, to become a partner with someone who you are introducing it to? Given that people had the skill, or they got on [with each other], because I think, then you’ve got someone being mentored by a graduate.
[Communication partner 1, focus group 2]
Full descriptions of all 100 considerations and strategies are provided in.
|Subdomain and consideration||Strategy tailored to the consideration||Implementer|
|People with ABIa|
|Clinician and developer|
|Clinician and developer|
|Clinician and developer|
aABI: acquired brain injury.
Collaborative Autoethnography of the Research
The collaborative autoethnography included reflections on the methods of the study from the perspective of the coauthors with living experience of ABI, close others, and clinicians. Reflections on the methods of the study were previously published in the study protocol , with academic writing interleaved with quotations from people with living experience to communicate the equal value of both academic and experiential knowledge. The following results continue this reflection from the final stages of the research.
Upon appraising the outcomes of this implementation research, author RM, who has living experience of ABI, shared the following reflection:
This implementation study has not come from professionals working on the basis of what they believe will work best for their patients, this comes from those living with experience of the condition having input into what treatments would have helped them and when those treatments should have been available. This is a rare example of supply meeting demand.
Likewise, author J-CE provided the following conclusion as both a clinician and person with living experience of ABI:
This Toolkit is going to be an invaluable resource for both patients and clinicians in the future.
Stakeholders reported experiencing personal satisfaction in coproducing new knowledge, with author RM adding the following:
Not only is a very great need being met with this research study, but it also represents the intersection of my professional life as an editor and my personal life as someone living with a brain injury. I’m more than thrilled—I’m over the moon that my experiences are being recognized academically. I can’t be more proud to be a coauthor of this study—it’s beyond fulfilling, it’s wonderful as well.
Author J-CE, who has both living and clinical experiences of ABI, shared a similar reflection:
Participating in and coauthoring this research was very interesting; being able to share both my living experience with my ABI and my experience caring for people with an ABI was extremely rewarding and validating. I got to learn a lot from everyone involved in this research, especially the different types of support people with an ABI received across different states in Australia.
Sharing the living experience of ABI in the writing stage was identified to be an emotional experience. From their living experience as a close other of a person with ABI, author MTM notes that their reflection was “a very difficult passage to write, with so many memories and feelings.”
Reflecting from the researcher perspective of facilitating coproduction, MM offers the following four key learnings for fellow researchers wishing to engage in coproduction:
1. I found that coproduction required continual courage and perseverance, as it requires a researcher to overcome the inherently elitist traditions of academia, procedural inertia where there is a lack of precedent, and personal vulnerability in doing the unconventional. I particularly experienced this when interleaving first-person accounts of living experience with traditional academic writing, trying to secure and provide payment for living experience expertise, and asking for this expertise to be published as equivalent to an academic degree. I found it helpful to find solidarity at conferences with other researchers who were coproducing their research, and to view setbacks within our collective, longer-term aim to normalize the democratization of knowledge. I also found co-authors with living and clinical experience were a constant touchstone and encouragement during the isolation and struggle of driving implementation efforts , and was greatly encouraged to find allyship in journal editors who were committed to inclusive and innovative publishing practices which recognize living experience expertise. Production Editor Hannah Reinhardt communicated: I agree with you strongly that the expertise of those with living experience should be acknowledged in the research community, and this is a cause that is of personal importance to me. I was therefore thrilled to have the opportunity to raise this with JMIR's management team, in hopes of expanding our affiliation regulations. I am fortunate enough to work with an excellent group of people who are always happy to take the time to review our processes and work on innovative solutions, so this is very much a shared achievement.
2. As CR expressed in the study protocol , coproduction is a deeply human, relational process, and it is only in the context of these relationships that its ethically and politically complex nature [ ] can be managed. Insofar as it can be known in advance, clearly communicating expectations upfront, such using accepted criteria for authorship [ ] to determine authorship expectations, was helpful to plan and achieve authentic collaboration. However, coproduction often contravenes a research tradition of obtaining a clearly replicable scientific procedure. I found it important to empower coauthors to make informed choices within its complexities. For example, although the collaborative autoethnographic method [ ] prescribes anonymization of co-author contributions in the interests of ‘confidentiality,’ author decisions and indeed enthusiasm to be identified by name, with informed discussion of the potential personal and professional risks of doing so, was potentially a less paternalistic approach than anonymizing all coauthors like participants. An additional example was the need for a trusting relationship in which to raise unforeseen challenges, such as receiving predatory conference and journal emails.
3. As my clinical background was not in ABI, my coauthors provided invaluable expertise in ABI that far exceeded what I could read in academic literature and gave me an invaluable sense of purpose in my work. Coproduction also transformed my perspective of the research process, particularly the many places patient and community voices are missing from it. As ABI does not discriminate, the final author lists are a unique collective of people from all walks of life, including film and television, professional sport, health, and education. As RM and J-CE mentioned, each person brought a wealth of personal and professional experience in addition to their living experience of ABI, and this diversity of knowledge and experience was an asset to addressing the complexity of implementation. However, as shown by a markedly lower response rate in member-checking and the ‘division of labor’ across publications for communication partner co-authors, close others stood out to me as potentially requiring more proactive inclusion and financial support to avoid their underrepresentation or absence from research.
4. Researchers may benefit from allocating additional time when planning and scoping coproduced projects, as a key challenge I observed was the incongruity between the natural timeline of relationship and the tight deadlines of academic research.
Complexity of Implementation
The post hoc network analysis yielded a total of 172 unique relationships between the 100 implementation considerations and strategies (). The VNA provided an explicit visualization of the complexity reported by leaders in digital health implementation, one of whom remarked that the implementation domains were “all going back to the same problem that I said before...Essentially, I don’t think your questions...are all independent; I think they’re all interrelated” (Digital health implementation leader 7).
As another leader in digital health implementation acknowledged, implementation findings were revealed to be interdependent across all domains and subdomains: “I don’t mean to scare you, but there’s layers and layers of complexity” (Digital health implementation leader 1). All considerations and strategies were revealed to be connected in a network, except for a singular external connection between the “complexity of technological requirements” (consideration 2N) and a targeted strategy to “tailor supply models to the technological complexity of each tool” (strategy 2.8).
By contrast, strategy 4.1, to “conduct user or persona testing,” represented the highest number (22) of outgoing connections. As 64% (14/22) of these connections targeted the considerations in the stakeholders’ top priority domain of the “condition” of ABI (considerations 1A-1N), this strategy represented both high impact and high stakeholder priority. With fewer than half (9/22, 41%) as many outgoing connections, ensuring a responsive web design (strategy 2.1) was the second most influential strategy, as responsive design allowed the intervention to be available on various mainstream device types (strategy 1.17) and enabled access to a myriad of technological features (considerations 2A-2G), including options that potentially benefit people with physical impairments (consideration 1I). The third most influential strategy, with 6 outgoing connections, was ensuring that the tools were compatible with alternative technology access methods, such as eye gaze, voice, or touch, as opposed to only standard mouse and keyboard peripherals (strategy 1.6). This strategy had the identified benefits of facilitating technology access for people with ABI with sensory (consideration 1J), physical (consideration 1I), and communication (consideration 1B) impairments, and facilitating the input of patient-reported measures (consideration 2J) and progress data (consideration 2K) by people with ABI with accessibility requirements. It was also noted that these accessibility requirements should be considered in tandem with the need to avoid conveying a sense of penalty and judgment (strategy 1.11), such as standard speech-to-text devices being unable to positively acknowledge dysarthric speech. The fourth most influential strategy, and the first strategy to focus on close others, was strategy 4.10, to “facilitate buddy or peer support for close others.” This had the potential to address the 5 identified assumptions and requirements of close others (considerations 4D-4H), including participating in CPT and overcoming performance anxieties around communicating effectively with someone with ABI.
Within the identified network, there were 6 implementation strategies that each influenced 4 other strategies or considerations. These included suggesting that speech-language pathologists refer people with ABI and their communication partners to the tools (strategy 3.3), providing positive feedback of progress (strategy 4.5), ensuring that tools are available on mainstream device types (strategy 1.17), providing a setup checklist (strategy 4.2), streamlining the number of steps for users (strategy 4.3), and providing reminders to close others (strategy 4.11). Of the 90 remaining nodes, there were 9 (10%) nodes with 3 outgoing connections, 24 (27%) nodes with 2 outgoing connections, and 31 (34%) nodes with 1 outgoing connection. A total of 26 nodes received only incoming relationships, including 25 considerations and the aforementioned strategy (2.8) of tailoring supply models.
This study aimed to support the implementation of the Social Brain Toolkit by coproducing new implementation knowledge with people with ABI, close others, clinicians, and leaders in digital health implementation. The first specific aim was to identify stakeholders’ implementation priorities from the NASSS framework . Stakeholders prioritized the investigation of the (1) target condition, (2) technology, (3) value proposition, and (4) adopters of the Social Brain Toolkit. The second specific aim was to identify implementation considerations in each of the top 4 prioritized domains. Stakeholders identified a total of 48 unique digital health implementation considerations across all domains and subdomains. The third specific aim, to coproduce implementation strategies tailored to these considerations, yielded 52 unique, tailored implementation strategies across all domains and subdomains for both developers and clinicians. A collaborative autoethnography revealed both the benefits and challenges of coproducing new implementation knowledge within traditional academic systems. The fourth post hoc aim was to explore potential interrelationships between the identified implementation considerations and strategies. A post hoc network analysis presented a network of 172 unique relationships between the 100 identified considerations and strategies, with highly connected strategies, or “attractors” within the CAS offering data-driven prioritization guidance for implementation effort.
Coproduction of Implementation Knowledge
To what was previously systematically evaluated to be a sparse evidence base , the 48 stakeholder-identified implementation considerations in this study contribute the first extensive compilation of considerations for web-based psychosocial interventions for people with ABI. In particular, this same systematic review [ ] identified that the complexities within the first priority domain of the “condition” of ABI have traditionally been simplified via participant exclusion criteria to establish a clear value proposition for interventions (stakeholders’ third priority domain). By contrast, stakeholders in this study identified 14 implementation considerations and 17 implementation strategies in the top priority domain, as well as 4 demand-side and 4 supply-side considerations ( ) for the Social Brain Toolkit and 13 tailored strategies to directly address these 8 considerations. There was also guidance on how to target the strategies to specific stakeholders, including people with ABI, close others, clinicians, and health services ( ). Therefore, the study both enacted and supported the review conclusions to collaborate with stakeholders to increase implementation knowledge.
The 52 tailored strategies identified in this study were largely consistent with generic implementation  and sustainment [ ] strategies but provided more specific implementation knowledge for ABI and digital health. For example, the general recommendation to “develop strategies with patients to encourage and problem solve around adherence” [ ] was addressed through 9 unique strategies (4.1-4.9) to support people with ABI and a further 5 strategies (4.10-4.14) to support close others and clinicians. The generic recommendation to “increase demand” [ ] was extended with very specific guidance on which value proposition messages were most important to stakeholders and how, when, and to whom these might be communicated. Thus, these strategies may provide more specific guidance for other digital health implementation endeavors to support people with ABI, as well as digital health implementation more broadly [ , ]. Furthermore, the identified implementation strategies were immediately relevant for current and future iterations of the Social Brain Toolkit [ ].
Collaborative Autoethnography of the Research
Positive final reflections of the reciprocal benefit in coproducing implementation research corroborated previously published coauthor reflections in the study protocol . Reflections on the emotional nature of the writing stage of research were also consistent with the emotion described during data collection [ ], supporting the use of trauma-informed principles in coproduced research [ , ]. Researcher experiences appeared to corroborate suggestions that research practices, cultures, and structures encumber the process of coproduction, making it challenging to achieve the aspirations of coproduction [ ]. Potentially of particular interest to researchers seeking to coproduce research with people with ABI was the experience of predatory email invitations being sent to all authors, which may warrant proactive warning, education, and follow-up, given the particularity of this type of scam to academia and the known potential vulnerability of people with ABI [ ].
Complexity of Implementation
In addition to the complex network revealed by the VNA, the findings of this study illuminated a clear way forward . A complexity-informed approach identified that user and persona testing (strategy 4.1) may be the single most impactful leverage point to support digital health implementation for people with ABI, reflecting findings from a recent systematic review highlighting the potential to address the complexities for “adopters” [ ]. The fruitful use of VNA in this study directly addresses an identified need in implementation science for data-driven methods to prioritize and select implementation strategies from long lists of possibilities [ ], especially given potential time and resource constraints. In addition, a focus on strategy 4.1 to target diverse considerations in the “condition” domain offers specific guidance on how to address a previously identified gap in implementation knowledge for people with ABI who have comorbidities and additional challenges [ ], specifically, how to practically address the complexity of ABI, rather than reduce it through participant exclusion criteria. It is possible that analogous synergies may be identified and exploited for other digital health interventions and other populations.
The results also provide much-needed guidance on how to select an appropriate theoretical underpinning for future studies [, ], as the critical position of user and persona testing in the network highlights the value of user-centered design [ ]. The sociotechnical focus of the 2 most well-connected “attractors” within the system highlights the importance of using a technology-specific, as opposed to a generic, implementation framework. Therefore, this study supports and contributes methodological knowledge on how to conduct a sound integration of complexity science and implementation science [ ] to prioritize digital health implementation strategies and targets.
This study has several strengths, primarily in relation to stakeholder input throughout the research. First, this study provides a multidimensional understanding of digital health implementation from the perspectives of people with ABI, their close others, clinicians, and leaders in digital health implementation, with some strategies identified specifically by certain subgroups (). In addition, stakeholder prioritization of the study targets enabled research efforts to be directed toward the implementation questions that are most important to end users. Complementarily, stakeholder input included a discussion of preliminary findings from a prior systematic review [ ] and the rigorous integration of the NASSS framework [ ] from data collection to interpretation. This arguably managed the “balance to be struck between a paternalistic overemphasis on the research literature and a disregard of evidence...Selection and tailoring methods are guided by the best available theory and evidence, while preserving the benefits of stakeholder engagement and preference” [ ].
An additional strength of this study was its methodological pluralism. By drawing on the complexity-informed NASSS framework  and methods such as network analysis, additional insights could be gained, which maximized the benefit of the initial list of implementation findings, providing new insights into the nature of and potential prioritization opportunities within digital health implementation for people with ABI. In doing so, this study also contributed an applied example of how a critical realist approach can be used to rigorously manage the conceptual, theoretical, and methodological tensions between complexity and implementation sciences to support the practical implementation of real-world interventions.
This study also has several limitations. First, although a focus on domains 1 to 4 reflected stakeholder priorities, additional implementation considerations and strategies from domains 5 to 7 were not reported, despite there being less research evidence to guide implementation in these domains . With regard to complexity, this limitation is notable, given known interactions between domains 5 to 7 and the results of domains 1 to 4. For instance, professional expectations and regulations in the “wider system” (domain 6) would also interact with the reported results. In domain 6, although “client stories” (strategy 3.2) were directly requested by people with ABI and close others to convey the “value proposition” (domain 3) of the Social Brain Toolkit, the implementation of this strategy is not straightforward when considered in relation to the Code of Ethics of Speech Pathology Australia [ ], which stipulates that all advertising “must not use testimonials / reviews. The use of ‘client stories’ may constitute testimonializing, which is prohibited” [ ].
Second, given that ABI is especially prevalent in low- and middle-income countries , there was a noteworthy lack of representation from these countries, where additional complexities and strategies may have been identified. In addition, the results should be interpreted with consideration of the majority representation of Australian participants. For instance, strategy 3.12 was a recommendation of hierarchical pricing that demands least to no payment from people with ABI and most funding from governments and insurers. The identification of this strategy may have been influenced by the Australian context, in which national Medicare and insurance schemes may increase expectations of government support in a way that differs from countries and markets where self-funding health care is the norm.
Finally, although the exploratory and mechanistic focus of this study is consistent with complexity science and its distinctive quality criteria , the identified implementation strategies and considerations and their interrelationships have not yet been empirically tested or observed and would, therefore, be considered lower on traditional health research evidence hierarchies [ ]. Therefore, future research might consolidate these preliminary findings by empirically testing the identified strategies, considerations, and interrelationships with a more confirmatory aim. This may also eventually allow for more in-depth network analytical methods in future, such as the weighting of certain links based on a deeper understanding of their importance or the use of alternative measures of centrality when digital health implementation mechanisms are better understood. However, it should be noted that a complexity-informed investigation would not expect the findings of this study to be precisely replicable at 2 points in time, given the qualitative nature of the data and the dynamic nature of complexity. Therefore, research resources may be better directed toward more practical implementation efforts, namely user and persona testing (strategy 4.1) for profiles that were not directly represented in this study.
People with ABI, close others, clinicians, and digital health leaders coproduced new knowledge of digital health implementation considerations for adults with ABI and their communication partners, as well as strategies tailored to address them. Several of the identified strategies may be applicable to other interventions for ABI, as well as interventions for conditions other than ABI, addressing current gaps in digital health implementation knowledge more broadly. Although this study was limited by its focus on 4 NASSS domains and the underrepresentation of some participant demographics, the wealth of immediately actionable implementation knowledge generated is aligned with the prospect of including stakeholders in the coproduction of implementation research, with mutually beneficial outcomes for stakeholders and researchers. A complexity-informed theoretical underpinning and a post hoc network analysis revealed that digital health implementation for people with ABI, their close others, and clinicians is a complex phenomenon. Traditional implementation science methods may therefore benefit from leveraging complexity-informed frameworks such as the NASSS and methods such as VNA to more fully understand, better prioritize, and more effectively address digital health implementation needs.
The authors acknowledge and thank all stakeholders who generously shared their time, expertise, and experience. Reimbursement of stakeholders with living experience of ABI and their communication partners, as well as the publication of their input, was funded by a University of Technology Sydney Center for Carers Research grant from the New South Wales Department of Communities and Justice. Transcription costs were also supported by the University of Technology Sydney Center for Social Justice and Inclusion. MM was supported by an Australian National Health and Medical Research Council Postgraduate Scholarship Grant (GNT1191284) and Australian Research Training Program Scholarship. The development of the Social Brain Toolkit was supported by icare New South Wales. LT was supported by an Australian National Health and Medical Research Council Senior Research Fellowship. MM would like to thank Ms Sophie Brassel, Associate Professor Kris Rogers, and all pilot participants for their advice and feedback on the design of the survey. MM would also like to thank Ms Paula Gleeson for her ongoing encouragement and practical support of the project. MM also gratefully thanks Caleb Rixon for his co-authorship of the study protocol and for being an inspiration and encouragement. The authors thank Production Editor Hannah Reinhardt and the JMIR team for supporting innovation in recognizing living experience expertise.
The data sets generated and analyzed during this study are not publicly available to protect the privacy and confidentiality of participants as per human research ethics approvals. However, reasonable requests to the first author will be considered.
AS, BM, MRW, MW, MTM, LW, RM, and J-CE contributed living, clinical, and caregiving knowledge of acquired brain injury and experience of research participation. MTM is a close other of a person with brain injury. AS, BM, MRW, RM, and J-CE are stakeholders with living experience of acquired brain injury. BM is also a Former World Boxing Association World Number 2, International Boxing Federation Super-Middleweight Champion, and Pan Asian Boxing Association Super-Middleweight Champion. MM conceptualized the study; selected the theoretical framework; designed prioritization methods; developed survey, interview, and focus group materials; collected survey and interview data; and initiated and managed coauthorship with stakeholders. EP, RR, LT, DD, and MB provided feedback on the survey design. MM, EP, RR, LT, and DD collected focus group data. AS, BM, MRW, MW, LW, RM, J-CE, MTM, DD, and MM completed data analysis. MM prepared the manuscript, and all authors critically revised the manuscript drafts. All authors approved the final version of the manuscript for submission.
Conflicts of Interest
MM, EP, RR, MB, and LT are developers of the Social Brain Toolkit in collaboration with end users, with no other conflicts declared. All other authors declare no other conflicts of interest.
Full descriptions of all 100 considerations and strategies.DOCX File , 171 KB
- Cieza A, Causey K, Kamenov K, Hanson SW, Chatterji S, Vos T. Global estimates of the need for rehabilitation based on the Global Burden of Disease study 2019: a systematic analysis for the Global Burden of Disease Study 2019. Lancet. 2020 Dec;396(10267):2006-2017 [CrossRef]
- GBD 2016 Neurology Collaborators. Global, regional, and national burden of neurological disorders, 1990-2016: a systematic analysis for the Global Burden of Disease Study 2016. Lancet Neurol. 2019 May;18(5):459-480 [https://linkinghub.elsevier.com/retrieve/pii/S1474-4422(18)30499-X] [CrossRef] [Medline]
- GBD 2016 Traumatic Brain Injury and Spinal Cord Injury Collaborators. Global, regional, and national burden of traumatic brain injury and spinal cord injury, 1990-2016: a systematic analysis for the Global Burden of Disease Study 2016. Lancet Neurol. 2019 Jan;18(1):56-87 [https://linkinghub.elsevier.com/retrieve/pii/S1474-4422(18)30415-0] [CrossRef] [Medline]
- Norman RS, Swan AA, Jenkins A, Ballard M, Amuan M, Pugh MJ. Updating and refining prevalence rates of traumatic brain injury–related communication disorders among post-9/11 veterans: a chronic effects of neurotrauma consortium study. Perspect ASHA SIGs. 2021 Oct 20;6(5):1060-1072 [CrossRef]
- Togher L, Douglas J, Turkstra LS, Welch-West P, Janzen S, Harnett A, et al. INCOG 2.0 Guidelines for cognitive rehabilitation following traumatic brain injury, part IV: cognitive-communication and social cognition disorders. J Head Trauma Rehabil. 2023;38(1):65-82 [https://digitalcommons.chapman.edu/comm_science_articles/26/] [CrossRef] [Medline]
- MacDonald S. Introducing the model of cognitive-communication competence: a model to guide evidence-based communication interventions after brain injury. Brain Inj. 2017 Oct 24;31(13-14):1760-1780 [CrossRef] [Medline]
- Togher L, Hand L, Code C. Analysing discourse in the traumatic brain injury population: telephone interactions with different communication partners. Brain Inj. 1997 Mar 03;11(3):169-189 [CrossRef] [Medline]
- Shelton C, Shryock M. Effectiveness of communication/interaction strategies with patients who have neurological injuries in a rehabilitation setting. Brain Inj. 2007 Nov 03;21(12):1259-1266 [CrossRef] [Medline]
- Togher L, Hand L, Code C. Measuring service encounters with the traumatic brain injury population. Aphasiology. 1997 Apr;11(4-5):491-504 [CrossRef]
- Miao M, Power E, Rietdijk R, Debono D, Brunner M, Salomon A, et al. Coproducing knowledge of the implementation of complex digital health interventions for adults with acquired brain injury and their communication partners: protocol for a mixed methods study. JMIR Res Protoc. 2022 Jan 10;11(1):e35080 [https://www.researchprotocols.org/2022/1/e35080/] [CrossRef] [Medline]
- Hewetson R, Cornwell P, Shum D. Social participation following right hemisphere stroke: influence of a cognitive-communication disorder. Aphasiology. 2017 Apr 26;32(2):164-182 [CrossRef]
- Theadom A, Starkey N, Barker-Collo S, Jones K, Ameratunga S, Feigin V, et al. BIONIC4you Research Group. Population-based cohort study of the impacts of mild traumatic brain injury in adults four years post-injury. PLoS One. 2018 Jan 31;13(1):e0191655 [https://dx.plos.org/10.1371/journal.pone.0191655] [CrossRef] [Medline]
- Ponsford JL, Downing MG, Olver J, Ponsford M, Acher R, Carty M, et al. Longitudinal follow-up of patients with traumatic brain injury: outcome at two, five, and ten years post-injury. J Neurotrauma. 2014 Jan 01;31(1):64-77 [CrossRef] [Medline]
- Douglas JM, Bracy CA, Snow PC. Return to work and social communication ability following severe traumatic brain injury. J Speech Lang Hear Res. 2016 Jun;59(3):511-520 [CrossRef]
- Langhammer B, Sunnerhagen KS, Sällström S, Becker F, Stanghelle JK. Return to work after specialized rehabilitation-An explorative longitudinal study in a cohort of severely disabled persons with stroke in seven countries: The Sunnaas International Network stroke study. Brain Behav. 2018 Aug 18;8(8):e01055 [https://europepmc.org/abstract/MED/30022609] [CrossRef] [Medline]
- Mitchell AJ, Sheth B, Gill J, Yadegarfar M, Stubbs B, Yadegarfar M, et al. Prevalence and predictors of post-stroke mood disorders: a meta-analysis and meta-regression of depression, anxiety and adjustment disorder. Gen Hosp Psychiatry. 2017 Jul;47:48-60 [CrossRef] [Medline]
- Caro CC, Costa JD, Da Cruz DM. Burden and quality of life of family caregivers of stroke patients. Occup Ther Health Care. 2018 Apr 26;32(2):154-171 [CrossRef] [Medline]
- Hua AY, Wells JL, Haase CM, Chen KH, Rosen HJ, Miller BL, et al. Evaluating patient brain and behavior pathways to caregiver health in neurodegenerative diseases. Dement Geriatr Cogn Disord. 2019 Jan 10;47(1-2):42-54 [https://doi.org/10.1159/000495345] [CrossRef] [Medline]
- Schofield D, Shrestha RN, Zeppel MJ, Cunich MM, Tanton R, Veerman JL, et al. Economic costs of informal care for people with chronic diseases in the community: lost income, extra welfare payments, and reduced taxes in Australia in 2015-2030. Health Soc Care Community. 2019 Mar 30;27(2):493-501 [CrossRef] [Medline]
- Hebert D, Lindsay MP, McIntyre A, Kirton A, Rumney PG, Bagg S, et al. Canadian stroke best practice recommendations: stroke rehabilitation practice guidelines, update 2015. Int J Stroke. 2016 Jun;11(4):459-484 [CrossRef] [Medline]
- Togher L, Wiseman-Hakes C, Douglas J, Stergiou-Kita M, Ponsford J, Teasell R, et al. INCOG Expert Panel. INCOG recommendations for management of cognition following traumatic brain injury, part IV: cognitive communication. J Head Trauma Rehabil. 2014;29(4):353-368 [CrossRef] [Medline]
- Chang HF, Power E, O'Halloran R, Foster A. Stroke communication partner training: a national survey of 122 clinicians on current practice patterns and perceived implementation barriers and facilitators. Int J Lang Commun Disord. 2018 Nov 27;53(6):1094-1109 [CrossRef] [Medline]
- Behn N, Francis JJ, Power E, Hatch E, Hilari K. Communication partner training in traumatic brain injury: a UK survey of Speech and Language Therapists' clinical practice. Brain Inj. 2020 Jun 06;34(7):934-944 [CrossRef] [Medline]
- Brunner M, Hemsley B, Togher L, Dann S, Palmer S. Social media and people with traumatic brain injury: a metasynthesis of research informing a framework for rehabilitation clinical practice, policy, and training. Am J Speech Lang Pathol. 2021 Jan 27;30(1):19-33 [CrossRef]
- Gould KR, Carolan M, Ponsford JL. Do we need to know about cyberscams in neurorehabilitation? A cross-sectional scoping survey of Australasian clinicians and service providers. Brain Impairment. 2022 May 05:1-16 [CrossRef]
- Brunner M, Rietdijk R, Togher L. Training resources targeting social media skills to inform rehabilitation for people who have an acquired brain injury: scoping review. J Med Internet Res. 2022 Apr 28;24(4):e35595 [https://www.jmir.org/2022/4/e35595/] [CrossRef] [Medline]
- Acquired Brain Injury Communication Lab. The Social Brain Toolkit. 2023. URL: https://abi-communication-lab.sydney.edu.au/social-brain-toolkit/ [accessed 2023-02-15]
- Rietdijk R, Power E, Attard M, Heard R, Togher L. Improved conversation outcomes after social communication skills training for people with traumatic brain injury and their communication partners: a clinical trial investigating in-person and telehealth delivery. J Speech Lang Hear Res. 2020 Feb 26;63(2):615-632 [CrossRef]
- Togher L, McDonald S, Tate R, Power E, Rietdijk R. Training communication partners of people with severe traumatic brain injury improves everyday conversations: a multicenter single blind clinical trial. J Rehabil Med. 2013 Jul;45(7):637-645 [https://doi.org/10.2340/16501977-1173] [CrossRef] [Medline]
- Togher L, McDonald S, Tate R, Power E, Ylvisaker M, Rietdijk R. TBI Express: Social communication training for people with TBI and their communication partners. Australasian Society for the Study of Brain Impairment (ASSBI); 2010. URL: https://www.assbi.com.au/TBI-Express [accessed 2023-11-01]
- Rietdijk R, Power E, Brunner M, Attard M, McDonald S, Tate R, et al. TBIconneCT clinician manual: connecting people living with traumatic brain injury to conversation training. Australasian Society for the Study of Brain Impairment (ASSBI); 2020. URL: https://www.assbi.com.au/TBIconneCT-Clinician-Manual [accessed 2023-08-11]
- Avramovic P, Rietdijk R, Attard M, Kenny B, Power E, Togher L. Cognitive and behavioral digital health interventions for people with traumatic brain injury and their caregivers: a systematic review. J Neurotrauma. 2023 Feb 29;40(3-4):159-194 [CrossRef] [Medline]
- Feigin VL, Owolabi M, Hankey GJ, Pandian J, Martins SC. Digital health in primordial and primary stroke prevention: a systematic review. Stroke. 2022 Mar;53(3):1008-1019 [CrossRef]
- Christie HL, Martin JL, Connor J, Tange HJ, Verhey FR, de Vugt ME, et al. eHealth interventions to support caregivers of people with dementia may be proven effective, but are they implementation-ready? Internet Interv. 2019 Dec;18:100260 [https://linkinghub.elsevier.com/retrieve/pii/S2214-7829(19)30009-0] [CrossRef] [Medline]
- Greenhalgh T, Abimbola S. The NASSS framework - A synthesis of multiple theories of technology implementation. Stud Health Technol Inform. 2019 Jul 30;263:193-204 [CrossRef] [Medline]
- Verma A, Towfighi A, Brown A, Abhat A, Casillas A. Moving towards equity with digital health innovations for stroke care. Stroke. 2022 Mar;53(3):689-697 [CrossRef]
- Powell BJ, Beidas RS, Lewis CC, Aarons GA, McMillen JC, Proctor EK, et al. Methods to improve the selection and tailoring of implementation strategies. J Behav Health Serv Res. 2017 Apr 21;44(2):177-194 [https://europepmc.org/abstract/MED/26289563] [CrossRef] [Medline]
- McHugh SM, Riordan F, Curran GM, Lewis CC, Wolfenden L, Presseau J, et al. Conceptual tensions and practical trade-offs in tailoring implementation interventions. Front Health Serv. 2022 Nov 17;2:974095 [https://europepmc.org/abstract/MED/36925816] [CrossRef] [Medline]
- Metz A, Kainz K, Boaz A. Intervening for sustainable change: tailoring strategies to align with values and principles of communities. Front Health Serv. 2022 Jan 18;2:959386 [https://europepmc.org/abstract/MED/36925846] [CrossRef] [Medline]
- Christie HL, Bartels SL, Boots LM, Tange HJ, Verhey FR, de Vugt ME. A systematic review on the implementation of eHealth interventions for informal caregivers of people with dementia. Internet Interv. 2018 Sep;13:51-59 [https://linkinghub.elsevier.com/retrieve/pii/S2214-7829(18)30022-8] [CrossRef] [Medline]
- Miao M, Rietdijk R, Brunner M, Debono D, Togher L, Power E. Implementation of web-based psychosocial interventions for adults with acquired brain injury and their caregivers: systematic review. J Med Internet Res. 2022 Jul 26;24(7):e38100 [https://www.jmir.org/2022/7/e38100/] [CrossRef] [Medline]
- Sturmberg J, Lanham HJ. Understanding health care delivery as a complex system: achieving best possible health outcomes for individuals and communities by focusing on interdependencies. J Eval Clin Pract. 2014 Dec 05;20(6):1005-1009 [CrossRef] [Medline]
- Sturmberg JP, O'Halloran DM, Martin CM. Understanding health system reform - a complex adaptive systems perspective. J Eval Clin Pract. 2012 Feb;18(1):202-208 [CrossRef] [Medline]
- Braithwaite J, Churruca K, Long JC, Ellis LA, Herkes J. When complexity science meets implementation science: a theoretical and empirical analysis of systems change. BMC Med. 2018 Apr 30;16(1):63 [https://bmcmedicine.biomedcentral.com/articles/10.1186/s12916-018-1057-z] [CrossRef] [Medline]
- Dadich A, Vaughan P, Boydell K. The unintended negative consequences of knowledge translation in healthcare: a systematic scoping review. Health Sociol Rev. 2023 Jan 20:1-19 (forthcoming). [CrossRef] [Medline]
- Ross J, Stevenson F, Lau R, Murray E. Factors that influence the implementation of e-health: a systematic review of systematic reviews (an update). Implement Sci. 2016 Oct 26;11(1):146 [https://implementationscience.biomedcentral.com/articles/10.1186/s13012-016-0510-7] [CrossRef] [Medline]
- Granja C, Janssen W, Johansen MA. Factors determining the success and failure of eHealth interventions: systematic review of the literature. J Med Internet Res. 2018 May 01;20(5):e10235 [https://www.jmir.org/2018/5/e10235/] [CrossRef] [Medline]
- Varsi C, Solberg Nes L, Kristjansdottir OB, Kelders SM, Stenberg U, Zangi HA, et al. Implementation strategies to enhance the implementation of eHealth programs for patients with chronic illnesses: realist systematic review. J Med Internet Res. 2019 Sep 27;21(9):e14255 [https://www.jmir.org/2019/9/e14255/] [CrossRef] [Medline]
- Papoutsi C, Wherton J, Shaw S, Morrison C, Greenhalgh T. Putting the social back into sociotechnical: case studies of co-design in digital health. J Am Med Inform Assoc. 2021 Feb 15;28(2):284-293 [https://europepmc.org/abstract/MED/33043359] [CrossRef] [Medline]
- Greenhalgh T, Papoutsi C. Studying complexity in health services research: desperately seeking an overdue paradigm shift. BMC Med. 2018 Jun 20;16(1):95 [https://bmcmedicine.biomedcentral.com/articles/10.1186/s12916-018-1089-4] [CrossRef] [Medline]
- Mowles C. Complex, but not quite complex enough: the turn to the complexity sciences in evaluation scholarship. Evaluation. 2014 Apr 10;20(2):160-175 [CrossRef]
- Churruca K, Pomare C, Ellis LA, Long JC, Braithwaite J. The influence of complexity: a bibliometric analysis of complexity science in healthcare. BMJ Open. 2019 Mar 23;9(3):e027308 [https://bmjopen.bmj.com/lookup/pmidlookup?view=long&pmid=30904877] [CrossRef] [Medline]
- Brainard J, Hunter PR. Do complexity-informed health interventions work? A scoping review. Implement Sci. 2016 Sep 20;11(1):127 [https://implementationscience.biomedcentral.com/articles/10.1186/s13012-016-0492-5] [CrossRef] [Medline]
- Thompson DS, Fazio X, Kustra E, Patrick L, Stanley D. Scoping review of complexity theory in health services research. BMC Health Serv Res. 2016 Mar 12;16:87 [https://bmchealthservres.biomedcentral.com/articles/10.1186/s12913-016-1343-4] [CrossRef] [Medline]
- Greenhalgh T, Wherton J, Papoutsi C, Lynch J, Hughes G, A'Court C, et al. Beyond adoption: a new framework for theorizing and evaluating nonadoption, abandonment, and challenges to the scale-up, spread, and sustainability of health and care technologies. J Med Internet Res. 2017 Nov 01;19(11):e367 [https://www.jmir.org/2017/11/e367/] [CrossRef] [Medline]
- Pinnock H, Barwick M, Carpenter CR, Eldridge S, Grandes G, Griffiths CJ, et al. StaRI Group. Standards for Reporting Implementation Studies (StaRI): explanation and elaboration document. BMJ Open. 2017 Apr 03;7(4):e013318 [https://bmjopen.bmj.com/lookup/pmidlookup?view=long&pmid=28373250] [CrossRef] [Medline]
- Haigh F, Kemp L, Bazeley P, Haigh N. Developing a critical realist informed framework to explain how the human rights and social determinants of health relationship works. BMC Public Health. 2019 Nov 27;19(1):1571 [https://bmcpublichealth.biomedcentral.com/articles/10.1186/s12889-019-7760-7] [CrossRef] [Medline]
- The United Nations Educational, Scientific and Cultural Organization. UNESCO Recommendation on Open Science. Nov 2021. URL: https://en.unesco.org/science-sustainable-future/open-science/recommendation [accessed 2021-12-23]
- Jeste DV, Palmer BW, Appelbaum PS, Golshan S, Glorioso D, Dunn LB, et al. A new brief instrument for assessing decisional capacity for clinical research. Arch Gen Psychiatry. 2007 Aug 01;64(8):966-974 [CrossRef] [Medline]
- Fetters MD, Curry LA, Creswell JW. Achieving integration in mixed methods designs-principles and practices. Health Serv Res. 2013 Dec;48(6 Pt 2):2134-2156 [http://hdl.handle.net/2027.42/101791] [CrossRef] [Medline]
- Greenhalgh T, Maylor H, Shaw S, Wherton J, Papoutsi C, Betton V, et al. The NASSS-CAT tools for understanding, guiding, monitoring, and researching technology implementation projects in health and social care: protocol for an evaluation study in real-world settings. JMIR Res Protoc. 2020 May 13;9(5):e16861 [https://www.researchprotocols.org/2020/5/e16861/] [CrossRef] [Medline]
- QSR International. Qualtrics XM. 2021. URL: https://www.qualtrics.com/ [accessed 2021-10-25]
- Teams desktop 1.4.00.16575 (64-bit). Microsoft Corporation. 2021. URL: https://www.microsoft.com/en-au/microsoft-teams/log-in [accessed 2023-08-11]
- Kyngäs H, Kaakinen P. Deductive content analysis. In: Kyngäs H, Mikkonen K, Kääriäinen M, editors. The Application of Content Analysis in Nursing Science Research. Cham, Switzerland. Springer; 2020.
- Microsoft Corporation. Microsoft Excel. 2021. URL: https://www.microsoft.com/en-us/microsoft-365/excel [accessed 2021-10-25]
- Ratnapalan S, Haldane V. We go farther together: practical steps towards conducting a collaborative autoethnographic study. JBI Evid Implement. 2022 Jun 01;20(2):113-116 [CrossRef] [Medline]
- World Wide Web Consortium. Web Content Accessibility Guidelines (WCAG) 2.1. Jun 5, 2018. URL: https://www.w3.org/TR/WCAG21/#dfn-supplementary-content [accessed 2022-05-19]
- Pokorny JJ, Norman A, Zanesco AP, Bauer-Wu S, Sahdra BK, Saron CD. Network analysis for the visualization and analysis of qualitative data. Psychol Methods. 2018 Mar;23(1):169-183 [CrossRef] [Medline]
- Decuypere M. Visual Network Analysis: a qualitative method for researching sociomaterial practice. Qual Res. 2019 Jan 22;20(1):73-90 [CrossRef]
- Packer MJ. The Science of Qualitative Research 2nd Edition. Cambridge, UK. Cambridge University Press; 2017.
- Savage M. Contemporary sociology and the challenge of descriptive assemblage. Eur J Soc Theory. 2009 Feb 01;12(1):155-174 [CrossRef]
- Gephi Consortium. Gephi. 2023. URL: https://gephi.org/ [accessed 2023-08-11]
- Tong A, Synnot A, Crowe S, Hill S, Matus A, Scholes-Robertson N, et al. Reporting guideline for priority setting of health research (REPRISE). BMC Med Res Methodol. 2019 Dec 28;19(1):243 [https://bmcmedresmethodol.biomedcentral.com/articles/10.1186/s12874-019-0889-3] [CrossRef] [Medline]
- Health Consumers NSW. Remuneration and reimbursement of health consumers. 2021. URL: https://www.hcnsw.org.au/for-health-consumer-organisations/remuneration-and-reimbursement-of-health-consumers/ [accessed 2021-05-17]
- Williams O, Sarre S, Papoulias SC, Knowles S, Robert G, Beresford P, et al. Lost in the shadows: reflections on the dark side of co-production. Health Res Policy Syst. 2020 May 07;18(1):43 [https://health-policy-systems.biomedcentral.com/articles/10.1186/s12961-020-00558-0] [CrossRef] [Medline]
- International Committee of Medical Journal Editors. Defining the role of authors and contributors. URL: https://www.icmje.org/recommendations/browse/roles-and-responsibilities/defining-the-role-of-authors-and-contributors.html [accessed 2023-01-13]
- Powell BJ, Waltz TJ, Chinman MJ, Damschroder LJ, Smith JL, Matthieu MM, et al. A refined compilation of implementation strategies: results from the Expert Recommendations for Implementing Change (ERIC) project. Implement Sci. 2015 Feb 12;10:21 [https://implementationscience.biomedcentral.com/articles/10.1186/s13012-015-0209-1] [CrossRef] [Medline]
- Nathan N, Powell BJ, Shelton RC, Laur CV, Wolfenden L, Hailemariam M, et al. Do the Expert Recommendations for Implementing Change (ERIC) strategies adequately address sustainment? Front Health Serv. 2022 Nov 3;2:905909 [https://europepmc.org/abstract/MED/36925827] [CrossRef] [Medline]
- Cherry L. Trauma informed co-production. West Yorkshire Health and Care Partnership; Apr 2022. URL: https://www.wypartnership.co.uk/application/files/6416/5104/5038/West_Yorkshire_Trauma_Informed_Co-Production_Guidance_April_22.pdf [accessed 2023-08-11]
- Lonbay S, Pearson A, Hamilton E, Higgins P, Foulkes E, Glascott M. Trauma informed participatory research: reflections on co-producing a research proposal. Gateways. 2021 Jul 12;14(1) [CrossRef]
- Farr M, Davies P, Andrews H, Bagnall D, Brangan E, Davies R. Co-producing knowledge in health and social care research: reflections on the challenges and ways to enable more equal relationships. Humanit Soc Sci Commun. 2021 May 06;8:105 [CrossRef]
- Schroeck FR, Ould Ismail AA, Haggstrom DA, Sanchez SL, Walker DR, Zubkoff L. Data-driven approach to implementation mapping for the selection of implementation strategies: a case example for risk-aligned bladder cancer surveillance. Implement Sci. 2022 Sep 01;17(1):58 [https://implementationscience.biomedcentral.com/articles/10.1186/s13012-022-01231-6] [CrossRef] [Medline]
- Birken SA, Powell BJ, Shea CM, Haines ER, Alexis Kirk M, Leeman J, et al. Criteria for selecting implementation science theories and frameworks: results from an international survey. Implement Sci. 2017 Oct 30;12(1):124 [https://implementationscience.biomedcentral.com/articles/10.1186/s13012-017-0656-y] [CrossRef] [Medline]
- International Organization for Standardization. ISO 9241-210:2019 Ergonomics of human-system interaction — Part 210: Human-centred design for interactive systems. Jul 2019. URL: https://www.iso.org/standard/77520.html [accessed 2023-01-29]
- Speech Pathology Australia. Speech Pathology Australia. URL: https://www.speechpathologyaustralia.org.au/ [accessed 2022-11-13]
- Advertising. Speech Pathology Australia. URL: https://www.speechpathologyaustralia.org.au/SPAweb/Members/Ethics/Advertising/SPAweb/Resources_for_Speech_Pathologists/Advertising/Advertising.aspx?hkey=bb42be19-1006-46c2-a9c1-9fdfff1dd3f7 [accessed 2023-08-11]
- Iaccarino C, Carretta A, Nicolosi F, Morselli C. Epidemiology of severe traumatic brain injury. J Neurosurg Sci. 2018 Oct;62(5):535-541 [CrossRef]
- Greenhalgh T, Fisman D, Cane DJ, Oliver M, Macintyre CR. Adapt or die: how the pandemic made the shift from EBM to EBM+ more urgent. BMJ Evid Based Med. 2022 Jul 19;27(5):253-260 [http://ebm.bmj.com/lookup/pmidlookup?view=long&pmid=35853682] [CrossRef] [Medline]
|ABI: acquired brain injury|
|CAS: complex adaptive system|
|CPT: communication partner training|
|NASSS: Nonadoption, Abandonment, Scale-up, Spread, and Sustainability|
|TBI: traumatic brain injury|
|VNA: visual network analysis|
Edited by T de Azevedo Cardoso; submitted 10.02.23; peer-reviewed by H Novak Lauscher, N Mungoli; comments to author 20.04.23; revised version received 10.05.23; accepted 03.08.23; published 19.09.23Copyright
©Melissa Miao, Rosemary Morrow, Alexander Salomon, Ben Mcculloch, Jean-Christophe Evain, Meg Rebecca Wright, Marie Therese Murphy, Monica Welsh, Liz Williams, Emma Power, Rachael Rietdijk, Deborah Debono, Melissa Brunner, Leanne Togher. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 19.09.2023.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.