Our refinement process followed an iterative approach involving multiple embedded mixed methods studies, guided by an adapted Information Systems Research (ISR) framework, the Design Thinking Process, and human-centered design (HCD) principles (an approach that prioritizes the needs, preferences, and experiences of end users throughout the design and development process [31,32]. The ISR framework provides a structured approach that guides the development and evaluation of technology-based interventions through iterative cycles of relevance, design, and rigor cycles. The Design Thinking Process emphasizes empathy, ideation, and iterative prototyping to address complex problems. HCD prioritizes the needs, preferences, and experiences of end users throughout design and development. Embedded studies included a randomized controlled pilot study, interviews, usability testing, and surveys with patients and experts to inform ongoing refinements (Figure 1).

Detailed recruitment processes and participant characteristics for each activity are outlined in their respective studies. This document summarizes the overall app refinement process and briefly describes the participant recruitment for each study or activity. For the pilot study, participants were recruited from Hospital Cetrángolo, a public hospital specializing in respiratory medicine in Argentina [10]. Eligible participants were adults (aged 18 or older) initiating treatment for drug-susceptible pulmonary TB, who had access to a smartphone and were able to use it themselves or with assistance from someone in their household. Informed consent was obtained in person prior to participation. Of 56 patients who began TB treatment during the recruitment period, 42 were enrolled and randomly assigned to either the intervention or control arm. The mean age of participants was 36.5 (SD 16.6) years, with equal representation of men and women. For the redesign cycles, TB experts and participants from pilot study provided feedback. For the usability testing cycles 1 and 2, participants were recruited via an email flyer using the University of Washington’s listserv for the School of Nursing and the School of Human Centered Design and Engineering, and from a web-based recruiting platform called Prolific Academic[33]. In usability testing cycle 3, the study was advertised via email flyers and posts on discussion boards for web-based TB communities, with additional participants being recruited from the UW listserv and Prolific Academic .

We conducted initial relevance, design, and rigor cycles to guide the first app version design [34, 27-29]. Refinement activities were informed by findings from the randomized controlled pilot study, feedback from patients and experts, HCD methods, and technical decision-making processes. Details of the study designs, primary aims, participant groups, data collection methods, and duration of each research activity are summarized in Table 1.

The relevance cycle aimed to identify areas for improvement and establish refinement priorities based on the pilot results [32,35]. We analyzed pilot study participant exit interviews and surveys, stakeholder interviews, app use data, and digital messages[10,36]. The participants were followed for the full course of treatment (6 mo); intervention participants used app version 1.1 (2019) to self-report medication adherence, complete daily surveys, and submit photos of direct adherence tests [10,37]. The treatment supporters reviewed daily submissions on the provider-facing app and communicated with participants via WhatsApp to resolve questions and issues. The app’s source code has been published as a “release” on GitHub [38]. This phase identified core usability and cultural adaptation needs based on pilot findings, informing priorities for app redesign in subsequent phases. We also used Hofstede’s Cultural Dimension Assessment to inform development and increase usability based on Argentinian culture [39]. This assessment uses a framework for understanding cultural differences across countries based on key dimensions such as individualism, uncertainty avoidance, and power dynamics, which can inform the design of culturally appropriate interventions [39].

In this phase, we conducted four Directed Research Groups (DRGs) focused on aspects of the app redesign with multidisciplinary teams including experts in HCD, global health, and software engineering. DRGs are faculty- and research scientist–led, quarter-long opportunities for students to apply their skills in hands-on research projects. The first DRG addressed pilot-study-related issues, the second focused on redesigning the patient interface, and the third and fourth DRGs focused on refining the provider dashboard and interface, including designing a mobile-optimized provider dashboard. Designers established a clear visual hierarchy to present essential information consistently across app sections. We referenced the Web Content Accessibility Guidelines 2.0 standards (internationally recognized guidelines that ensure web content is accessible to people with disabilities, focusing on principles of perceivability, operability, understandability, and robustness) and conducted accessibility reviews to identify improvements for enhanced accessibility. Across the DRGs, we developed high-level plans and preliminary design refinements based on issues identified in Phase 1, including an information architecture diagram and low-fidelity design prototypes. We sought feedback on the redesigns from experts in TB, prior study participants, and clinical staff to assess if identified issues were addressed. This phase produced high-level design solutions through iterative prototyping and multidisciplinary collaboration, laying the foundation for a more accessible and user-friendly platform.

Three usability testing cycles were conducted using a mixed methods approach, including think-aloud interviews and standardized usability surveys. Surveys included the Mobile Health App Usability Questionnaire—a validated tool used to assess the usability of mobile health apps, focusing on ease of use, interface design, and user satisfaction—and the Health IT Usability Evaluation Scale—a customizable instrument designed to evaluate the usability of health information technologies based on user perceptions of usefulness, ease of use, and impact on workflow [33]. A software developer built basic prototypes based on findings and feedback, with iterative improvements to the app interfaces made by the DRG and usability testing teams. This process continued until the team was satisfied with the app’s performance. Field testing over 3 weeks involved experts in TB and health care team members. Objectives included training treatment supporters on the TB-TST intervention, identifying technical issues, and preparing for the clinical trial. Treatment supporters and research team members acted as test users, reporting problems, raising questions, and interacting with treatment supporters. This phase validated usability and led to targeted final refinements and readiness for use in the next stage evaluation in a pragmatic clinical trial.

The pilot study (STUDY00007533) received approval from the Institutional Review Board at the University of Washington and the ethics committee of the research site. All participants provided informed consent prior to participating in the research activities. The trial was registered in ClinicalTrials.gov (ClinicalTrials.gov identifier: NCT03544476). The usability testing studies were deemed exempt from full Institutional Review Board review by the University of Washington Human Subjects Division under exemption categories: STUDY00014454 (Categories 2 and 101), STUDY00010640 (Category 2), and STUDY00003401 (Category 2). Ensuring patient privacy and ethical data use was a central consideration in the development of the TB-TST intervention, particularly given its implementation in low-resource settings. The app collects minimal personal information, limited to the user’s phone number, and the app is not publicly available via app stores. It is accessed only through an activation key provided by health care staff, and users must log in to enter the system. All participants underwent a comprehensive informed consent process covering both the digital platform and the urine-based metabolite test. Patients who participated in the pilot study and usability testing received compensation equivalent to US $25 for their time and contribution, in accordance with local ethical guidelines. Data transmission is encrypted, and the intervention was deployed in coordination with local health care teams to ensure compliance with ethical standards and data protection norms. Future iterations will continue to strengthen privacy safeguards and expand user education regarding digital data use.

We present findings organized according to the three phases of the refinement process. Within each phase, we detail key design improvements informed by stakeholder feedback, usability testing, and technical considerations. A detailed mapping of research activities, findings, and corresponding redesign recommendations is provided Table S1 in Multimedia Appendix 1. Textbox 1 shows key improvements made throughout each phase. Table S2 in Multimedia Appendix 1 provides select screenshots of before and after refinement.

This study presents a comprehensive approach to guide the TB-TST intervention’s iterative redesign and refinement process. Most TB-related apps target health care workers or provide general TB information, with few focusing on patients and none supporting patient self-tracking or side effects monitoring [22]. Consequently, this study represents a significant advance in combining a patient-focused support app with a direct drug adherence test, leading to an enhanced digital adherence technology tool to improve TB treatment outcomes. Moreover, the app’s support for multiple languages is a notable strength, as non-English options for TB-related apps are limited in the marketplace [22].

While the ISR framework is typically applied during the initial development phase, we also used it later in this study to guide iterative refinement. To enhance the app’s design, we developed meta-artifacts, such as app flow diagrams, IA, and feature outlines, which were instrumental in re-evaluating and optimizing the placement of various features. In parallel, we considered how the tool could function effectively across diverse global health contexts [31]. Although a common criticism of the ISR framework is that it may privilege researchers’ ideas over user perspectives, our integration of HCD and engineering approaches prioritized user-centeredness, ensuring that the voices of people using the system in real-world environments remained central. Throughout the design process, we consistently aimed to ensure that the app’s development was driven by the needs, preferences, and lived experiences of end users.

Although conducting a randomized controlled pilot study for testing may not be feasible during every design cycle, it was instrumental in gathering valuable user feedback on the app and tools and in assessing the initial impact on treatment outcomes. Following the pilot study, our focus shifted toward addressing critical issues and integrating the needs and preferences of patients and treatment supporters into the app redesign process [41]. We systematically addressed the problems identified and incorporated user feedback into subsequent iterations. Research activities included evaluating app utilization, conducting interviews, analyzing patient-provider communications, and undertaking usability testing. Our HCD approach prioritized ease of use, considering challenges such as limited internet access and varying levels of technological literacy among users. In response to the feedback gathered, we introduced new features to enhance the overall patient treatment experience and to align with World Health Organization guidelines for digital health interventions [42]. This iterative approach enabled continuous refinement of the TB-TST intervention, ensuring it meets the unique needs of patients and health care providers across diverse settings.

The use of targeted DRGs proved to be an effective strategy for guiding research activities. This approach enabled us to draw on diverse perspectives while redesigning the patient app and provider dashboard interfaces. We streamlined these activities to increase efficiency. Open communication channels among stakeholder groups were fostered to optimize processes and facilitate discussions on emerging design prototypes and research findings. We developed onboarding materials to train new team members rapidly and used project management methods to plan weekly activities and target specific outputs. Engaging stakeholders and incorporating qualitative and quantitative feedback is crucial to developing valuable and widely adoptable mobile health apps [43]. We recommend maintaining a consistent core team across cycles, with new members shadowing experienced colleagues before joining a DRG. Establishing clear project objectives and priorities at the outset of each design cycle is essential to managing scope creep [35]. Regular reviews and evaluations of design features, conducted through weekly or biweekly meetings, help ensure alignment with project and user goals.

A commonly cited limitation in digital interventions is their dependence on smartphones, internet access, and digital literacy, which may restrict their usability in low-resource or rural settings—often those most in need of adherence support. However, this concern was proactively addressed through offline functionality, plain language, and supportive videos. Although the original concept was SMS text messaging–based, formative research revealed that WhatsApp had become the predominant communication platform in the study context, prompting a shift in design. During the pilot study, only two participants were excluded due to a lack of mobile phone or internet access. Despite a few individuals being unable to participate due to phone access, it is still critical to incorporate flexible implementation strategies and ensure that future iterations of the intervention are inclusive of users with varying levels of technological access. Although all pilot study participants were invited to in-person exit interviews, not all attended, potentially omitting valuable insights. Re-engaging participants for subsequent iteration evaluations also proved challenging. To mitigate these issues, we conducted supplementary interviews and usability testing with individuals familiar with regular medication use or respiratory conditions. Qualitative findings from the design process highlighted the central role of treatment supporters’ accompaniment and the value of the two-way communication and self-monitoring features. The metabolite test primarily functioned as a prompt for enhanced monitoring and supportive conversations when adherence concerns arose.

Structured usability studies are ongoing to support further app refinement. Lessons from this iterative process have been incorporated into smaller refinement cycles, with all design changes and underlying rationales rigorously documented to enhance transparency and track the app’s development. While this study focused on the iterative design and refinement of the TB-TST intervention, we will evaluate and report the combined and individual contributions of its core components.

This study demonstrates the value of combining multiple embedded mixed methods, guided by the ISR framework and elements of the Design Thinking Process, to refine the TB-TST intervention. Feedback from diverse stakeholders, including experts of TB, patients with TB, and HCD specialists, was instrumental in enhancing usability and addressing key issues. Our engagement with DRGs validated their role in strengthening refinement processes, advancing knowledge, and creating meaningful research opportunities for skilled students. The framework presented here offers a practical guide for future digital health research, highlighting the importance of iterative design in developing effective and innovative interventions. Embracing such approaches can enable researchers and developers to harness the strengths of mixed methods designs to improve health care interventions.