To identify key characteristics that can improve the acceptability of a chatbot for adolescents, it is essential to collaborate directly with this population. For this reason, we convened a YAB comprised of adolescents representative of the end-user population. YAB members can be selected from Step 1 participants (in our case, adolescents living with HIV), prioritizing individuals who were particularly engaged during focus groups—those who openly shared their opinions, demonstrated an understanding of adolescent mental health, and recognized the challenges faced by other adolescents living with HIV while adhering to treatment. Small advisory groups are recommended to be diverse yet compact enough to ensure meaningful participation and sustainability [36,37], and research suggests that groups of 5-7 members optimize discussion, problem-solving, and consensus building [38]. Ultimately, our YAB consisted of 4 females and 2 males, aged 11-19 years (Table 1).

The YAB’s primary task should be to test the chatbot during in-person sessions, providing feedback on its design, content, and functionality, and suggesting ideas for improvement. Sessions should be held at regular intervals, allowing sufficient time for the team to discuss, develop, and test changes between sessions. Each session should last 90‐120 minutes, including breaks, in accessible, comfortable locations for participants.

Adolescents should access the chatbot individually on their own smartphones, when possible, while additional devices can be provided for those without access. After the interaction period, participants should share their recorded opinions and discuss points of agreement and disagreement to identify key modifications for the next version of the chatbot. With guidance from a facilitator, the YAB can then proceed to evaluate the next feature or content area selected for that session. Structured feedback tools may also be incorporated. For example, we developed a Likert-scale questionnaire (1=very dissatisfied and 5=very satisfied) to assess satisfaction with videos, usefulness and clarity of information, ease of use, and overall experience, with additional items added to evaluate specific modules such as self-help activities or pathways for seeking professional support.

Plan a series of group sessions spaced approximately 30 days apart, scheduled at times convenient for adolescent participants. Sessions should be held in a safe, youth-friendly environment. For EVA, we conducted 5 group sessions between August 2023 and January 2024. All sessions took place on Saturdays to accommodate participants’ availability and were held at a nongovernmental organization that promotes health among the lesbian, gay, bisexual, transgender, queer or questioning, intersex, and asexual populations through research and programs focused on the prevention, diagnosis, and treatment of sexually transmitted infections, HIV, and AIDS. During the sessions, YAB members accessed the chatbot through a web link and spent an average of 20 minutes interacting with the content selected for that session, writing down their impressions and ideas on paper.

Because YAB meetings can generate a wide range of ideas and expectations, teams should prioritize refining prototypes to enhance functionality and implement practical solutions to address challenges and suggestions that emerge during the development process.

The first YAB session should introduce the study and foster participant engagement. Group activities, such as icebreaker exercises to introduce the YAB and study team members, and informal conversations, can help to build rapport and establish a trusting environment that encourages adolescents to share opinions, offer critiques, and stay meaningfully involved throughout chatbot development. Establishing trust and supportive group dynamics has been shown to enhance engagement and make digital health interventions more relatable and trustworthy to young people when they are actively involved in cocreation processes, improving the relevance and impact of the resulting resource [39].

The session should also provide an overview of the study goals and planned activities for future meetings and introduce preliminary concepts related to the chatbot’s content, its proposed functionality, and structural design, to gather adolescents’ perspectives and feedback before developing the first version. The session may also include opportunities for the YAB to shape visual and identity-related elements, such as the chatbot’s name and avatar. Inviting adolescents to name the chatbot is especially valuable, as it can strengthen their sense of ownership in the development process.

A brainstorming activity should be conducted with the YAB to discuss key chatbot features, such as confidentiality, health content, and ideas for making the chatbot engaging for adolescent users. For EVA, the YAB provided feedback on the chatbot’s avatar design (a graphic character representing the chatbot), reviewed the script for an introductory video, and decided to name the chatbot “EVA,” in accordance with its main functions.

An important early activity in chatbot development is creating a flowchart that maps the chatbot’s structure to its core themes and objectives. Once key modules are defined, content can be added based on each module’s purpose. The first chatbot prototype, developed after Session 1, should prioritize structure over visual polish or complex interactions, allowing the YAB to explore how the chatbot communicates and what types of interactions are possible. Even a simple version helps adolescents better understand the tool’s potential than explanations or external examples alone. Including basic but functional content enables users to navigate the system and provide informed feedback on information needs, usability, interaction styles, and content organization, which can then be refined in later iterations.

After the first session, our team developed an initial web-based prototype EVA (version 0.1) guided by YAB priorities and with a strong focus on confidentiality: the chatbot only asked users to enter a name and did not store conversations on their devices. Upon entry, 3 buttons directed users to the EVA modules. The “Education” module allowed users to select topics or type queries, though content was limited to 8 mental health and HIV-related terms. The “Linkage to Care” module simulated real-time written communication with a study team member as an example of connection to professional support, while the “Self-Help” module was not yet active.

After a workable, limited-feature prototype chatbot is developed, repeat consultation with the advisory board is essential. Two important reasons for early consultation with the advisory board, albeit with an underdeveloped chatbot, are to confirm the core features of the chatbot planned during Step 1 and to strengthen the collaborative relationship with the advisory board by demonstrating how their input shaped the chatbot programming (and explaining why some features could not be included). For the first chatbot testing experience with a YAB, ensure that access to the chatbot is simple and reliable. Provide direct entry (eg, via web link) and verify beforehand that the platform runs smoothly on common mobile devices. Facilitators should ideally be members of the development team so they can monitor real-time use, identify navigation or performance issues, and resolve technical problems immediately. Stable Wi-Fi access is essential to prevent connectivity disruptions, and backup devices (eg, smartphones or tablets) should be available for participants whose personal devices are incompatible or have limited functionality.

Although adolescents should be given time for independent exploration proportional to the amount of content under review, the first session should also include a guided walkthrough. Early prototypes are often not fully intuitive, so facilitators should briefly explain the purpose, structure, and function of each module before or during use. This helps align expectations and ensures that feedback reflects the tool’s intended design rather than confusion about navigation. The criteria for which suggestions to implement, adapt, or exclude should consider the development team’s technical capacity, available time, and financial resources, and the relevance of each modification to the intervention’s goals.

For EVA’s second session, YAB members accessed EVA (version 0.1) via a web link and explored it for 20 minutes. Afterward, technical issues were documented, and group feedback was collected on content and functionality. Adolescents confirmed that only essential information, such as name and age, should be required to access the chatbot, avoiding sensitive data such as phone numbers and ensuring that conversations are deleted after use.

For content, the YAB prioritized anxiety and depression as key topics and suggested expanding the content to include self-esteem, bullying, eating and nutrition issues, and HIV, along with practical self-help resources (eg, relaxation, social skills, stress management, and coping with negative thoughts). They also recommended using simple language to make the content more accessible for adolescents of different ages, reducing text, and adding more visual elements.

Following each session, the team systematically reviewed the feedback, identifying priority improvements and defining key technical or design challenges. Then, we developed an updated version of the chatbot that incorporated feasible recommendations while maintaining its core structure and objectives (Table 2). To enhance the chatbot’s presentation and navigation, an interactive web environment was created using the Wix (Wix.com Ltd) platform [40]. This transition gave the study team full control over the chatbot’s interface, enabling real-time error corrections during subsequent sessions with the YAB.

For the third session, focus on reviewing the first improved version of the chatbot, highlighting updates made after initial feedback, including content integration, visual design adjustments, usability improvements, and correction of previously detected errors. At this stage, conduct a detailed and paced exploration of each selected chatbot section. Facilitators should guide navigation to ensure all components are reviewed and understood. Use simple guiding questions such as: “What do you think about this section? Is the information easy to understand? What would you change to improve it?” After reviewing each section, hold a brief group discussion before moving on to the next one. This structure allows feedback to remain specific and helps the group reach agreement on priority modifications related to navigation, presentation, and content.

In our project, each YAB member received blank sheets to record ideas and observations while reviewing each section of EVA (version 0.2). These notes were then used during the section-by-section discussions to help participants recall key impressions, compare perspectives, identify similarities and differences, and agree on priority changes for the next chatbot iteration. Participants found EVA (version 0.2) more visually appealing but noted that the initial screen was not intuitive, recommending clearer user instructions (Figure 2). The welcome message and introductory video—developed in the Wix web environment—were well received (Figure 3). While navigation buttons were considered useful, adolescents suggested limiting their number and using them mainly for simple choices, as excessive button use felt stressful, with a preference for more text-based interaction to increase engagement. They also reviewed 2 new videos created for the chatbot and recommended adding subtitles to all videos to improve accessibility and comprehension.

The “Education” module was viewed as clear and useful, though participants suggested a more engaging and optimistic tone, particularly for sensitive topics such as depression and anxiety. They further recommended including short, relatable stories about common adolescent experiences (eg, bullying, school stress, body image, eating concerns, HIV, and home situations) to make the content more meaningful.

Incorporating a quantitative method to monitor progress helps identify which chatbot features adolescents find most and least acceptable, enabling the team to track improvements and prioritize refinements. For EVA, a brief satisfaction scale using Likert-type items (1=very dissatisfied and 5=very satisfied) assessed the videos, the usefulness of information, content clarity, and engagement (Table 3). Additional items can be incorporated in later sessions to evaluate emerging features as the chatbot evolves. Results showed generally positive perceptions of usefulness and educational value, while lower ratings identified issues such as unintuitive design and repetitive navigation, both of which required further improvement.

At this stage of development, once key design principles have been defined—such as appropriate text length, simple and age-appropriate language, the chatbot’s tone when addressing users, preferred input and interaction formats, and priority mental health topics—it becomes easier to expand the chatbot with new modules or content while maintaining coherence with its objectives. Teams should draft scripts for each new section to maintain a structured overview of content and flow, which facilitates revisions and consistency. When sharing scripts with adolescents, adapt them with visual aids (eg, illustrations or diagrams), especially when the text is extensive, as this supports comprehension and allows for more focused feedback. Reviewing scripts before implementation is particularly effective once participants are familiar with the chatbot’s structure.

For this step, prioritize accessibility, intuitive navigation, and engaging presentation. Introduce new ways of displaying information—even for previously reviewed topics—and invite feedback on these interaction formats. Since both the development team and YAB members become increasingly familiar with the chatbot over time, emphasize presenting new or updated content rather than repeating unchanged sections. Temporary navigation shortcuts (eg, quick-access buttons to specific modules) can help streamline testing sessions.

During EVA (version 0.3) testing, quick-access buttons were added to the home screen to streamline navigation during the session. The YAB reviewed a new “Introduction to Depression” module (Figure 4), first annotating feedback on a printed script and then testing its functionality within the chatbot. Alternative navigation formats, including sliding cards, were introduced to improve content presentation and reduce repetitive button use (Figure 5), alongside updated self-help activities and new videos.

Participants reported that the interface felt more intuitive and preferred text-based interaction rather than repeatedly selecting buttons. The videos were well received, though members suggested incorporating additional visual elements. While the new module was considered useful, it was perceived as too lengthy. Extended use of the chatbot also revealed additional programming issues. The YAB recommended expanding educational topics (eg, eating disorders, bullying, and the roles of mental health care professionals). Satisfaction scores indicated improvements in navigation and engagement (Table 3). Final refinements focused on condensing content to enhance readability and resolving remaining technical issues.

In the final validation session, present the chatbot in a format that closely reflects how a new user will experience it. Remove temporary testing features such as quick-access buttons or shortcuts and allow adolescents to navigate the tool under realistic conditions. Provide sufficient time for free exploration of all content; if the material is extensive, organize the review by modules or sections while maintaining a natural flow of use.

After each review segment, collect adolescents’ feedback. Because participants are already familiar with the chatbot, facilitators should probe more deeply into their responses, asking why they hold certain opinions or suggest specific changes. Comparing the current version with earlier prototypes can help identify which modifications were most positively received and which areas may still need refinement.

Validation is considered achieved when YAB members show broad agreement that the content, tone, and functionality are appropriate for the chatbot’s intended purpose and future implementation. To support this, clearly explain how the chatbot will be used in the next phase of the project. If significant suggestions or unresolved issues remain, schedule an additional review session before finalizing the tool.

The YAB approved the overall interface and audiovisual materials, suggesting only minor navigation improvements, such as adding text labels (eg, “Back”) to icon-only buttons to make navigation more intuitive. They responded positively to the shortened “Introduction to Depression” module, noting that it delivered key information without overlapping with other content in the “Education” module. The self-help activities were found to be engaging and clear, particularly an interactive emotion-regulation story. The “Linkage to Care” module was also considered easy to access and use (Figure 6).

Including multimedia components is strongly recommended when developing digital tools for adolescents, as brief audiovisual materials can enhance engagement, comprehension, and information retention, particularly when addressing sensitive topics. Evidence from digital health interventions [21,41] shows that short, structured multimedia content improves user engagement and perceived usefulness in youth-focused eHealth tools. Accordingly, videos integrated into chatbots, or similar applications, should be concise (ideally under 1 minute), use clear and age-appropriate language, and present relatable scenarios or examples.

From a user-centered design perspective, decisions about audiovisual tone, style, and priority topics should be explored in early stages and validated with adolescent advisors before chatbot development begins. Because chatbot content often evolves during development, a practical first step is to create a short introductory video that explains the chatbot’s purpose, structure, and main modules. The design, characters, and visual style defined in this first video are crucial, as they establish the aesthetic and narrative coherence for subsequent audiovisual materials. As content is defined, additional short videos can be developed to introduce key concepts, explain unfamiliar terms, or highlight essential features, such as self-help strategies or linkage to professional care, ensuring alignment between multimedia elements and the tool’s overall objectives.

For EVA, an audiovisual consultant produced 5 animated videos of approximately 30 seconds each (Table 4), based on scripts and visual guidelines from the research team, which were subsequently reviewed with the YAB. The animation style was selected following adolescent consultation and in consideration of the broad target age range (10‐19 years) to ensure relevance and acceptability across developmental stages. The videos were stored in a dedicated YouTube (Google LLC) [42] account and embedded within the chatbot, allowing users to view them directly through the chatbot interface.