During this phase (December 2021 to January 2022), core project team members (software development team and VT WIC leadership) envisioned possibilities for the project, set goals and key performance indicators, and identified constraints, assumptions, and essential requirements for project success. Furthermore, to inform all aspects of the project, VT WIC convened an advisory council that included individuals representing the State of Vermont IT staff, VT WIC IT department, local agency WIC staff, and WIC participants. The core project team met with the advisory council to gather their insights on potential users, app benefits and risks, and to receive feedback on the alpha-, beta-, and field-testing plans, as well as evaluation-related questions.

Using this input, the software development team developed “User Personas,” fictional profiles leveraged in user-centered design to describe potential users’ motivations, concerns, needs, and values [32]. User personas are widely used in health care and in the co-design of mHealth and eHealth technologies [32-34] to ensure that the proposed concept and planned functionality align with the needs of diverse potential users, rather than relying solely on the perspectives of those directly involved in the design process. For this project, User Personas were collaboratively designed to represent a wide range of perspectives, including technology access limitations, minimal baseline knowledge of the learning material, and socioeconomic characteristics and experiences not well represented within the core project team or the advisory council. Then, the software development team created accompanying “User Stories,” short feature descriptions written from a user perspective with information on the value each feature may provide to a specific User Persona (Figure 2) [35]. Together, the User Personas and User Stories provided a shared reference point for key partners in this stage and, in later stages, served as an evaluative framework to assess how the product design mapped against the range of qualities, experiences, sentiments, and perceptions of both potential end users and key partner groups in later stages.

Lastly, VT WIC leadership served as subject matter experts to identify key sources of evidence-based content for the app, which included USDA’s WIC Breastfeeding Curriculum and relevant resources from other state WIC agencies, USDA, Centers for Disease Control and Prevention, National Institutes of Health, La Leche League, and the Institute for the Advancement of Breastfeeding and Lactation Education.

This phase was conducted between January and March 2022. The software development team leveraged the findings from preproduction into a wireframing product screen by screen, without aesthetic or brand styling applied (Figure 3). The purpose of a wireframe is to define simple screens that can be quickly and easily changed based on review and feedback [37]. The software development team developed simple screen concepts based on the user stories and presented them to the core project team for review and feedback. The software development team also worked on stylistic development to establish a visual style and aesthetic approach for the product. Various style concepts were created and narrowed down based on the core project team’s feedback.

Together with the findings from the wireframe and style review sessions, the software development team refined and redeveloped the screens to create a clickable mockup to serve as a prototype (Figure 3). The clickable mockups were designed to showcase the desired product experience, intentionally excluding custom user input, final content, and any user data, thereby encouraging additional review. The software development team presented the clickable mockups to the advisory council for feedback early in the development process in order to identify areas for improvement.

Following the wireframe and mockup design phase, the software development team worked to establish an early technical prototype between March and May 2022. In the manner of prototype, test, and evolve, the software development team created a horizontal slice, which focuses on building simple versions of the app features first. It excludes learning material and visuals to focus on the “must-have” features and systems. Once completed, the software development team included the key content selected by the subject matter experts to achieve an alpha level of completion. Software is considered alpha complete at the earliest point at which all core functionality and systems are ready for review by key partners.

Key feedback at this phase included ensuring that games provided flexibility for a variety of human milk feeding approaches, allowing users to complete the introductory, setup questions for the app in an intuitive and easy way, increasing customization options, and leaning into the microlearning approach as an alternative to other digital information approaches. This feedback largely validated the initial wireframe design, while resulting in the design of a mascot with further customization options and game results that were neutral or open-ended rather than having strict right-or-wrong answers in all cases. The software development team applied the feedback from the advisory council to the clickable mockups and used the design as a blueprint as the project moved into the development phase. This early review served to refine and mitigate critical product risks at an early development stage, thus reducing overall project expenses.

The User Personas established a preference for fast, pick-up-and-play experiences that could fit within busy schedules. Thus, the app was designed to operate through short microinteractions within a playlist, with rapid rewards for each interaction and the ability to advance game progress over relatively short durations. Minigames were selected for their recognizability and simplicity, relying on common interaction patterns such as drag-and-drop, matching, image comparison, and multiple-choice. These interaction styles reflect widely used user experience heuristics and were intended to reduce cognitive load while supporting a wide range of human milk feeding learning content.

Based on these design principles, 6 types of minigames were developed: matching games (eg, matching images and text), identifying differences in images, card sorting, and games to identify myths or facts about human milk feeding. When users opened the app, they accessed a “Daily Playlist” consisting of 5 minigames, enabling progress through short, self-contained sessions. User performance was tracked through points awarded for correct responses and deductions for incorrect answers.

At the start of the app experience, users completed the Breastfeeding Attrition Prediction Tool (BAPT) [38], a validated survey to assess readiness and supports around human milk feeding among pregnant individuals routinely used during lactation services by VT WIC [1,39]. The gamification pathway was personalized based on the user’s BAPT score domains, which relate to an individual’s knowledge of human milk feeding, support, and confidence. Each minigame was tagged to 1 of the 3 BAPT domains, and domain scores could range between –50 and 50 points depending on user responses. For the first 10 active days of gameplay, the “Daily Playlist” presented mostly minigames tagged to the user’s lowest domain score. After the first 10 days, minigames were randomly selected from a list of the 10 least recently shown minigames across all domains, and the 10 least recently shown minigames within the user’s lowest domain, supporting both reinforcement and content variety.

To reinforce app engagement, the app’s reward framework was informed by the Quantic Foundries Gaming Motivation Framework [40], leveraging “Completion” and “Achievement” motivation clusters. Completionist users are motivated to finish everything the game has to offer, for instance, complete every mission, find every collectible, and unlock all content. This motivation cluster has the broadest appeal across age and gender and aligns well with pick-up-and-play designs that unfold over multiple sessions. In Daily Drop, completionist engagement was supported through in-game rewards, including “Gems” earned for correct responses and redeemable for mascot costumes and accessories, randomized item unlocks, item rarity, playlist performance progress tracking and scoring, a daily emotion log, a knowledge library, and a minigame library. Additional commercially successful engagement mechanics were incorporated into the app, including streak tracking and a “Challenge Playlist” that allowed users to retry higher-difficulty content.

Alpha testing is an assessment typically done in-house as a quality check to ensure accessibility, usability, accuracy of information, and alignment with technical, legal, and ethical standards [41]. However, to remain consistent with the agile development approach, the software development team invited potential users and key partners between May and June 2022 to respond to an early version of the project to identify potential access and usage barriers that could limit product impact [42]. Alpha testing procedures included conducting 4 focus groups via video call with 2 groups: WIC participants (n=6) and WIC staff, including full-time nutritionists (n=2) and breastfeeding peer counselors (n=1). WIC staff in 5 local offices recruited WIC participants who met the inclusion criteria (aged 18 years or older and pregnant) during routine appointments. If WIC participants were interested in taking part in alpha testing, they provided their contact information, which was shared with the project team. A project team member then reached out to formally invite them to participate. The project team also emailed all WIC staff and breastfeeding peer counselors to invite them to participate in the focus groups.

Each focus group had 1-2 moderators and an observer. Focus groups began with an overview of the project, a demo of the early prototype, and examples of the planned look and feel. Then, focus group participants were asked questions related to (1) experiences with human milk feeding, (2) current technology use and barriers to technology use, (3) preferences on the art style, (4) the alignment of the prototype with project goals and staff and user needs, and (5) overall suggestions for improvements. Finally, focus group participants completed a survey regarding app usability and motivations for future use.

Following alpha testing, the software development team incorporated feedback through weekly development, testing, and review with the core project team between June and August 2022. Changes and updates were checked against the available project scope before being applied. If any changes were deemed out of scope, design work was completed to identify alternative mechanisms to address the underlying feedback. The product gradually achieved greater stability through this process, and its functionality was expanded and refined until it reached a nearly final state. Quality assurance testing was also conducted toward the end of the beta development phase to identify any high-priority issues that need resolution before beta testing [43-45]. Examples of procedures included testing the app with older phones or unstable internet connection and ensuring appropriate functionality for all app features.

Beta testing aims to validate the product in real-world conditions before release, evaluating user experiential learning and interactivity to ensure alignment with the project’s goals [41]. In August 2022, WIC participants (n=12), staff (n=5), and professionals (ie, WIC experts outside of VT WIC, n=5) served as beta testers. Some WIC participants who were alpha testers were invited to participate as beta testers, in addition to new testers. WIC staff recruited new WIC participant testers during routine appointments at 5 local offices, provided they met the inclusion criteria (aged 18 years or older and currently or recently pregnant). If WIC participants were interested in taking part in beta testing, they provided their contact information, which was shared with the project team. A project team member then reached out to formally invite them to participate. WIC participants were involved in beta testing over a 4-week period, which included 2 meetings with asynchronous testing sessions in between.

The first meeting of beta testing was an onboarding session for groups of no more than 4-6 WIC participants. During this meeting, the software development team walked WIC participants through downloading the game onto their smartphones, demonstrated how to use the app, and discussed the testing purpose in this phase. WIC participants were given access to the game and asked to use it for 4 weeks. While testing, data on their app usage was collected and linked to their WIC ID. Collected metadata included time spent in the app, game engagement streaks (ie, consecutive days using the app), and scores in each BAPT domain (knowledge, support, and confidence). At the close of the testing window, the VT WIC team conducted qualitative interviews with each WIC participant via video call. Interview questions asked about testers’ barriers and facilitators to using the product, motivators for future use, and overall experience. After the interview, WIC participants also completed a survey (developed by the core project team) regarding the acceptability and ease of use of the app and changes in perceptions regarding their human milk feeding experience, informed by key feedback from alpha testing.

WIC staff and professionals were invited to participate in beta testing via email. However, the instructions for testing were more flexible for WIC staff and professionals than for participants, encouraging them to explore the app to become familiar with its features and educational framework. Then, WIC staff and professionals who tested the app were invited to complete the same survey as WIC participants. WIC staff did not receive financial incentives in compliance with federal and/or state policies.

The core project team collaborated in a final development phase for approximately 4 months (August 2022 to January 2023) to design upgrades and changes to the app based on the findings of beta testing. These designs were built into the app, and the software development team completed additional iterative rounds to ensure the user-identified issues were addressed. Lastly, the team completed a rigorous quality assurance phase during which the tool was tested on various devices and conditions to ensure the app was stable and ready for the pilot trial (Figure 4).

During this phase, the core project team solidified the data transfer and storage processes, which included establishing the file structure, creating the data dictionary, and developing a data visualization dashboard. The dashboard was vetted with the advisory council and revised based on their feedback. This dashboard was used to track key components of users’ app experience, such as time in app, time per session, and changes in learning scores during the field-testing stage (stage 3).

Lastly, the core project team hosted a 2-part training for WIC staff in preparation for the field-testing stage. Part 1 included introducing the app, its purpose and features, a preview of the clinical report tool, and instructions on how to download and use the test app. Part 2 included providing details on the pilot trial and their role in it, how to promote the app and document its use in WIC’s management information system, and practice using the clinical report tool in breakout rooms.

Analyses were conducted in SAS (version 9.8; SAS Institute). Descriptive statistics were performed to summarize responses to alpha- and beta-testing survey items (via means and SDs) among WIC participants, staff, and professionals.

A thematic analysis [46] of alpha testing focus groups was conducted using a deductive approach. Two members of the software development team developed a codebook and summary template in alignment with focus group questions (behavioral motivations for app engagement, validation of design and mechanical choices, content suggestions, and functionality requests), and summarized the content of focus groups transcripts into the template. To ensure trustworthiness and credibility of findings [47], and alignment with the project goals, themes, subthemes, and exemplar quotes were member-checked with the advisory council and members of the core project team and discussed until consensus was reached.

Survey results indicated that alpha testers perceived the app as very useful for pregnant parents both in rural Vermont and overall, and indicated that “building skills and knowledge” and “having a quick way to get answers to questions or help with challenges” were the most prominent factors to make them interested in using the app (data not shown). Key suggestions from focus groups included (1) allowing app users to complete the BAPT questionnaire within the first couple of days using the app vs completing it during sign-up due to its length, (2) providing educational information after incorrectly answering questions, and (3) adding affirmations throughout gameplay to encourage users and keep the app feeling supportive.

Survey results indicated that all WIC participants and staff considered their app experience as positive (mean scores 5.0-6.0 out of 7), highlighting the app’s potential to improve the breastfeeding experience (mean scores 5.4-6.0 out of 7; Table 1). However, the mean score of 4.0 for the statement “Was the application motivating for you to keep playing over time?” was indicative of neutral motivation for WIC participants. This finding was supported during qualitative interviews, during which WIC participants indicated wanting the app to provide more answer clarity and content diversity. This feedback led to the following improvements: (1) diversification of progress tracking or reward mechanics, particularly emphasizing knowledge growth and expertise development, (2) adjustments to the variety and length of the daily content, (3) improvements to navigation and user experience, and (4) updates to improve user experience on a broader range of mobile devices.

The project team pilot tested the (1) acceptability, feasibility, and usability of Daily Drop among WIC participants and (2) the acceptability and usability of Daily Drop to support lactation services among WIC staff. Between January 2023 and September 2023, 5 WIC local agencies participated in the pilot. WIC participants were recruited using multiple methods. First, WIC staff recruited WIC participants during their routine appointments, provided they met the inclusion criteria (aged 18 years or older, currently or recently pregnant, and had access to a smartphone or tablet, n=339). If WIC participants were interested in taking part in field testing, they provided their contact information, which was shared with the project team. A project team member then reached out to formally invite them to participate, provided them with an electronic study information sheet, and obtained their consent. Second, WIC staff also posted physical flyers at the 5 local WIC offices and sent emails and SMS text message blasts inviting potentially eligible WIC participants. Interested WIC participants could contact the project team to receive an electronic study information sheet and consent to participate.

Upon consenting, WIC participants were invited to download the app on their mobile device and asked to provide information to guide them through gameplay. This information included WIC ID, local agency, preferred term (breastfeeding or chestfeeding), pregnancy status, anticipated due date, and baby’s birth date (if they indicated they recently gave birth). After the initial setup, users were asked to complete the BAPT questionnaire. Additionally, WIC staff responsible for delivering lactation support at participating agencies were invited to participate in the evaluation.

WIC participants were asked to complete up to 3 surveys during the evaluation, including a baseline survey (n=74), a 1-month user survey (n=49; those who played Daily Drop at least once) or a 1-month nonuser survey (n=25), and a 3-month postpartum user survey (n=42). Electronic surveys with embedded consent language were programmed using Alchemer (Alchemer), and links were sent by VT WIC staff via email or SMS text message. Up to 2 reminders were sent for the baseline and 1-month surveys. One reminder was sent for the 3-month postpartum survey following the initial text/email. This manuscript presents results for questions related to acceptability, satisfaction, and usability among app users in the 1-month survey, which were adapted from valid and reliable tools [48-51]. Adaptations were necessary in this context, given that no existing tools captured all the aspects of WIC human milk feeding education, support, and lactation staff-participant interactions.

Moreover, a subset (n=26) of WIC participants who used Daily Drop were invited to an interview at the end of the evaluation period to provide additional insights about their app use. This subset was selected using random stratified sampling based on their app use (n=13 per stratum; stratum 1: low use [fewer than 10 sessions]; stratum 2: moderate to high use [10 or more sessions]). The semistructured interview guide was tested with members of the advisory council and refined based on their feedback. Interviews were scheduled for 30 minutes and digitally recorded and transcribed. Interviews were conducted until saturation was reached (ie, no new emergent themes) [52].

Eligible WIC staff (n=24) were invited to participate in the evaluation via email with an embedded survey link. Staff surveys with embedded consent information were distributed electronically via Qualtrics (Qualtrics) 3 times during the evaluation: once in the early phase after project implementation (January-February 2023; n=16), once at the midphase (July 2023; n=13), and once in the late phase after the pilot concluded (October 2023; n=8). Up to 2 reminders were sent via email to eligible staff who did not complete a survey, 1 week and 2 weeks after the initial email invitation. Similar to WIC participant surveys, staff survey items were adapted from valid and reliable tools [48,53-55] to ensure that all the aspects of WIC human milk feeding education, support, and lactation staff-participant interactions were captured in responses.

Additionally, WIC staff and local agency directors at participating agencies were invited to participate in interviews. A semistructured interview guide, informed by the Reach, Effectiveness, Adoption, Implementation, and Maintenance [56] and the Consolidated Framework for Implementation Research (CFIR) [57] implementation frameworks to assess key implementation aspects (eg, relative advantage, compatibility, complexity, and trialability), was used to guide interviews. Interviews were conducted via video call in the early (March 2023, n=6), mid (August-September 2023, n=6), and late phases (October 2023, n=2) of the implementation period. The interviews were scheduled for 1 hour and were digitally recorded and transcribed. Given that there was a small number of eligible staff, the number of interviews at each phase was dictated by the number of staff willing to participate rather than saturation.

Analyses were conducted in Stata (version 18; StataCorp LLC). Descriptive statistics were performed to summarize acceptability, satisfaction, and feasibility (via frequencies and percentages, and means and SDs) among WIC participants and staff.

A thematic analysis [46] of WIC participants and staff interviews was performed using a deductive approach, with a preliminary codebook developed using CFIR [57] and the Evaluation Framework for Telemedicine [58] prior to analysis. A total of 10 trained qualitative researchers coded interview transcripts, with each transcript being independently coded by 2 researchers using NVivo (version 13; Lumivero). Coders met regularly to discuss coding disagreements, reach consensus, and update the codebook, if needed.

The results of the 1-month user survey indicated that app users enjoyed using it (35/49, 71.4%), appreciated its simplicity (46/49, 93.9%), and improved their breastfeeding learning experience (39/49, 79.6%; Table 2). Users also expressed satisfaction with the app, with 71.4% (35/49) indicating playing Daily Drop was a good use of their time. About 89% (44/49) suggested WIC offices should use Daily Drop. About two-thirds (33/49, 67.4%) of respondents stated that Daily Drop allowed them to learn about breastfeeding more quickly and easily. Qualitative interviews largely supported these findings, with users discussing that it was “fun,” “helpful,” and “trustworthy.” Users mentioned that the app increased their self-efficacy: “It gave me more confidence in my ability because I had no idea how to breastfeed or if I was doing it right or even to know like if my baby was full. It gave a lot of knowledge that I didn’t have before.”

However, some users noted that the app might be less beneficial for individuals with previous human milk feeding experience, as the information could feel repetitive and may not provide as much value compared to first time parents: “I don’t think I would share it with somebody that’s had seven kids and breastfed all of them, but somebody that’s new to breastfeeding or new to just motherhood. I would share it because, like I said, there’s a lot of information on there that you can learn from it.” Lastly, the absence of push notification features and the inability to play more than one playlist per day were highlighted as barriers to using the app: “My only issue was remembering to do it because it didn’t have any reminders, or if it did, I didn’t have them on.”

WIC staff agreed that Daily Drop was an acceptable way to provide WIC services (Table 3). The average agreement score for the usefulness of the app declined over time, but the difference was not significant. Mean interest scores in using Daily Drop feedback for breastfeeding support were neutral in the midphase and low in the late phase. However, feasibility measures declined steadily over time. Early phase scores for learning gameplay feedback on breastfeeding education approached agreement on ease of use, while late-phase scores indicated disagreement. Average scores for understanding reports and experience with Daily Drop reports showed low feasibility for providing client feedback in the clinic.

These findings were further supported during qualitative interviews, where WIC staff mentioned that the app would benefit WIC participants due to its interactive features, which help deliver lactation information in an engaging manner. Staff also appreciated the self-paced nature of the app, which eased the burden of information overload often experienced during lactation support appointments:

Furthermore, staff indicated that BAPT scores and gameplay reports allowed them to provide even more tailored discussions and conversations based on the participants’ needs:

Nevertheless, during mid- and late-phase interviews, staff noted that more comprehensive training on using the app and integrating gameplay in the lactation appointments would have made it more likely for them to use it. Staff’s concerns were less about the app’s design and more about its implementation and their readiness to incorporate it into their routine WIC appointments: