Mothers were randomly recruited from Praeventis, the Dutch National Immunization Register, and three Internet panels. The latter was to assure a high response rate [7]. This gave us the opportunity to assess differential effects of the intervention under (1) more controlled conditions (ie, panel sample) and (2) more naturalistic conditions as will be the case when the intervention has become part of the national implementation strategy (ie, Praeventis sample). This provided a basis for inferences concerning the intervention’s efficacy and effectiveness [18]. This is also why we did not reward the mothers from the Praeventis sample. The panel members received a small financial reimbursement for each survey that they completed (2-3 euros per survey). The amount of money received was panel specific. Extra financial reimbursement was provided to those in the intervention condition who completed all surveys (1-3 euros extra). In total, the participants in the control group could receive between 4 and 6 euros (completing baseline and follow-up), whereas participants in the experimental group could receive between 5 and 9 euros (completing baseline, intervention, and follow-up). Panel members were prestratified by region to ensure geographic diversity.

Power analyses indicated that 1200 mothers per sample (ie, Praeventis and panels) were needed at baseline (600 per arm) to detect a 10% difference in HPV vaccination uptake between the intervention and control group, and small effects on the continuous secondary outcomes (Cohen d=0.10-0.30), with a power of 0.80, a two-sided alpha of .05, and an expected dropout of 30% at the last survey. Given previous experiences [7,8], a response rate of 3% was expected in the Praeventis sample. A total of 36,000 mothers were initially invited to participate via Praeventis and 1200 mothers via the Internet panels.

This study, approved by the Ethical Committee of the VU Medical Center in Amsterdam, was conducted between January 2015 and March 2015. Effectiveness was evaluated by a 2-arm randomized controlled trial (RCT). Intervention effects on HPV vaccination uptake were assessed objectively using Praeventis. Because invited girls were given the opportunity to catch up on their missed HPV vaccinations, complete data on uptake were only available 18 months after baseline (ie, July 2016). The effects on secondary outcomes were examined by two Web-based surveys; at baseline, just before they had access to the experimental education, and at follow-up, just before they received the first HPV vaccination (time intervals around 2 months). Participants in the intervention condition were invited to visit the Web-based intervention between baseline and follow-up. Participants in both arms had access to the universal information about the HPV vaccination as part of the regular invitation for the HPV vaccination.

An invitation letter to participate in the study was sent in January 2015 by postal mail to the Praeventis sample and by email to the panel sample. This letter included information about the study, a link to a secured website, and a unique code for entrance to the baseline survey. The same code was used for entering the follow-up survey and for gaining access to the tailored intervention (only participants in the intervention condition). Securing the website and providing unique codes was done to reduce the risk of spillover effects. The mothers in the intervention condition were explicitly requested not to share the link with others. On the website, the participants were assured of their privacy, confidentiality, and security in handling their responses and were informed that they could withdraw from participation at any time. Participants were then asked to provide informed consent and to give us permission to derive their daughters’ HPV vaccination status from Praeventis. After having provided informed consent, participants were randomly assigned to either the control or intervention condition. A reminder was sent 1 week after the first invitation. One week after the reminder, participants in the intervention condition received an email inviting them to visit the Web-based tailored feedback. Two weeks after this invitation, a reminder was sent. The website could be visited until the invitation to complete the follow-up questionnaire, 8 weeks after the initial invitation for the baseline questionnaire. All participants were given 2 weeks to complete the follow-up survey; a reminder was sent after the first week. These timelines (see Figure 1) were chosen as these fit with the standard procedures for the HPV vaccination in the Netherlands.

The intervention consisted of a website providing mothers with tailored feedback from 2 virtual assistants, one being visualized in Figure 2. In Multimedia Appendix 1, a selection of screenshots of the website is presented. Computer-tailoring was the basic method for change and fitted the outcome of a previously conducted needs assessment indicating that the mothers preferred personalized feedback [8]. Tailoring is a health communication strategy in which messages are individualized to the person’s preferences and needs [19]. Meta-analyses have shown computer-tailored interventions to be more effective than universal interventions in achieving behavioral change outcomes (eg, [20,21]). Examples of change techniques that were used in addition to tailoring were consciousness raising (targeting risk perception) [22], belief selection (targeting beliefs) [23], active learning (targeting knowledge retention) [24], and motivational interviewing (targeting decisional conflict and attitude change) [25]. Before completing the concept intervention, we experimentally pretested three different intervention components [26,27] and conducted three focus groups for pretesting the prototype’s feasibility.

Also, innovative was the use of 2 virtual assistants for delivering the tailored feedback; a mother- and doctor-like assistant. They provided opportunities for two-way interactions and for creating a highly personal experience. The added value of using a virtual assistant over a text and/or picture-based website is that it improves information recall [28], transfer of learning [29], amount of learning [30], self-efficacy expectations, literacy, and behavioral change [31-33].

The website consisted of four menu options: (1) two-sided information about the HPV vaccination, (2) a decisional balance, (3) practical background information, and (4) frequently asked questions (see Multimedia Appendix 1). In the first menu, mothers were able to collect tailored information about the HPV vaccination such as information about the risk of contracting an HPV infection, which may cause cervical cancer, as well as the risks and effectiveness of the HPV vaccine. In the second menu, a decisional balance gave mothers the opportunity to weigh their perceived pros and cons to balance the mothers’ position toward vaccinating versus not vaccinating the daughter (see Figure 2). In the third menu, mothers received practical information such as how and where to get the HPV vaccination and how to talk to their daughter and/or partner about the HPV vaccination.

Composite scores were computed for determinants of HPV vaccination uptake in case the scaled items showed internal consistency (Cronbach alpha >.60 or Pearson r>.64). See Table 1 for an overview of the primary and secondary outcome measures. All scores on scaled items were averaged into a scale because they showed sufficient internal consistency (Cronbach alpha ≥.78/Pearson r ≥.64); Cronbach alpha was used for scales consisting of more than 2 items, whereas Pearson r was used for scales consisting of 2 items; items with an (R) were reverse coded.

Sociodemographics were modeled as background variables (ie, age, educational level, country of birth, and religion). Level of education referred to the highest level the mother had completed. Educational level was classified into low (less than secondary or vocational education), intermediate (secondary through preuniversity education) or high (professional or university education) [7,8]. Country of birth was dichotomized into “Netherlands” versus “other,” as in our sample only 562 (6.97%) of 8062 (100%) mothers appeared to be born in a country other than the Netherlands. Religion was measured by asking the mothers about their religious convictions (Protestant, Roman Catholic, Muslim, Jewish, Buddhist, Hindu, other, or no religion). This was later classified as “Protestant” versus “not Protestant” as Protestants most refrain from vaccination compared with other religious or nonreligious groups in the Netherlands [7,8].

Subjective program evaluation was assessed at follow-up for mothers in the intervention condition by asking them to evaluate the website and the virtual assistants on a 10-point scale; the higher the score, the more positive the evaluation. Objective program use was evaluated by computer logs assessing the number of visits and amount of time logged in per session. If participants logged in more than once, these were summed.

Descriptive statistics were used to describe the baseline sample. For analyzing the effects of the intervention, we used intention-to-treat (ITT) instead of complete case analysis. By using ITT, power increases while the risk of bias possibly caused by selective dropout decreases [42]. To deal with missing data, we applied multiple imputation by chained equations [42,43]. There were 15 imputed datasets generated using the predictive mean matching algorithm in Statistical Package for the Social Sciences (SPSS, IBM Corp) . The results from the imputed datasets were pooled together using Rubin’s rules [37]. Convergence of the imputations was checked by inspecting the iteration plots.

Intervention effects were examined by logistic and linear regression analyses (for dichotomous and continuous variables, respectively) by using the outcome at the follow-up as the criterion and the outcome-score at baseline and condition as the independent variables [44]. In view of multiple testing, an effect was considered significant when P<.003 (Bonferroni corrected alpha=.05/15 factors). The odds ratio was used as an indicator for effect size (Bonferroni corrected alpha=.05/14 factors). Effect sizes for the linear regressions were calculated in R (R Development Core Team) [45] using Cohen ƒ² statistic, (R²_AB – R²_A)/(1 – R²_AB), in which B is the variable of interest (ie, condition), A is the set of all other variables (ie, the outcome at baseline), R²_AB is the proportion of variance accounting for A and B together, and R²_A is the proportion of variance accounted for by A. These were interpreted as 0.02=small, 0.15=medium, 0.35=large [46]. Complete case analyses were performed as a sensitivity check for substantial differences with the results based on ITT.

Furthermore, we performed exploratory moderation analysis (Bonferroni corrected alpha=.05/15) to examine differences in effects regarding sociodemographics (ie, age, country of birth, education level, and religion) and sample (ie, Praeventis vs panels) by including a two-way interaction term (eg, condition × sample) in each of the aforementioned regression analyses [47]. In addition, we explored whether intention (at baseline) moderated the effects found in the primary and secondary outcomes. For this, intention was divided into three subgroups: (1) mothers with a negative intention (scores below half a standard deviation (SD) below the centered mean score of intention at baseline), (2) mothers who were hesitating (scores between half an SD below and above the centered mean of intention at baseline), and (3) mothers with a positive intention (scores more than half an SD above the centered mean score). Finally, subjective evaluations and objective use of the program were assessed by using descriptive analysis. IBM statistical package SPSS version 23 was used for analyzing the data [44].

We invited 36,000 participants via Praeventis and 2483 via the panels. A flow diagram of the recruitment and response is shown in Figure 3. From the 9124 participants who were initially randomized at T0, 8593 (94.18%; 4277 in the intervention group and 4316 participants in the control group) completed the baseline questionnaire, whereas 4678 (51.27%; 2197 in the intervention group and 2481 participants in the control group) completed the follow-up questionnaire 8 weeks later (T1). Dropout analysis showed significantly more dropout in the Praeventis sample. There was also selective nonresponse with regard to condition (ie, more dropout in the intervention condition), sociodemographics (ie, more dropout in those not born in the Netherlands, and in those low in education), HPV vaccination uptake (more dropout in mothers having a daughter not being vaccinated), and secondary outcomes (ie, more dropout in mothers with low levels of IDM, risk perception, and self-efficacy and in mothers with high attitude scores). In total, 1067 participants were excluded (564 in the intervention group and 503 participants in the control group), as they did not meet the inclusion criteria (ie, being a mother of a daughter born in 2002 and aged 24-62 years) or were found to be duplicates across the two samples. The final sample for ITT analysis consisted of 8062 mothers: 3995 mothers in the intervention condition versus 4067 in the control condition.

See Table 2 for the sample description. As there were no data available on sociodemographics of the population from which the sample was derived (ie, Dutch mothers of girls aged 12 years in the Praeventis database), we were unable to assess the representativeness of the study sample. The mean age of mothers was 43.64 years (SD 4.25). On average, mothers had a positive intention toward the HPV vaccination of their daughter at baseline (mean 5.35 [SD 1.69]). Compared with the national HPV vaccination uptake, uptake was higher in the study sample (n=59,866; 60.98% vs n=5880; 72.93%, respectively).

In Table 3, an overview of the intervention effects on the primary and secondary outcomes is given. A higher mean score means a higher X (eg, more positive attitude) except for decisional conflict; here, a higher mean score means less decisional conflict If an odds ratio (OR) is higher than one, this means that the higher the score on a factor, the higher the outcome of IDM or the higher the chance of the daughter being vaccinated. If an OR is less than one, this means that the higher the score on a factor, the lower outcome of IDM or the lower the chance of the daughter being vaccinated. ITT analyses showed that there was no effect of the intervention on HPV vaccination uptake (odds ratio, OR=1.03, P=.60). The intervention had a significant positive effect on all secondary outcomes (P<.001), except for risk perception when not vaccinated, anticipated regret, and self-efficacy (P=.01; P=.01; P=.03, respectively). Compared with the control group, at follow-up, mothers in the intervention group were more informed (dichotomous measure: OR=1.28, P<.001; continuous measure: beta=1.72, P<.003), experienced less decisional conflict (beta=.21, P<.003), were more intended to vaccinate their daughter (beta=.18, P<.004), had a more positive attitude toward vaccinating their daughter (beta=0.15, P<.004), had more positive beliefs (eg, beliefs about the safety and effectiveness of the HPV vaccination; beta=.12, P<.003), had a lower risk perception when they imagined that their daughter was vaccinated (beta=−.11, P<.003), perceived more positive subjective norms (beta=.82, P<.003), reported a higher relative effectiveness (beta=.46, P<.003), and had more knowledge (beta=.35, P<.003). Effect sizes were small (see Table 3). Results from complete case analyses were similar, except for an additional effect of the intervention on anticipated regret and self-efficacy.

Regarding sociodemographics, no significant interaction effects on any of the outcome measures were found for country of birth (P ≥.08) or religion (P ≥.08). For educational level, we found an interaction effect with condition on relative effectiveness (beta=.59, P<.001): the intervention had more positive effects on relative effectiveness for those with high education compared with those low in education. There was no significant interaction between condition and sample on any of the outcome measures (P ≥.04). For the interaction effects between intention at baseline and the outcome measures, See Table 4. If an OR is higher than one, this means that the higher the score on a factor, the higher the outcome of IDM or the higher the chance of the daughter being vaccinated. If an OR is less than one, this means that the higher the score on a factor, the lower outcome of IDM or the lower the chance of the daughter being vaccinated. In the first 2 columns, the reference category is those with a negative intention. For a comparison between those in doubt (reference category) and a positive attitude, see the third column. Significant interaction effects between intention at baseline and condition were found on intention, attitude, decisional conflict, subjective norm, and relative effectiveness . For mothers who had a negative intention, the intervention had more positive effects on intention and relative effectiveness compared with mothers who were doubting (beta=.26, P=.002; beta=.39, P=.001, respectively) or had a positive intention (beta=.40, P<.001; beta=.53, P<.001, respectively). In addition, for mothers with a negative intention, the intervention had a more positive effect on attitude (beta=.21, P=.001) and on subjective norms (beta=1.64, P<.001) compared with mothers with a positive intention. For mothers who were doubting, the intervention had more positive effects on decisional conflict compared with mothers who had a negative intention (beta=.26, P=.001). No differences on intervention outcomes were found between mothers who were doubting and mothers who had a positive intention (P ≥.004).

Mothers in the intervention condition evaluated the website with a 7.6 (SD=1.36) and the virtual assistants with a 7.4 (SD=1.53). According to the computer logs, 2509 (62.80%) of the 3995 (100.00%) invited mothers logged on to the website. Of these, 1835 (73.14%) visited the website once, 498 (19.84%) visited twice, and 176 (7.02%) more than twice. On average, mothers spent 22 min on the website (SD=13 min).

This study investigated the effectiveness of a Web-based tailored intervention with virtual assistants promoting HPV vaccination acceptability among mothers of invited girls. As hypothesized, positive intervention effects were found with respect to the social cognitive determinants of the mothers’ decision making about the vaccination (eg, HPV vaccination-related intention, attitude, and outcome beliefs), levels of IDM, and levels of decisional conflict. The positive effect of tailored education on HPV vaccination intention was also found by Gerend et al [12] among young women. However, they did not assess other determinants of HPV vaccination acceptability (next to intention) nor did they measure levels of IDM, levels of decisional conflict, or actual HPV vaccination uptake.

The findings described above suggest that this intervention has potential in promoting HPV vaccination acceptability and IDM. This is important, given the currently moderate HPV vaccination uptake and the fact that large proportions of the mothers do not actively acquire and process information about the pros and cons of this HPV vaccination and that many feel ambivalent about the decision [7,8]. Less informed decisions are decisions constituted in rather instable beliefs that are susceptible to counterarguments. Nowadays, counterarguments are all around on the Internet and Web-based social media [48]. Because the intervention initiated active processing of verifiable information about the risks and effectiveness of the HPV vaccination, it inoculates mothers with arguments that become accessible at the moment they are confronted with (new) information that might challenge their initial positive attitudes and intentions [49,50].

No effects were found on mothers’ perceived risk of their daughter getting cervical cancer without the HPV vaccination, anticipated regret in case their daughter would get cervical cancer later in life, and self-efficacy. As for risk perception, baseline scores indicated that the mothers overestimated the probability of contracting cervical cancer to a great extent when taken into account the actual population incidence [51]. Because the intervention presented mothers this actual low probability of attracting cervical cancer, it seems unlikely that their perceived risk was brought to higher levels. The lack of effect on anticipated regret might be explained by the fact that we removed the intervention component specifically targeting anticipated regret. This was removed because our pilot studies and focus groups revealed that resistance was evoked by asking mothers how much regret they would have if they did not vaccinate their daughter against HPV and their daughter developed cervical cancer later in life. Also, emphasizing the impact of cervical cancer might be fear-arousing, which, in turn, may have been detrimental for exploring and processing other information provided by the program [52]. Finally, it appeared that we encountered a ceiling effect for self-efficacy as the scores at baseline among both groups were above 6 on a 7-point scale.

No effects of the tailored intervention were found on HPV vaccination uptake. This is contrary to both our expectations and to what has been found by others; Hopfer found that among female college students, HPV vaccination uptake doubled after they were exposed to a tailored video compared with controls (22% vs 12%, respectively) [11]. However, in their study, vaccination rates were quite low in the control condition (12%), leaving much room for improvement. In our study, however, the uptake rates were high in both conditions (intervention: n=3995; 73.17% and control: n=4067; 72.71%) especially when compared with the national Dutch uptake (n=59,866; 60.98) [6]. This may explain why we, as opposed to Hopfer, did not find an effect on HPV vaccination uptake. After all, we did find a larger increase in intention among mothers in the intervention compared with the control condition, and according to theory [23] and empirical findings [53], intention is an important predictor of (HPV vaccination) behavior.

Mothers evaluated the intervention as positive, specified by the high subjective evaluation of both the website (7.6 on a 10-point scale) and the virtual assistants that were used to deliver the tailored feedback (7.4 on a 10-point scale). Objective program use was also high, with 62.80% (n=2509) of the invited mothers having visited the website. In addition, on average, they spent quite some time on the website (22 min). Taken together, the intervention has potential for broad national dissemination and implementation.

Furthermore, subgroup analysis with 3 intention groups (ie, negative, in doubt, and positive) showed that the intervention had the most positive effects on decisional conflict for mothers who were doubting. For mothers with a negative intention, the most positive effects were found on intention, attitude, and subjective norms. This is promising, as for a population-wide program it is relevant to guide those in doubt toward making an informed choice without decisional conflict and to persuade those having negative intentions toward vaccination. Fortunately, we did not find any adverse effects of the intervention in mothers with a negative intention, such as a further decrease in their intention or attitude.

There are two methodological considerations. First, with large sample sizes, as in this study, even small effects can become statistically significant [54]. However, the positive intervention effects were consistently found on almost all outcomes. In addition, the large sample provided us with sufficient data for conducting analyses on subsamples (ie, based on sociodemographics, sample, and three intention groups) while maintaining sufficient levels of power [54]. The effect sizes that we found are in line with other Web-based interventions targeting health behavior outcomes [55]. We believe these, even small effects, are of relevance in public health as they become substantial at the population level. In addition, the intervention was of help for those in doubt and did not have any detrimental effect. We therefore find this Web-based tailored program is a substantial step forward in improving both research and practice in the context of the promotion of HPV vaccination acceptability.

Second, we used a scale comprising two items of the Self-Report Habit Index for measuring habit [41]. This accounted for the extent to which getting the HPV vaccination was something mothers did (1) naturally and (2) without thinking. The positive intervention effect on the composite measure might indicate that the intervention induced perceptions about the HPV vaccination as something you take for granted, without thinking. The latter is unwanted, considering the aim of initiating active processing of information about the vaccination. Fortunately, secondary analysis separating the two items showed that there was a positive effect of the intervention on the first item (ie, “naturally”) but not on the second (ie, “without thinking”). In retrospect, the label “habit” attributed to the two-item operationalization appears misleading, though the intervention strengthened the mothers’ belief about getting the HPV vaccination as something natural.

Important strengths of this study are the randomized controlled design, adequate sample sizes, and the reliable objective assessment of HPV vaccination uptake. Furthermore, the systematic, stepwise development of the intervention was a notable strength, as well as the mothers’ positive subjective evaluation of the intervention and the objectively assessed high level of program use.

Some limitations of this study should be considered as well. First, the study was subject to a considerable amount of attrition. Unfortunately, attrition is quite common in studies on Web-based interventions [56]. Dropout analysis showed that dropout was selective. For instance, there were higher dropout rates in the intervention condition, which has also been reported for other (tailored) interventions [57-59]. In this study, we handled the missing data and selective dropout by using multiple imputation [60]. Results from the complete case analyses only slightly differed from those from the ITT analyses. Thus, it seems unlikely that the observed effects are spurious or due to selective dropout. Second, caution is needed when generalizing the results of this study to the general population (ie, Dutch mothers of girls aged 12 years) because we were unable to check the sample’s representativeness. However, we did not find any differences in effectiveness of the intervention in specific subgroups of participants, as indicated by the conducted moderation analyses with sociodemographics.

The study findings suggest that this Web-based tailored intervention has the potential to improve both HPV vaccination acceptability and IDM, and decrease decisional conflict among mothers of invited girls. Therefore, we recommend nationwide dissemination and implementation. Furthermore, we recommend developing (tailored) interventions targeting the daughter and the mother’s partner, as these have appeared to be important social referents [7,8]. Research has indicated that tailoring the intervention could have beneficial effects for girls, as they also expressed their need for interactive and personal information about the HPV vaccination [8]. This still needs to be investigated for the partners. In addition, boys may also become a relevant target group. In other countries, such as Australia, boys are already included in the national immunization program. This may contribute to the achievement of herd immunity and to a reduction of the global burden of a variety of HPV-related cancers in women and men [61-63].