This was an external pilot study [14,15] conducted to inform the design and implementation of a separate full-scale trial. The study design was a randomized encouragement trial.

The target population for this study was Kenyan women who had an unmet need for family planning, that is, women who were not using family planning but wished to delay or prevent pregnancy. The accessible population was limited to women with an unmet need living in Bungoma County, Kenya.

The investigational intervention was a digital health marketplace for family planning called Nivi [18]. At the time of the study, any woman (or man) in Bungoma County could send a toll-free SMS text message to the Nivi service to ask a question about reproductive health or trigger a free callback to complete an automated family planning counseling session via interactive voice response. This session resulted in a set of recommended methods that fit the client’s preferences and goals, along with referrals to local public and private providers offering one or more of these methods. After a period of time, clients were prompted to provide details about their experience with family planning providers and were eligible to receive a transportation voucher (approximately USD $2) as a nudge toward behavior change. The investigational intervention remained under active development during the pilot trial. Participants could text or call customer service representatives as needed.

Since the service was available to anyone living in Bungoma County, it was not possible to restrict access and estimate the impact of the service through a randomized controlled trial. In situations like this, a randomized encouragement design can be very effective [19]. In a randomized encouragement design, participants are randomized to receive an invitation or special encouragement to receive an intervention. Not everyone who is encouraged will try the intervention (and some who are not invited will try it on their own); however, as long as those randomly assigned to receive the encouragement—“the treatment group”—try the intervention at a higher rate than those not encouraged—the “control group”—it is possible to estimate the impact of the intervention. This design has been used to study various interventions where two-sided noncompliance is possible [20-23].

In this pilot trial, we randomly allocated the sample of 112 enrolled women to the treatment or control arm (1:1). At the end of the recruitment period, the first author used the blockTools package [24] in R [25] to block randomize by age and baseline indicators of having attended postsecondary schooling, previous use and discontinuation of contraception, and being married or living in a union. One month after the end of the recruitment period, on October 2, 2017, women randomized to the encouragement arm received an invitation via SMS text message to try the service and complete a free family planning screening (plus bonus phone credit of approximately US $2, not conditional on the use of service). Women randomized to the control arm received a different set of messages thanking them for participating in the study; the control messages did not mention the investigational service.

We conducted a follow-up survey between February 14 and March 13, 2018, approximately 4 months after we invited the treatment group to try the service. Participants could complete the survey for free via an SMS text message in their preferred language or choose to receive a free callback from a study enumerator to complete the survey over the phone. Any woman who attempted to send an SMS text message but experienced an error was flagged for enumerator follow-up. The study enumerator was blind to each participant’s assignment until the end of the survey. We sent up to 4 SMS text message reminders from our study shortcode (mask “DGHI”) to study participants who did not reply. Women who completed the survey received an honorarium of KES 200 (approximately US $2) to appreciate their time and effort.

The primary outcome under investigation was self-reported use of a modern method of contraception [26] since the baseline survey. This included women who adopted and subsequently discontinued a method during this period. The reference point for the start of the recall period was the national election conducted on August 8, 2017, several days after the end of the baseline survey. We obtained a binary indicator of attempted service use by querying the system logs for participant phone numbers. If a participant’s phone number was present in the system logs, we coded her as having tried the service.

Because encouragement designs lead to two-sided noncompliance, we planned to use instrumental variables regression to obtain an unbiased local average treatment effect (LATE) of the impact of service use on contraceptive uptake. We used the AER [27] package in R [25] to estimate LATE via two-stage least squares regression. In the first stage, we regressed the indicator of service use on the instrumental variable—a binary indicator of random assignment to the treatment group. In the second stage, we regressed the primary outcome of contraceptive uptake on the predicted values of service use from the first stage regression. Both regressions included baseline controls and the mode of follow-up survey. We used the ivpack [28] package to obtain corrected Huber-White SEs. The results of nonlinear specifications are presented in Multimedia Appendix 3.

For this approach to be valid, the instrumental variable (or instrument) must meet 3 assumptions: (1) The instrument is randomly assigned (independence assumption), (2) the instrument increases use of the investigational intervention, and (3) the instrument only affects the outcome through use of the intervention (exclusion restriction) [29]. We satisfy the independence assumption (1) through the randomized design, and we demonstrate assumption (2) to be true empirically. There is not a direct test of the exclusion restriction (3), but it seems reasonable to assume this is met because the encouragement to try the service did not itself encourage women to adopt contraception or otherwise counsel them on the importance of family planning.

One aim of the study was to test the recruitment procedures and examine the potential for attrition. We based the target sample size for the full trial on the assumption that a sample size of 50 would be needed in an individually randomized trial (25 per arm) to detect a difference in contraception uptake of 30 percentage points between the control group (10%) and the treatment group (40%), given an alpha of 5%, power of 80%, and a one-tailed test. We increased this sample size estimate by a factor of 2.8 to account for the fact that only a subset of the treatment group was expected to uptake the intervention (70%) and that there would be a differential rate of service uptake in the control group that was not encouraged (10%). The inflation factor was 1/(0.7 − 0.1)², producing an adjusted target sample size of 139 [30].

Institutional Review Boards at Duke University and Moi University reviewed and approved this study protocol. This pilot study is registered with ClinicalTrials.gov (NCT03224390).

As shown in Figure 1, we assessed 772 women for eligibility and enrolled 112 women. A total of 660 women were excluded because they did not meet the inclusion criteria; 33.0% (218/660) of excluded women had a met need for contraception.

Table 1 summarizes the characteristics of the enrolled sample. The average age of participants was 24.7 (SD 4.8) years. The majority of women in the study were married or in a union, and two-thirds reported previous pregnancies. The average woman gave birth to 1.6 (SD 1.6) children and desired to have a total of 3.6 (SD 1.3) children. Most women reported an unmet need for spacing, rather than limiting. As is typical of women in Bungoma County, according to the most recent DHS, the women in this study were familiar with family planning methods. Most women indicated that they had recently been exposed to family planning messages in the media, and the average woman said she had heard of 9.6 (SD 2.2) out of 12 methods assessed.

The randomized encouragement design had only a modest effect on the probability of trying the intervention. Four months after the treatment group was encouraged via SMS text message to try the service, 33.9% (19/56) of women in the treatment group initiated a session, compared with only 1.8% (1/56) in the control group. The encouragement did produce a differential rate of uptake of 32.1 percentage points, but the difference was smaller than anticipated.

Table 2 shows the correlates of intervention use among the treatment group. Age was negatively associated with use, which was expected. No other baseline characteristics of participants were significantly associated with use.

As shown in Figure 1, there was a substantial amount of attrition. We obtained follow-up data from 44.6% (50/112) of enrolled participants. Slightly more than half (56.0%, 28/50) of participants who completed the follow-up survey did so via an SMS text message (vs via a phone call with a study enumerator). Table 3 shows that attrition was higher among the control group, but this difference was not statistically significant. Attrition was significantly associated with a few baseline characteristics, including postsecondary education, nulligravida, and the mean number of children born; participants found at endline were more likely to have attended postsecondary schooling, have never been pregnant, and have fewer children. The impact analysis controls for these baseline characteristics and the mode of survey administration. Missing follow-up observations were imputed with baseline values (last observation carried forward), which in this study was no contraceptive use on study entry.

Table 4 presents preliminary evidence of the impact of the investigational intervention on the adoption of contraception. We found that assignment to the treatment group (ie, assignment to receive an encouragement to try the intervention) led to an increase of 12.7 percentage points in the likelihood of contraception use. This is the reduced form estimate (ie, the effect of the invitation—encouragement—on the uptake of contraception). The causal effect of interest, however, is the ratio of the reduced form estimate to the first stage estimate: 41.0 percentage points. This effect is known as LATE, and it represents the average causal effect for women whose use of the intervention was determined only through the random encouragement to try the intervention. In other words, it is the effect of using the intervention on contraceptive uptake. The sign of this estimate appears to be positive, but the CI is wide.

Two additional specifications are presented in Multimedia Appendix 3: (1) ordinary least squares estimates produced without the use of last observation carried forward imputation for missing data and (2) probit regression estimates. In the models based on the subset of complete data (1), the estimates and CIs are slightly wider than those presented in Table 4. In the nonlinear specifications (2), the results are consistent with the linear results presented in Table 4.

This pilot study demonstrates that the proposed recruitment, encouragement, and data collection procedures are feasible, but some modifications are necessary prior to conducting a full trial. Additionally, analysis of the pilot data suggests that the investigational intervention may have a positive effect on contraceptive uptake among women with an unmet need in Kenya, but a full trial is required to replicate the direction of this effect and more precisely estimate the effect size.

During a recruitment period that lasted 4 weeks, we screened 772 women for eligibility, but only enrolled 14.5% (112/772) in the study. At this rate, it would have taken another week to reach our original target sample size. While this approach was feasible in terms of time and resources, it was inefficient in two ways. First, two-thirds of women who were ineligible to enroll did not meet the basic eligibility criteria such as age, residence, and phone ownership. Screening out these women was not time intensive, but we could have eliminated some work and inconvenience to interested women by more clearly stating the criteria on the market stall signage. Second, 1 out of every 3 ineligible women was ineligible because they did not have an unmet need for family planning. To some extent, this was unavoidable because we did not directly recruit women with an unmet need, but rather embedded checks for eligibility in a short screening available to all women in the eligible age range. In a future trial, it may be advantageous to recruit from other subpopulations in addition to open-air markets to increase the probability that the pool of potential participants will have an unmet need. For instance, recruiting from postsecondary institutions would enable us to reach younger, unmarried women who may be sexually active but not using contraception. Postnatal clinics are another potential venue for recruitment as there is a high unmet need among new mothers in this region.

We used a randomized encouragement design to account for expected two-sided noncompliance with treatment assignment. Women assigned to the treatment group received an invitation via an SMS text message to try the intervention, and 33.9% of these women accepted the invitation, a conversion rate that appears to be consistent with SMS text message marketing conversion rates observed in industry [31]. By comparison, 1.8% of control participants tried the intervention. The encouragement led to a differential rate of intervention uptake of 32.1 percentage points, thereby making causal identification possible using assignment to treatment as an instrument.

The intervention uptake rate is important because incomplete uptake requires an inflation of sample size estimates that are based on fixed parameters for power, alpha, and the desired minimal detectable effect size for traditional randomized controlled trials. Another important consideration for the optimal sample size is attrition. In this study, 44.6% of enrolled participants completed the follow-up survey via an SMS text message or a phone call with a study enumerator. We did not collect detailed tracking information from participants during the recruitment process, so we could only invite participants to complete the survey via an SMS text message. In a future trial, it will be important to have the option to conduct in-person follow-up to reduce study attrition. Other studies that relied solely on SMS text message invitations as we did have encountered similar challenges [13].

A third key consideration for sample size calculations is the minimal detectable effect size. In this study, the instrumental variables estimate of the treatment effect was an increase in the likelihood of contraception uptake of 41.0 percentage points among the treatment group members who tried the intervention. This is an approximate standardized effect size of 1.1; however, this is only a point estimate, and 95% CI is wide. While the results suggest that the intervention effect may be positive, the point estimate is not measured precisely. The effect observed in this study is large relative to other SMS text message interventions for health behavior change [13,32,33], so it will be important to use a more conservative estimate to determine the optimal sample size for the full trial.

The main limitation of this study was attrition. While attrition was not significantly associated with treatment assignment, found and unfound participants at endline differed on a few baseline characteristics. The preliminary impact analysis controls for these differences, but selection bias is a concern. Our reliance on self-reported data, while standard for a trial like this, also has the potential for bias.

As this study was conducted in only one, largely rural county in Kenya, the results may not generalize to urban or international markets. Additionally, the study was conducted at a unique and challenging time. A few days after the end of the recruitment period, Kenyans voted in a national election that was ultimately nullified by the Supreme Court. A second election took place on October 26, 2017, roughly 2 weeks after the treatment group was encouraged to try the intervention. Then in early November, a 5-month national nurse’s strike came to an end, and nurses around the country—including the bulk of the country’s family planning service providers—returned to work. In short, the pilot study was conducted during a period of uncertainty, likely distrust of SMS text message marketing amid heavy political advertising, and a significant decrease in the availability of family planning providers. Given these extenuating circumstances, we attempted to follow-up with participants approximately 4 months after the treatment group was encouraged to try the service rather than 1 month as originally planned.

This randomized encouragement design and study protocol is feasible but requires modifications to the recruitment, encouragement, and follow-up data collection procedures.