This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.
Health messages may have the highest impact if they are given early in life. Therefore, the Dutch Ministry of Health identified pregnant women as a target population and initiated an innovative eHealth program to serve as a communication channel of health messages to pregnant women.
The aim was to study the feasibility of implementing this eHealth program into standard midwifery care and to investigate use and user perceptions of the online program among pregnant women.
All midwifery practices in Amsterdam affiliated with the Dutch Organization of Midwives (n = 25) were requested to implement the eHealth program within a pilot study from March to August 2006. Structured interviews were used to study feasibility of implementation among midwives. During the study period, 488 pregnant women registered themselves on the program website, after which monthly emails were sent to them. The emails were tailored to the stage of pregnancy and provided interactive questions plus answers on six topics and links to related websites. User statistics were registered until January 2007, and user perceptions were assessed with electronic questionnaires.
In total, 80% (20/25) of midwifery practices implemented the program. These midwives gave a short oral explanation about the eHealth program to their clients (n = 1382; about 45% of the total number of clients during this period) and handed out an information brochure. After the pilot, 12 midwifery practices were willing to integrate the eHealth program into their standard care procedures. Regarding program usage, 84% (408/488) of the enrolled women accessed health information within the program. They opened 59% (1296/2213) of the quiz emails and accessed, on average, 16 topics (SD 11). Only 35% (143/408) of users used the hyperlinks to visit related websites. Most women perceived the eHealth information as easy to understand (96%; 157/163) and reliable (81%; 130/161), but only 39% (48/153) agreed that the information was available at the right time. Accessing more topics within the quiz emails was associated with a more positive perception of the program (
It is feasible to integrate an innovative eHealth program in standard midwifery care, and about half of the practices would like to continue doing so. Program users accessed a substantial proportion of available health information; however, user perceptions were mixed. Therefore, this eHealth program may be a feasible communication channel to promote a healthy lifestyle to pregnant women after suggested revisions have been carried out.
Most women welcome health-related information during their pregnancy and search for pregnancy-related information on the Internet [
In the Netherlands, 85% of pregnant women start their maternity care at a midwifery practice [
We have performed a pilot study among almost 1400 pregnant women to study the feasibility of integrating the eHealth program into standard midwifery care. The process evaluation of this study is carried out in line with the RE-AIM model [
Therefore, the purposes of this paper are (1) to study the feasibility of implementing an online healthy lifestyle program into standard midwifery care, including midwife perceptions of the program, and (2) to analyze use and user perceptions of the eHealth program among pregnant women. The results of both objectives served to further improve the program before its countrywide implementation.
The program consisted of monthly emails containing a link to a quiz, with a maximum of six questions, tailored to the stage of pregnancy. The quizzes were stored on a website. On the program website women could register for the program and find more information about it. However, health-related information was only accessible through the emails. The content of the intervention program was especially developed for women with a low education level, by presenting the information in plain language and short text blocks, thus requiring only basic literacy skills. Five health-promoting institutes were responsible for the content of the topic of their relative expertise (nutrition, exercise, lifestyle, smoking, or safety), and the Dutch Organization of Midwives was responsible for the content of the pregnancy topic. In this way, existing pregnancy-related information was brought together within one program.
Each email was personalized with the first name of the participant and the number of weeks she was pregnant, and it contained one question from the quiz. The participant was invited to click on the specified link to find out the answer to that question. Clicking on this link automatically opened a new screen with the quiz.
After opening the quiz screen, the six topics were displayed (
Screenshot of the quiz
In the Netherlands, midwives are independent paramedical practitioners, qualified to provide full maternity care to women whose pregnancy and childbirth are uncomplicated. The primary task of the Dutch midwife is to monitor the health status of pregnant women and their unborn children. Because of their frequent contacts throughout pregnancy and because of their expertise, midwives are considered to be important information providers for pregnant women, especially for women pregnant with their first child. Therefore, midwifery practices are a unique platform to inform pregnant women about the existence of an eHealth program with relevant and reliable pregnancy-related information. Furthermore, it is likely that information about the eHealth program could easily be incorporated into standard care since general health and lifestyle information is already given during the first midwifery visit.
All 25 midwifery practices in Amsterdam affiliated with the Dutch Organization of Midwives were requested to implement the eHealth program from March to August 2006. They were instructed to give oral information about the program and hand out an information brochure to all of their clients before the 16th week of pregnancy in order to motivate their clients to enroll in the eHealth program. The midwives informed 1382 women about the eHealth program, and 238 women (17%) subsequently registered themselves (at home) on the program website, as described in more detail elsewhere [
In July 2006, the 25 midwifery practices were contacted by telephone to make an appointment for an interview. At this stage we checked if the practice had implemented the eHealth program. Practices that did not (actively) implement the program were asked a few short questions to discover why they did not participate. For the participating practices, a date was set for a structured interview. Preferably, the interview was conducted face-to-face within the midwifery practice with one or more midwives; when this was not possible, the interview was done by telephone. One midwifery practice was unavailable for the interview due to time restraints and personal circumstances of the midwives. However, the practice assistant gave a general impression of the implementation. The questions within the interview were categorized into four sections: implementation, program perception by midwives, points for improvement, and some general information about the midwifery practice. It took approximately 15-30 minutes to complete the interview. The answers of the midwives were summarized by the interviewer and manually recorded during the interview. To estimate the delivery rate of the program within the midwifery practices, the midwives were asked to record each client who was informed about the program on a specifically designed card. Furthermore, during the interview, the midwife gave an estimate of the number of clients visiting the practice.
User statistics were registered until January 7, 2007, by a third party specialized in Internet technology (OOiP, The Netherlands) and included enrollment data, emails sent, quiz questions accessed, practical tips accessed, and related websites visited through hyperlinks within the program. A quiz email was considered as opened if at least one question was accessed. A self-completed questionnaire was used to assess participant characteristics at baseline.
An online self-administered questionnaire was used to measure user satisfaction with the eHealth program. Invitations to complete the questionnaires were sent by email the week after the participants received the third quiz email, but no invitations were sent after the 40th week of pregnancy. The feedback questionnaire contained several statements about the program that could be rated on a five-point scale (“totally agree” to “totally disagree”). Participants could also indicate which of the six lifestyle topics they enjoyed most and found most useful, and overall satisfaction was rated with a score between 1 (“very bad”) and 10 (“very good”). The questionnaire ended with an open-ended question (“What could we do to improve the program?”). The participants who gave feedback were divided into three groups based on their scores on five user perceptions of the program: interest, usefulness, timing, pleasantness, attractiveness (see last five statements in
Upon enrollment, a unique personal identification number (ID) was assigned to each program user. The data from the baseline and feedback questionnaires were manually linked to an ID by means of name, zip code, and email address.
Descriptive analyses using means, standard deviations, medians, and percentages were used to describe program use and user attitudes. Chi-square tests were used to detect associations between participant characteristics and user attitudes. Analysis of variance (ANOVA) with follow-up trend analyses was used to detect associations between user satisfaction and program use. Furthermore, linear regression analyses were used to examine the association between five individual perception components (interest, usefulness, timing, pleasantness, attractiveness) and program use (dependent variable). SPSS version 12.0.1 (SPSS Inc, Chicago, IL) was used for all analyses, and statistical significance was set at a level of 0.05.
In the pilot project, 80% (20/25) of midwifery practices informed their clients about the program. The other practices did not implement the program, mainly because they considered the program unsuitable for the majority of their clients (highly educated or non-Dutch-speaking women). One practice reported time constraints as the main reason.
The midwives in the 20 participating practices informed their clients about the eHealth program during a standard visit. They all handed out an information brochure, and, with the exception of one midwifery practice, they all gave a short oral explanation about the eHealth program (
Suggestions to facilitate further implementation of the program were to use more promotion materials and to reconsider the moment most appropriate for giving information about the eHealth program.
About half of midwifery practices (12/25) wanted to integrate the eHealth program into their standard care after the end of the study period. Five practices that did not implement the program during the study period would also not implement it after the study period. The other eight practices would not like to continue implementation and had more negative program perceptions than midwives from practices with a positive intention to continue implementation (
Practice characteristics, program implementation characteristics, and program perceptions of midwifery practices, stratified by the intention to integrate the eHealth program into standard midwifery care after the study period
Not Implemented |
Negative Intention |
Positive Intention |
|
|
|||
Method of interview | |||
Face-to-face | 0 | 4 | 7 |
Telephone | 5 | 4 | 4 |
Estimated births in study period, mean (range)b | 97 (25-170) | 210 (150-335) | 137 (20-195) |
Client ethnicityb | |||
More Dutch than immigrants | 2 | 5 | 5 |
Equal numbers of Dutch and immigrants | 1 | 2 | 1 |
More immigrants than Dutch | 1 | 1 | 4 |
|
|||
Information brochure handed out together with other information | N/Ac | 2 | 3 |
Short explanation of eHealth program | N/A | 7 | 11 |
|
|||
Opinion on content | N/A | ||
Positive | 0 | 6 | |
Neutral | 2 | 1 | |
Negative | 4 | 0 | |
Has not seen content | 2 | 4 | |
Opinion on quiz format | N/A | ||
Positive | 1 | 6 | |
Neutral | 4 | 5 | |
Negative | 3 | 0 | |
Opinion about additive value | N/A | ||
Valuable | 1 | 9 | |
Not valuable | 7 | 1 | |
Don’t know | 0 | 1 | |
Perception of effectiveness on knowledge about a healthy pregnancy | N/A | ||
Effective | 2 | 9 | |
Not effective | 3 | 0 | |
Don’t know | 3 | 2 | |
Opinion about national implementation | |||
Positive | 1 | 2 | 11 |
Neutral | 2 | 2 | 0 |
Negative | 2 | 4 | 0 |
aOne practice was not available for interview, but the assistant indicated that the practice continued implementation after the pilot period.
bData for one midwifery practice that did not implement the program are missing.
cN/A = not available or not applicable.
During the study period, 488 pregnant women registered themselves on the program website, and 16% (80/488) of them enrolled but did not access any information available within the program; therefore, they are excluded from all further analysis. The baseline characteristics of participants are summarized in
Baseline characteristics of participants (n = 343)a
% | |
|
|
Age (years), mean ± SD | 30 ± 5 |
High level of education | 66 |
Full-time job (≥ 32 hours per week) | 56 |
Non-Dutch ethnicity | 26 |
First-time pregnancy | 65 |
|
|
Overweight before pregnancy (BMI ≥ 25)b | 19 |
Smoking during pregnancy | |
Yes | 3 |
Until pregnancy was known | 14 |
No | 84 |
Alcohol use during pregnancy | 17 |
No use of folic acid supplements | 10 |
|
|
Internet at home | 96 |
Used internet > 1 hour per week | 86 |
Used internet to find pregnancy-related information | 87 |
aThe total number of participants was 488; response rate for self-completed questionnaire at baseline was 70%.
bBMI (body mass index) = weight/height2(kg/m2)
Program users (n = 408) received a mean of 5.4 (SD 1.6) quiz emails and accessed an average of 3.2 (SD 1.8) of those quizzes during their pregnancy (
Health-related information received and accessed by program users (n = 408)
n | % | |
|
||
1-2 | 29 | 7 |
3-5 | 142 | 35 |
6-7 | 237 | 58 |
Mean (SD), median | 5.4 (1.6), 6 | |
|
||
1-2 | 172 | 42 |
3-5 | 186 | 46 |
6-7 | 50 | 12 |
Mean (SD), median | 3.2 (1.8), 3 | |
|
||
1-10 | 166 | 41 |
11-20 | 110 | 27 |
21-30 | 88 | 22 |
31-40 | 44 | 11 |
Mean (SD), median | 15.5 (10.7), 13 | |
|
||
None | 65 | 16 |
1-8 | 242 | 59 |
9-16 | 75 | 18 |
17-33a | 26 | 6 |
Mean (SD), median | 5.8 (6.2), 4 | |
|
||
None | 266 | 65 |
1-6 | 112 | 27 |
7-14 | 21 | 5 |
15-28 | 9 | 2 |
29-40 | 0 | 0 |
Mean (SD), median | 2.3 (4.1), 0 | |
|
||
None | 266 | 65 |
1-2 | 13 | 3 |
3-4 | 32 | 8 |
5-6 | 97 | 24 |
Mean (SD), median | 1.7 (2.5), 0 |
aThe maximum number of available tips was 33 because not all questions included practical information.
Not all participants received three quiz emails and therefore did not receive an invitation to complete the feedback questionnaire (n = 32). Taking this into account, the response rate for the feedback questionnaire was 43% (163/376). Most women evaluated the program as easy to understand (96%; 157/163) and reliable (81%; 130/161), although only 39% (61/161) agreed with the statement that the information was available at the right time (
User perceptions of the eHealth program (n = 163)
The participants who completed the feedback questionnaire were divided into three satisfaction groups based on their responses to the statements about the program (
Participation measures and program rating by user satisfaction
Low Satisfaction |
Intermediate Satisfaction |
High Satisfaction |
Linear Trend, |
|
Quiz emails received | 5.9 ± 1.2 | 5.7 ± 1.3 | 5.9 ± 1.0 | .81 |
Quiz emails opened | 3.9 ± 1.5 | 4.2 ± 1.7 | 4.5 ± 1.4 | .04 |
Quiz questions opened | 19.3 ± 10.0 | 21.7 ± 9.9 | 23.5 ± 9.0 | .02 |
Practical tips opened | 6.5 ± 5.9 | 8.2 ± 7.0 | 9.7 ± 7.4 | .02 |
Clicks to related websites | 2.7 ± 4.4 | 2.9 ± 4.0 | 3.5 ± 4.9 | .32 |
Overal program rating | 5.2 ± 1.4 | 6.5 ± 1.0 | 7.6± 0.8 | <.001 |
We used regression analyses to investigate the association between individual perception components and program use and found that higher perceived interest and usefulness were associated with accessing more questions (β = −2.4,
A unique feature of this study is that the feasibility of an eHealth program was studied among both health professionals who provide the program as well as program users. This pilot study shows that it is feasible to integrate an eHealth program into standard midwifery care. About half of the midwifery practices would like to continue implementation of the program. Program users accessed a substantial proportion of available health information, although user perceptions were mixed. Therefore, this eHealth program may be a feasible communication channel to provide reliable information about a healthy lifestyle to pregnant women.
A high proportion (80%) of midwifery practices implemented the program. However, this adoption rate may not be achieved with the countrywide implementation. Some of the midwives were actively involved in the development of the program, and there were personal contacts with the publisher of the online program, which probably increased the adoption rate. However, the study implementation was done in a real-life setting and shows promise for the future countrywide implementation since half of the practices would like to integrate the program in their future standard care. Unwillingness to continue implementation of the program was not related to practical barriers, but to how midwives perceived the program. More practices may integrate the program into their future standard care if it was emphasized that the program is designed to bring existing information together and that this feature helps women to gather all relevant information easily, since several midwives had a negative perception of the program because it contained little information that was not already available from other sources.
The program users accessed, on average, 16 different health promotion messages during their pregnancy, which means that, in general, about half of the available health information was seen. Because user perceptions were mixed and several revisions were suggested, the program will be improved before the countrywide implementation, which may increase program usage.
It is thought that program satisfaction within the study is somewhat overestimated. The women who completed the feedback questionnaire accessed more topics (
Our results showed that the perceived attractiveness of design was an important factor influencing program use.
A strength of our study is that we studied feasibility of the program among both users and midwives using a combination of quantitative and qualitative data. The user perceptions are supplemented with objective usage data, which provide further insight into the program features that need improvement. Furthermore, the evaluation among midwives indicated that the implementation of the program is feasible in a real-life situation.
In most midwifery practices several midwives work together. A limitation of the study is that the interviewed midwives might have had more positive attitudes toward the program than their non-interviewed colleagues. Therefore, we asked all midwives to register all clients to whom they gave information about the program. These data were used to estimate a more objective program delivery rate. The interviews gave insight into the reasons why midwives did not advocate integration of the program into standard care.
Some midwives expressed concerns about the reach and effectiveness of the program. Indeed, previous work showed that disadvantaged women were least easily reached by the health information available in the eHealth program [
To our knowledge, only one other eHealth program that made use of emails to promote health behaviors has been evaluated previously [
To our knowledge, there are no other studies that evaluate the feasibility of integrating an Internet program with health information within standard health care. One study introduced an eHealth program to general practitioners to support their patients with lifestyle changes. However, this study focused on practitioner views toward program usability and program design [
It seems feasible to integrate an eHealth program into standard midwifery care. About half of midwifery practices would like to continue recommending an eHealth program to their pregnant clients, and hardly any practical barriers to implementation were reported. Program users accessed a substantial proportion of available health information; however, user perceptions were mixed. Therefore, this eHealth program may be a feasible communication channel to promote a healthy lifestyle to pregnant women after suggested revisions have been carried out. Further research should evaluate email intervals, user needs, and attractiveness of embedded links to optimize transfer and uptake of information.
This research was financially supported by the Dutch Ministry of Health, Welfare and Sport. The authors wish to thank the midwives for participating in this study, Jan-Arie Dekker, the publisher of the eHealth program, for approaching and informing the midwives, and Roelf Otten and Martin de Jong from OOiP for their assistance in collecting the user statistics.
None declared.