This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.
Assisted reproductive technologies (ARTs) are considered to be physically and mentally stressful. During their treatment trajectory, couples express high information and communication needs. They appreciate using the internet to obtain fertility-related information. In a previous study, we developed myFertiCare, an eHealth tool providing personalized information and interactive functionalities for infertile couples in order to improve patient-centered care. The app has already been successful in qualitative evaluations of usability.
The aim of the current study is to quantitatively evaluate the implementation of myFertiCare by using the human, organizational, and technology–fit (HOT-fit) framework and to study the effects of using myFertiCare on couples’ knowledge about infertility, their experience of the burden of infertility, and their experience of patient-centered care. With these results, implementation can be further improved, and patient-centered care can be enhanced.
A quantitative study was performed based on the HOT-fit framework using validated questionnaires focusing on the
In the human and technology domains, myFertiCare showed good system usability, high user satisfaction, and high information and interface quality. In the organizational domain, implementation was considered to be sufficient by both patients and staff. Use of the app increased knowledge about the treatment, improved coping with the treatment, and enhanced the experience of patient-centeredness. User data showed that women were the main app users and that use of the app gradually declined during the treatment trajectory.
A multi-faceted online app, myFertiCare, has been successfully evaluated quantitatively for implementation by using the HOT-fit framework. Use of the app increased knowledge about the treatment, improved coping with the treatment, and enhanced the experience of patient-centeredness. App use could be improved by creating more publicity. By providing myFertiCare, professionals in fertility care are supported in guiding patients through their treatment trajectory and in delivering patient-centered care.
Subfertility is defined as the failure to obtain a pregnancy after more than 12 months of unprotected intercourse [
Subfertile couples appreciate using the internet to obtain fertility-related information [
As described in our previous study, we were the first to design and develop an online app (myFertiCare) for infertile patients that provides personalized information and interactive functionalities regarding their fertility treatment in order to improve the patient-centeredness of care [
We used a quantitative study design to evaluate the implementation of myFertiCare according to the HOT-fit framework. This framework states that a fit between human, organization, and technology factors is needed to successfully implement an eHealth intervention [
Ethical approval was proposed but was not required according to the local research ethics committee (Commissie Mensgebonden Onderzoek Arnhem Nijmegen).
The study was established at a Dutch university medical center specializing in ART and surgical sperm retrieval for men with azoospermia. ICSI with surgical sperm retrieval is a multidisciplinary treatment involving a urologist, who is responsible for the andrological evaluation and surgical sperm retrieval; a gynecologist, who is responsible for the subsequent ICSI procedure; a psychologist, who is present for easily accessible mental support; and, if necessary, a clinical geneticist. In January 2016, the online app myFertiCare was launched and made available via the clinic’s website, the Apple App Store, and the Google Play Store.
All couples visiting the outpatient clinic for possible ICSI with surgical sperm retrieval between January 1, 2016, and July 1, 2017, were invited to use myFertiCare and to participate in the questionnaire study. Men undergoing surgical sperm retrieval for fertility preservation purposes and couples in which neither partner understood the Dutch language were excluded. All participating couples signed an informed consent form.
The study comprised 2 separate questionnaires: one was targeted at users of myFertiCare and one was targeted at nonusers. The questionnaires were available both on paper and digitally (using Castor electronic data capture). They were sent out in June 2017. Both questionnaires contained questions on the demographic and medical characteristics of the participants.
The user questionnaire was based on the principles of the HOT-fit framework [
The questionnaire for nonusers of myFertiCare consisted of 4 questions regarding familiarity with the availability of myFertiCare, motivations for not using the app, and suggestions to increase its use and the use of other online sources of information. The questions were self-developed by the research team to match the specific context of the organization.
Both questionnaires were pilot-tested with 3 couples attending the outpatient clinic. The couples considered the questions clear and understandable, so no major changes were necessary. The questionnaires were sent out via postal mail by a member of the treatment team. One questionnaire was sent per couple, addressed to the couple, because a close connection and interaction between partners during the treatment was assumed. Each couple received both the user and nonuser questionnaires and had to decide which of the two was applicable. Use was defined as the minimum of one log-in to the app; nonuse was defined as never having logged in. Patients had to write down who completed the questionnaire: the male partner, the female partner, or both. Couples could return the questionnaire via postal mail or email. Nonresponders were sent a reminder 2 weeks after the initial invitation. Questionnaires were collected from July 1, 2017, until August 18, 2017, and assigned a code that was only available to the researchers. All data were analyzed confidentially.
User data were automatically transferred to an anonymized Microsoft Excel file. In this way, all app visits and page views could be logged by date and time and analyzed later. All patients were included who used myFertiCare between January 1, 2016, and July 1, 2017, whether they participated in the questionnaire study or not.
Statistical analysis was performed using Microsoft Excel and SPSS (version 22). Baseline characteristics of the study population, user data, and the results of the questionnaires were analyzed using descriptive statistics: median (range), mean (SD), or frequency. We also focused on finding possible differences in user data between men and women.
In total, 314 ICSI couples were invited to participate (
Overview of the participating couples.
Demographic and clinical characteristics of men and women in the study, both users and nonusers.
Characteristics | Users (n=35 couples) | Nonusers (n=107 couples) | |||||||
|
|||||||||
|
Age (years), mean (SD) | 37 (7.7) | 37 (7.6) | .80a | |||||
|
Dutch background, n (%) | 35 (100) | 103 (96) | .57b | |||||
|
|
.44d | |||||||
|
|
Low | 6 (18) | 10 (10) |
|
||||
|
|
Medium | 12 (36) | 40 (40) |
|
||||
|
|
High | 15 (46) | 51 (50) |
|
||||
|
|
.47e | |||||||
|
|
Children | 12 (34) | 44 (41) |
|
||||
|
|
No children | 23 (66) | 63 (59) |
|
||||
|
|||||||||
|
Age (years), mean (SD) | 32 (4.3) | 32 (4.0) | .51a | |||||
|
Dutch background, n (%) | 32 (91) | 103 (96) | .36b | |||||
|
|
.60d | |||||||
|
|
Low | 2 (6) | 5 (5) |
|
||||
|
|
Medium | 13 (37) | 48 (47) |
|
||||
|
|
High | 20 (57) | 50 (48) |
|
||||
|
|
.86e | |||||||
|
|
Children | 12 (34) | 35 (33) |
|
||||
|
|
No children | 23 (66) | 72 (67) |
|
||||
|
Currently pregnant | 7 (20) | 31 (29) | .48e |
aObtained with an independent-sample
bObtained with the Fisher exact test.
cEducational status: Low educational status included no education and lower general secondary education; medium educational status included higher general secondary education and intermediate vocational education; high educational status included higher vocational education and a university degree.
dObtained with the Fisher-Freeman-Halton test.
eObtained with the chi-square test.
Demographic and clinical characteristics of couples, both users and nonusers.
Characteristics | Users (n=35 couples) | Nonusers (n=107 couples) | |||
|
.24b | ||||
|
Low | 11 (31) | 19 (18) |
|
|
|
Medium | 21 (60) | 78 (73) |
|
|
|
High | 3 (9) | 10 (9) |
|
|
Duration of infertility (months), median (range) | 24 (8-120) | 30 (2-120) | .05c | ||
|
N/Ad | ||||
|
Out of treatmente | 17 (49) | 80 (75) |
|
|
|
Before surgical sperm retrievalf | 1 (3) | 3 (3) |
|
|
|
After surgical sperm retrieval, before ICSIg,h | 2 (6) | 3 (3) |
|
|
|
During first ICSI cyclei | 6 (17) | 7 (7) |
|
|
|
During a following ICSI cycle or cryo cycle | 9 (26) | 14 (13) |
|
aClassified according to the Dutch Social and Cultural Planning Office definitions: low socioeconomic status was a status score of ≤–1; medium socioeconomic status was a status score between –1 and 1; high socioeconomic status was a status score of >1.
bObtained with the chi-square test.
cObtained with the Mann-Whitney test.
dN/A: not applicable; not calculated because of small subgroups.
eThese couples were either pregnant or had exhausted all treatment options.
fThese couples were undergoing the diagnostic process before surgical sperm retrieval.
gICSI: intracytoplasmic sperm injection.
hThe results of surgical sperm retrieval were being evaluated in preparation for ICSI.
iThese couples were starting the first ICSI cycle; no pregnancy test had yet been adminstered.
We analyzed questionnaires from 35 couples who were myFertiCare users according to the HOT-fit framework.
In 42% (13/31) of the couples, the female partner was the only user of myFertiCare. In 19% (6/31), only the male partner used the app. In 39% (12/31) of the couples, both partners used the app, all of whom had at some point used the app together; in 92% (11/12) of these couples, the female partner was the main user.
The results of the 3 validated questionnaires for the
Users stated that they would recommend myFertiCare to a friend (25/26, 96%) because they considered the app to be informational and easy to handle. They stated they would use similar apps if they were available when visiting other medical departments (25/26, 96%). The most appreciated functionality was the visualized treatment trajectory, which showed the couple’s scheduled and as-yet unscheduled appointments on a visual timeline. Some couples gave suggestions for future app functionalities, such as a medication schedule and a mood tracker. The couples were confident that the app safeguarded their personal information (25/26, 96%). Suggestions to achieve more frequent app use included increasing publicity for the app, increasing activity on the forum, and using the app during outpatient appointments.
Scores for the System Usability Scale, End-User Computing Satisfaction, and Computer System Usability Questionnaire scales of users of myFertiCare.
Questionnaire | Response ratea, n | Median score (range) | Possible range of scores | ||||
System Usability Scale (higher scores are better) | 25 | 73 (43-93) | 0-100 | ||||
|
|||||||
|
Content | 24 | 15 (4-19) | 4-20 | |||
|
Accuracy | 23 | 8 (2-10) | 2-10 | |||
|
Format | 23 | 8 (2-10) | 2-10 | |||
|
Ease of use | 24 | 8 (2-10) | 2-10 | |||
|
Timelines | 22 | 8 (2-10) | 2-10 | |||
|
Total | 21 | 47 (12-59) | 12-60 | |||
|
|||||||
|
Information quality | 22 | 18 (6-36) | 7-49 | |||
|
Interface quality | 22 | 9 (3-20) | 3-21 |
aThe response rate was defined because not every couple (n=35) answered every question.
The couples were asked how they found out about the availability of myFertiCare. Most of them remembered being informed verbally or in writing by a member of the treatment team (25/32, 78%). A minority did not remember being informed, but said they found the app on the hospital website (3/32, 9%). Information about the app was considered complete and well timed. Couples who could not remember being informed about the availability of myFertiCare stated they would have appreciated this. The majority of couples felt that myFertiCare was well used in fertility care (23/28, 82%) and that the treatment team was sufficiently familiar with the app (19/25, 76%). One third of couples were not aware of whom to ask questions to about the app (10/28, 36%). All couples that did ask a question about the app to a member of the treatment team stated they received a satisfying answer.
The
To supplement the HOT-fit evaluation, we studied the effect of using myFertiCare on (1) knowledge about infertility and treatment, (2) the burden of infertility and treatment, and (3) the patient-centeredness of care.
Half the couples reported that using myFertiCare did not increase at all or only slightly increased knowledge about the causes of infertility. However, 79% did feel that myFertiCare increased knowledge about fertility treatment (
Effect of myFertiCare on knowledge.
Respondents, n | myFertiCare has increased my knowledge about the causes of infertility | myFertiCare has increased my knowledge about the treatment of reduced fertility |
Not at all | 11 | 5 |
Slightly | 3 | 1 |
Moderately | 8 | 5 |
Strongly | 6 | 11 |
Very strongly | 2 | 7 |
Half the couples indicated that myFertiCare contributed positively to coping with treatment (
Effect of myFertiCare on the burden of infertility and treatment. The respondents were asked to complete the following sentence: “myFertiCare positively contributed to my...”.
The couples were positive about the effect of myFertiCare on their experience of patient-centered fertility care. All 8 surveyed items (ie, accessibility, information, communication, involvement, attention to wishes and needs, agreement and collaboration, professionality, and organization of health care) were improved by the use of myFertiCare (
Effect of myFertiCare on the experience of patient-centeredness of care. The respondents were asked to complete the following sentence: “myFertiCare increased...”.
Questionnaires from 107 couples who were nonusers of myFertiCare were analyzed. Although providing information about myFertiCare was part of routine care, only 25% (27/107) of nonusers remembered being informed about the availability of myFertiCare. The main reason for nonuse was a lack of need or interest. Of the couples that did not remember being informed about myFertiCare, half (54/107) said that they would have liked to use the app. The other half felt that they did not need the app because they obtained sufficient information via other sources. The majority of nonusers of myFertiCare used other sources of online information about infertility or treatment (71/107, 66%).
We analyzed user data as a supplement to the questionnaire study. In total, 163 individual patients used myFertiCare, who were included in 139 couples. In 21% (29/139) of couples, only the male partner used the app, while in 47% (66/139) of couples, only the female partner used the app, and in 32% (44/139) of couples, both partners used the app. When both partners were users, the female partners were the more frequent users, with a median of 9 visits, compared to a median of 3 visits for the male partners. All user data are shown in
User data for myFertiCare from December 8, 2015, until August 4, 2017.
User data | Total (n=163) | Men (n=61) | Women (n=102) | |
Visits, n | 951 | 192 | 759 | N/Aa |
Page views, n | 17,097 | 3734 | 13,363 | N/A |
One-visit-users, n (%) | 51 (31) | 25 (41) | 26 (26) | .04b |
Visits per user, median (range) | 3 (1-85) | 2 (1-23) | 3 (1-85) | .001c |
Frequency of visits (interval between visits in days), mean | 5.4 | 12 | 3.6 | .006c |
Duration per visitd (minutes), median (range) | 1.03 (0-107) | 1.55 (0-107) | 0.93 (0-51) | .12c |
Time between first and last visit (days), median (range) | 30 (1-499) | 63 (1-499) | 20 (1-347) | .01c |
Total duration of use (minutes), median (range) | 9.6 (0-268) | 5.9 (0-119) | 13 (0-268) | .01c |
Page views per usere, median (range) | 59 (0-1254) | 47 (0-375) | 76 (1-1254) | .002c |
Page views per visite, median (range) | 11 (0-139) | 10 (0-88) | 11 (0-139) | 0.96c |
aN/A: not applicable.
bObtained with chi-square test.
cObtained with Mann-Whitney test.
dUsers were logged out automatically after 20 minutes of inactivity.
eExcluding view of navigation pages (ie, log-in or log-out pages).
Use of myFertiCare per treatment phase. The phases included (1) before surgical sperm retrieval; (2) after surgical sperm retrieval, during preparation for ICSIa; (3) during ICSI treatment, but before visits to the outpatient clinic; (4) during ICSI treatment, between the first and last visits to the outpatient clinic; (5) during ICSI treatment, between the embryo transfer and the pregnancy test; and (6) after ICSI. Phase 6 included (6A) the period between a negative pregnancy test or cancellation of treatment until the start of a new treatment; (6B) the period between a positive pregnancy test and the first ultrasound; (6C) pregnancy, during the period after the first ultrasound; and (6D) exhaustion of all treatment options (without pregnancy).
|
Treatment phase | ||||||||
|
1 | 2 | 3 | 4 | 5 | 6A | 6B | 6C | 6D |
App users, n | 90 | 70 | 47 | 39 | 26 | 26 | 14 | 11 | 12 |
Page views, n | 7260 | 3345 | 1791 | 1856 | 789 | 828 | 459 | 237 | 253 |
aICSI: intracytoplasmic sperm injection.
In this quantitative study, the implementation of myFertiCare was evaluated with the HOT-fit framework. In the
The current study shows that women were the main app users. Either the female partner was the only user or was the main user when both partners used the app. This observation is in agreement with previous research [
User data showed that use of the app was highest before surgical sperm retrieval and gradually declined thereafter. The observed gradual decline in app use contradicts previous research, which found that the highest user activity occurred in later treatment phases, namely between oocyte retrieval, embryo transfer, and the pregnancy test [
It is remarkable that the majority of nonusers of the app did not remember being informed about the availability of myFertiCare. All couples received this information as part of standard care during an informative group consultation conducted by a specialized nurse at the beginning of the treatment trajectory. It is known that patients’ memory for medical information is often poor and inaccurate, especially when the patient is anxious [
By studying the effects of app use, we found that users considered myFertiCare to be mainly a source of information, rather than a tool able to significantly decrease the burdens of infertility and treatment. However, the couples were outspoken that myFertiCare improved their experience of patient-centeredness of care, meeting the goal we set at the start of the study. Therefore, myFertiCare constitutes an innovative tool to help professionals provide patient-centered care. We hypothesize that myFertiCare could score higher for influence on the burdens of infertility and treatment if it were supplemented with functionalities targeting this effect. In the current study, we developed an online app for a pilot population and were not equipped with resources to add extra functionalities. Another possible way of improving app use would be to develop an app that also provides a benefit for the treatment team, such as by making it easier or more efficient to provide care to patients or by making that care better. Therefore, we call for other medical professionals to continue developing online interventions in collaboration with their patients and staff, so that patient-centered care can be improved from the perspective of the patient and the professional.
Numerous eHealth interventions with different functionalities targeted at a variety of patient categories have been reported. It is remarkable that most evaluations of these interventions are only performed at the end of the intervention, although the importance of conducting evaluations throughout an intervention is regularly discussed [
Our study also has limitations. There was a relatively low response rate of 25% (35 of 139 couples) to the user questionnaires. For the nonuser questionnaires, the response rate was 61% (107 of 175 couples). A possible explanation could be that we sent the questionnaires in June 2017 to all couples who were treated between January 2016 and July 2017. It could be that a significant proportion of these couples had already dropped out or finished treatment. Another hypothesis is that the response rate could have been affected by the length of the questionnaires. Because we used multiple validated questionnaires on different domains, the user questionnaire was quite extensive, whereas the nonuser questionnaire contained only 4 questions. However, there was good consistency in the data from the user questionnaires, which supports the reliability of the study data. Another limitation is that we studied the effects of app use (ie, knowledge about infertility and treatment, the burdens of infertility and treatment, and patient-centeredness of care) based on self-reported differences before and after app use, rather than measurements before and after app use. We chose this approach because we did not want patients to feel that they had to perform an exam instead of a questionnaire, and we did not want to make the questionnaire more extensive to study patients’ knowledge.
A multi-faceted online app, myFertiCare, has been successfully evaluated quantitatively for implementation with the HOT-fit framework. Use of the app increased knowledge about the treatment, improved coping with the treatment, and enhanced the experience of patient-centeredness of care.
In our previous study, we successfully designed, developed, and qualitatively evaluated myFertiCare for usability. In the current study, implementation of the app was positively and quantitatively evaluated based on the HOT-fit framework, and the effects of app use were studied. Through these consecutive studies, a framework has become available that can be used throughout the complete trajectory of app development, implementation, evaluation, and improvement, and which involves both patients and professionals in every study phase.
Providing myFertiCare encourages professionals in fertility care to guide patients through their treatment trajectory and to deliver patient-centered care. Furthermore, myFertiCare offers an opportunity to empower patients and help them manage their own treatment trajectories. It would be valuable for future research to improve the app based on the couples’ and professionals’ suggestions, so that more support is perceived and app use can be expanded to other patient categories and medical departments. We appeal to professionals in both fertility care and other medical departments to provide eHealth initiatives to their patients in which both patients and professionals are involved in every phase of design, development, implementation, and evaluation.
assisted reproductive technology
Computer System Usability Questionnaire
End-User Computing Satisfaction
human, organization, and technology–fit
intracytoplasmic sperm injection
in vitro fertilization
System Usability Scale
The authors wish to thank all study participants and the medical and technical experts who participated in this study. This study did not receive any specific grants from funding agencies in the public, commercial, or not-for-profit sectors.
All authors contributed to the design of the study. EMS and JGML performed data collection and analysis, which was supervised by the other authors. EMS drafted the manuscript. All authors critically revised the manuscript and approved the final version for submission for publication.
None declared.