The SENTIPAT multicenter (five adult acute care units in a Parisian teaching hospital participated voluntarily: departments of digestive and general surgery; gastroenterology; hepatology; infectious diseases and tropical medicine; and internal medicine) randomized trial focused on the evolution of patients’ health on returning home post hospitalization (follow-up duration of 6 weeks). The general objective was to determine whether the information on patient’s health evolution shared by volunteer patients after returning home directly via a dedicated website was comparable with that obtained via telephone interviews. The randomization of 2050 patients (410 from each unit, 205 randomized in the Internet group and 205 randomized in the telephone group) was initially planned. The study was conducted in accordance with French regulation on ethics requirements in biomedical research.

Consecutive inpatients with Internet access at home were eligible for inclusion. Inclusion criteria also required inpatients who were not cognitively impaired and did not have a behavioral disorder, who spoke and wrote French, and were returning home after an acute care hospitalization, regardless of the type of stay—standard hospitalization (scheduled or not) on weekdays only (maximum Monday to Friday or any combination thereof) or outpatient hospitalization (1 day). Inpatients were enrolled on the day of hospital discharge by a clinical research technician of the trial. At that time, patients were informed about the study. Eligible patients not opposed to participate in the study were randomized into two parallel groups: Internet or telephone follow-up (inherently resulting in an open-label trial) at a ratio of 1:1. On the basis of a centralized randomization that allocated the eligible patient either to the Internet or to the telephone group through a website and using an underlying permutation block randomization stratified by service, the computer-generated list of permutation was established by a statistician independent from the study. At the time of patient inclusion, the technician also collected baseline variables (length of stay [LOS], sex, age, relationship status, level of education, activity, and type of insurance). Patient was then informed and discharged with documents explaining corresponding questionnaire administration.

The participation rates observed in the Internet and telephone groups were compared using the Fisher exact test, as well as the proportions of nonrelevancy answers observed in these two groups. The delays of questionnaire completion observed in the Internet and telephone groups were compared using Wilcoxon-Mann-Whitney test. Internal consistency of questionnaires was measured by calculating Cronbach alpha [31], taking into account every score that could be calculated according to the abovementioned rules. An alpha coefficient value of greater than .7 was considered as satisfactory. Dimensions’ scores were calculated for each patient as the mean of the corresponding dimensions’ items, and global score was the mean of all answered items of the questionnaire. CIs were obtained by bootstrap. Standardized Cohen d-type effect size was measured between the scores of the two groups [32].

Comparisons between the Internet and telephone groups in terms of Cronbach alpha coefficients and in terms of satisfaction scores (including dimensions’ scores) were made using a permutation test [33], with the null hypothesis distribution (distribution of the difference between the two groups under the hypothesis of no difference) generated through 1,000,000 shuffled datasets. A P value of ≤.05 defined the significance of comparisons.

Missing data were handled as follows: First, nonresponding patients were excluded from score analyses. Patients for whom less than 16 items were completed were also excluded from score analyses (ie, handled as nonresponders in the analyses). Second, the scores issued from the remaining partially completed questionnaires were calculated as mentioned above (see subsection on Score Construction).

All analyses were made with the R statistical computing freeware version 3.3.0 [34].

The SENTIPAT study was approved by the Comité de Protection des Personnes Ile de France IX (decision CPP-IDF IX 12-014, June 12, 2012); the Comité Consultatif sur le Traitement de l'Information en matière de Recherche dans le domaine de la Santé (Decision 12.365, June 20, 2012); and the Commission Nationale de l’Informatique et des Libertés (Decision DR-2012-582, December 12, 2012).

The investigation of patient satisfaction after a hospital stay resulted in several differences when comparing the two modes of questionnaire administration: self-reported Internet completion or telephone interview. The comparison between these modes of administration may be discussed according to three topics: response rate, questionnaire reliability, and satisfaction scores. The response rate observed in the group of patients randomized in the Internet group (39.3%) was much lower than that observed in the group of patients randomized in the telephone group (88.4%). Such a difference might have resulted in unbalancing the initial comparability of responders in the two groups even if Table 1 indicates that baseline variables are similar in the responders of the two groups. Unsurprisingly, the observed difference between the two groups in terms of participation rate is in accordance with the previous results issued from the same cohort focusing on patient satisfaction with regard to the hospital discharge process [26]. The difference between the participation rates observed with the two administration modes of the survey might be, at least in part, owing to the fact that it is easier to ignore an email than a phone call scheduled at a date chosen by the patient. The participation rates observed in our study are also similar to those reported by Harewood et al [20] who investigated patient satisfaction with endoscopy and observed a response rate of 34% and 78% in the Internet group and telephone group, respectively. However, comparing the response rates observed in our study with other rates previously reported is probably of limited interest as the study design widely varies from one study to another, as response rates are likely to be highly sensitive to the detailed underlying procedures for selecting participants (eg, face-to-face enrollment vs random selection in an administrative database, issues related to the initial comparability of the participants allocated in the Internet and telephone arms), reaching/soliciting responders (including reminding procedures for soliciting Web participants to complete the survey or procedure for scheduling the phone calls), and collecting answers (eg, attractiveness of the website and ease in accessing/completing the questionnaire form).

With the exception of the values for the hospital room convenience dimension, which raise concerns, the values of Cronbach alpha were satisfactory for all dimensions investigated and for the global satisfaction score, favoring the conduction of surveys with this questionnaire using either administration mode. Besides, interestingly, considering all six dimensions of the questionnaire, the values of Cronbach alpha were always higher in the Internet group than in the telephone group, with a statistical significance observed for two dimensions (global care and hospital room convenience) and for the global satisfaction score. Here, the adjunction of an interviewer in the telephone group (as compared with self-completion in the Internet group) might be considered as an undesired burden disturbing initial signal.

The observed score differences between the Internet and telephone groups (see Table 3) are contrasted, depending on the dimension investigated. On the one hand, considering hospital room convenience and hospital catering dimensions, both telephone and Internet modes of administration resulted in very similar satisfaction scores, and the difference was only slight when considering global care dimension. On the other hand, in the three dimensions related to interactions with health care providers, that is, information to patients, communication with health care providers, and behavior of health care providers, scores were significantly lower in the Internet group than in the telephone group, although it is worth mentioning that corresponding effect sizes never exceeded 0.35, a value below the medium threshold proposed by Cohen [32]. Such an observation raises a general comment on the surveys conducted in this domain. Those surveys are deployed for investigating patient satisfaction with hospital services, for bringing into light the elements which require improvements and for assessing evolution with time. Additionally, French authorities require the analysis of 120 patients per medical center each year. In such a context, our study’s finding that a mean difference of 7 points based on a sample size of 498 responders is modest in terms of effect size, suggests that potential improvements on patient satisfaction are very difficult to evidence. Dynamic trends within a given center from one year to another should be interpreted with great caution and must take into account the underlying variability of the scores, and a similar caution should be required in the interpretation of differences between centers. A potential explanation for the higher scores observed in the telephone group is that a patient might be more reluctant to provide low scores to an interviewer (moreover potentially identified as a member of the hospital staff) than when completing a strict anonymous form via the Internet. Previous studies [35,36] have also mentioned such a social desirability bias [23] as a potential explanation for the higher patient satisfaction scores issued from a questionnaire administered by a telephone interviewer as compared with a self-completed form administered by mail [35] or via the Internet [36]. In addition, the distribution of the delay between hospital discharge and questionnaire completion was more variable in the Internet group. However, as shown in the Multimedia Appendix 1, the scores of the Internet responders did not significantly vary according to the delay of questionnaire completion, indicating that the wider variability in the delay of questionnaire completion observed in the Internet group had a very limited impact (if any) on the differences of scores that were observed between the telephone and Internet modes of administration.

A strength of the study relies on the fact that it is the first randomized trial reported to date that compared inpatient satisfaction collected either via a telephone interview or via the self-completion of a similar questionnaire on a dedicated website and involved a reasonable sample of inpatients, both in terms of case-mix variability (patients originating from 5 very different hospital wards) and in terms of sample size (498 questionnaires were eventually analyzed). A home access to the Internet and a phone number were two required inclusion criteria for patient eligibility, ensuring the initial comparability of the individuals randomized in the two administration modes of the questionnaire. Moreover, to our knowledge, this study is the first to date that explores, in detail, critical issues relating to the I-Satis questionnaire, which is dedicated to be deployed in all inpatient structures in France; on the one hand, this constitutes an additional strength of the study, while on the other hand, the fact that this questionnaire is yet restricted to France constitutes a limitation of the study.

In conclusion, our study shows that the lower response rate observed with the Internet mode of administration than that observed with the telephone mode of administration must be balanced with other positive features associated with the Internet. Using the latter mode of administration has a potential lower cost than telephone [14-17], and the quality of satisfaction estimates is likely improved because the potential veil of a telephone interviewer is discarded, allowing patients to express more freely. They are more likely to rate their satisfaction about hospital stay with lower scores. This study indicates that some of these score decreases are statistically significant but the corresponding effect sizes are small, indicating that the decreases relate to moderate differences. Communication habits are evolving, and the Web form is progressively adopted as a reference mode for administrating surveys as well as a reference mode for completing questionnaires. With the exception of the higher response rate observed with telephone interview in this study, all other study results support the deployment of Web-based questionnaires for exploring inpatient satisfaction.