Published on in Vol 23, No 8 (2021): August

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/30861, first published .
Comparison of the Differences Between Web-Based and Traditional Questionnaire Surveys in Pediatrics: Comparative Survey Study

Comparison of the Differences Between Web-Based and Traditional Questionnaire Surveys in Pediatrics: Comparative Survey Study

Comparison of the Differences Between Web-Based and Traditional Questionnaire Surveys in Pediatrics: Comparative Survey Study

Original Paper

Department of Child Health Care, Children’s Hospital of Chongqing Medical University, Chongqing, China

Corresponding Author:

Yan Hu, PhD

Department of Child Health Care

Children’s Hospital of Chongqing Medical University

No.136, 2nd Zhongshan Road

Yuzhong District

Chongqing,

China

Phone: 86 023 63622764

Email: hy420@126.com


Background: A web-based survey is a novel method for data capture. Some studies have applied web-based surveys in pediatrics, but few of them have reported data on the differences between web-based and traditional questionnaire surveys.

Objective: The objective of our study was to evaluate the internal consistency of a web-based survey and compare it with a traditional questionnaire survey in pediatrics.

Methods: A convenience sample of caregivers was invited to participate in the survey on feeding patterns and their children’s eating behaviors if their children were aged 2 to 7 years. A web-based survey and a traditional questionnaire survey were carried out between October 2018 and July 2019. A total of 1085 caregivers were involved in this study, and they were divided into the following three groups based on methods and sources: (1) web-based survey from a web source, (2) web-based survey from a hospital source, and (3) traditional questionnaire survey from a hospital source. The data were then compared and analyzed.

Results: A total of 735 caregivers participated in the web-based survey and 350 caregivers participated in the traditional questionnaire survey, and 816 cases were then included in the analyses after data processing. The effective rate of the web-based survey was 70.1% (515/735), and the completeness rate of the traditional questionnaire survey was 86.0% (301/350). There were no significant differences between web-based surveys from different sources. However, demographic characteristics were significantly different between the web-based and traditional questionnaire surveys, mainly in terms of age and caregivers (χ²4=16.509, P=.002 and χ²4=111.464, P<.001, respectively). Caregivers of children aged 2 to 3 years and grandparents were more likely to respond to the web-based survey. Age-specific stratified analysis showed that the score of “monitoring” and the reporting rate of “poor appetite” in children aged 2 to 3 years were significantly higher in the web-based survey compared to the traditional questionnaire survey after adjusting for demographic characteristics.

Conclusions: A web-based survey could be a feasible tool in pediatric studies. However, differences in demographic characteristics and their possible impacts on the results should be considered in the analyses.

J Med Internet Res 2021;23(8):e30861

doi:10.2196/30861

Keywords



A questionnaire survey is an important method in social science studies. The method was invented in the 1930s and has been widely adopted in psychological studies. With the advent of technology, a web-based survey has been appreciated as a novel method and has been applied in clinical studies [1]. To date, the number of “netizens” (citizens of the internet) in China has reached 989 million, with a 70.4% rate of internet penetration [2], which has provided an effective platform for web-based surveys. In addition, the development of mobile internet technologies has resulted in flexible web-based surveys due to mobile phone apps. The technology has a simple design and production, and has a growing frequency of usage.

Due to convenience, there has been more adoption of web-based surveys in large-scale studies. Although a larger sample size is associated with more statistical confidence of the results, this depends on the requirements and specifications of the survey design. Thus, robust study designs determine the web-based survey results, especially in the medical and health fields [3]. Although previous studies have shown that the accuracy of a web-based survey may be higher, it is primarily used in basic information collection, satisfaction evaluation, and marketing [4]. In addition, findings from fields related to subjective cognition have been shown to be highly vulnerable to survey methods [5]. In pediatrics, the participants of questionnaire surveys are mainly caregivers, who are often subjective; thus, the adopted survey methods might yield varying data. While some studies have applied web-based surveys in pediatrics, data on the differences between web-based and traditional questionnaire surveys remain scant. Here, we evaluated the internal consistency of a web-based survey and then compared it with a traditional questionnaire survey in pediatrics.


Study Design

This study was conducted between October 2018 and July 2019. We used two survey methods to investigate eating behaviors in children aged 2 to 7 years, as well as the feeding patterns of their caregivers. We conducted a web-based survey in October 2018, while a traditional questionnaire survey was conducted in July 2019. Based on the methods and sources, the participants were divided into the following three groups: (1) web-based survey from a web source (group A), (2) web-based survey from a hospital source (group B), and (3) traditional questionnaire survey from a hospital source (group C).

The study was reviewed and approved by the Ethics Committee of the Children’s Hospital of Chongqing Medical University (2019-202).

Traditional Questionnaire Survey

The traditional questionnaire survey was mainly conducted by the investigators through face-to-face questioning in the child health care clinic, and informed consent was obtained orally before the investigation. The completed questionnaires were then collected and kept by the investigators. The questionnaire (43 items) included basic information about the children and the caregivers, and required the caregivers to evaluate the children’s performance in the previous month using the Likert scoring method [6].

The simplified Child Feeding Questionnaire (CFQ) revised by Lixia et al [7] was used to assess the feeding patterns of the caregivers. The content included “monitoring,” “pressure,” and “restriction,” each of which involved four items, making a total of 12 items. The score for each dimension was expressed as an average score for the four items in that dimension.

On the other hand, the simplified Chinese version of Identification and Management of Feeding Difficulties (IMFeD) [8] was used to investigate the children’s eating behaviors. The content included five aspects of eating behavior problems, including “poor appetite” (four items), “food preference” (three items), “poor eating habit” (four items), “parental misperception” (two items), and “fear of feeding” (two items). An eating behavior problem existed when the score was lower than 40% of the total in each aspect.

Web-Based Survey

The web-based survey was set as an open survey. It was built by “wjx” (an online questionnaire platform in mainland China) and was promoted remotely through WeChat on the internet (web source), as well as a paper poster (Multimedia Appendix 1) with a QR code in the child health care clinic (hospital source). Initial contact with potential participants was made on the internet, and the questionnaire could be accessed directly through WeChat. The questionnaire contained five pages, and a brief introduction of the questionnaire was shown on the first page. The main content and scoring methods for the web-based survey were similar to the traditional survey, and basic information was collected by adaptive questioning. There were, however, three additional items in the web-based survey that were used to distinguish between (1) the web and hospital source, (2) healthy and sick children, and (3) high and poor quality of the response (self-assessment), resulting in a total of 46 items. The survey time was set at about 10 minutes. To ensure the honesty of the participants, survey participation was anonymous and voluntary. To ensure participation, feedback and suggestions would be shared at the end of the survey. To reduce repeated responses, each WeChat account of the survey could only send a single response. In addition, to reduce the possibility of misunderstanding, 10 caregivers were invited to adjust the expressions through pretests prior to the survey. Finally, the participants were required to complete all items before submitting the survey, and the answers could not be reviewed or changed.

Data Processing

Only investigators had access to the data. Data, such as IP address, phone number, and children’s and caregivers’ basic information, were recorded in Excel 2016 (Microsoft Corp). Then, the data were checked for duplication, and cases with a high degree of overlap (if any of the three aspects mentioned above were the same) were considered as repeated responses. Because psychological theory appreciates the first impression as more accurate [9], late responses, which were repetitive, were excluded. Finally, cases of sick children (eg, chronic liver or kidney diseases, developmental diseases, and food allergy) and poor quality of responses (ie, the participants themselves thought the answers were not rigorous) in the web-based survey were also excluded from the analyses.

Statistical Analyses

The sample in this study was a convenience sample, and the sample size was calculated using a single proportion sample size estimating algorithm [10]. An 82.8% rate of incidence of eating behavior problems was taken from our previous study [11], with a margin of error of 5%, confidence level of 95%, and nonresponse rate of 10%. Calculation provided 217 as the minimum sample size for each group.

Only completed questionnaires were analyzed. All statistical analyses were performed using SPSS 26.0 (IBM Corp). Descriptive statistical tools were used to analyze the demographic characteristics of the participants. To test whether the children from the hospital sources (groups B and C) had nutrition-related problems, comparison between BMI-for-age Z scores and the standard mean was carried out using a one-sample t test. We used analysis of variance (ANOVA) and the least significant difference (LSD) test to compare the feeding patterns. On the other hand, comparison of eating behavior problems was carried out using the chi-square test. Bonferroni correction was utilized for multiple comparisons. A P value <.05 was considered statistically significant.


Data Processing and Response Rates

Data processing and response rates are shown in Figure 1. A total of 1085 caregivers (735 were in the web-based survey and 350 were in the traditional questionnaire survey) were involved in this study. It took 9.9 (IQR 4.9) minutes for the web-based survey and about 10 minutes for the traditional questionnaire survey. There were 232 cases in group A, 283 in group B, and 301 in group C after data processing. The effective rate of the web-based survey was 70.1% (515/735), and the completeness rate of the traditional questionnaire survey was 86.0% (301/350).

Figure 1. Data processing and the response rates of the surveys.
View this figure

Characteristics of the Participants

Our data showed that the mean BMI-for-age Z scores in groups B and C were −0.06 (SD 1.08) and −0.09 (SD 1.31), respectively, which had no significant differences with the standard mean (t237=−0.890, P=.37 and t271=−1.133, P=.26, respectively). Thus, the children from hospital sources had no nutrition-related problems.

The chi-square test showed significant differences in age and caregivers among the three groups (χ²4=16.509, P=.002 and χ²4=111.464, P<.001, respectively). In addition, multiple comparisons showed that there were no significant differences between groups A and B. On the contrary, the results showed that there were significant differences between groups A and C in age and caregivers (χ²2=14.218, P=.001 and χ²2=103.387, P<.001, respectively), as well as between groups B and C in age and caregivers (χ²2=9.359, P=.009 and χ²2=96.940, P<.001, respectively). The findings showed that caregivers of children aged 2 to 3 years and grandparents were more likely to respond to the web-based survey (Table 1).

Table 1. Comparison of demographic characteristics among the three groups.
CharacteristicGroup Aa (N=232), n (%)Group Bb (N=283), n (%)Group Cc (N=301), n (%)χ2 (df)dP value
Age


16.509 (4).002

2-3 years106 (45.7%)117 (41.4%)90 (29.9%)e,f


3-5 years87 (37.5%)119 (42.0%)141 (46.8%)


5-7 years39 (16.8%)47 (16.6%)70 (23.3%)

Gender


1.297 (2).52

Male122 (52.6%)163 (57.6%)167 (55.5%)


Female110 (47.4%)120 (42.4%)134 (44.5%)

Caregivers


111.464 (4)<.001

Parents146 (62.9%)185 (65.4%)290 (96.3%)e,f


Grandparents79 (34.1%)90 (31.8%)6 (2.0%)


Others7 (3.0%)8 (2.8%)5 (1.7%)

aGroup A: web-based survey from a web source.

bGroup B: web-based survey from a hospital source.

cGroup C: traditional questionnaire survey from a hospital source.

dChi-square analysis with Bonferroni correction.

eP<.01 vs group A.

fP<.01 vs group B.

Feeding Patterns and Eating Behavior Problems

Age-specific stratified analysis was performed following selection of children whose caregivers were parents to avoid the influence of demographic characteristics on the data. The results showed that in children aged 2 to 3 years, the scores on “monitoring” and the rates of reporting “poor appetite” were significantly different among the three groups (F2=12.549, P<.001 and χ²2=6.579, P=.04, respectively). Multiple comparisons then showed that the scores on “monitoring” in groups A and B were significantly higher compared to that in group C (P<.001 and P<.001, respectively), while the rate of reporting “poor appetite” in group B was significantly higher than that in group C (χ²1=6.138, P=.01). In children aged 3 to 5 years, the scores on “restriction” were significantly different among the three groups (F2=3.221, P=.04), but multiple comparisons showed no significant differences between the groups. In children aged 5 to 7 years, the rates of reporting “poor appetite” were significantly different among the three groups (χ²2=6.472, P=.04), but multiple comparisons showed no significant differences between the groups.


Principal Findings

Compared with a traditional questionnaire survey, a web-based survey is widely accepted due to its convenience, low cost, efficiency, wide-range coverage, and semiclosed and anonymous nature. However, it presents other shortcomings, such as repeated responses as well as information and selection bias, which might significantly affect the results of descriptive studies [12-14]. It is, therefore, necessary to conduct appropriate pretests [15] and choose reliable scientific data processing methods before executing a web-based survey. In our study, a variety of methods, such as pretests and outcome feedback, were adopted to improve the robustness of the web-based survey. Our results showed that only 2.8% (18/640) of the cases were classified as repeated responses, while 1.8% (11/622) were of poor quality. Moreover, the effective rate of the web-based survey was 70.1% (515/735), which was lower than the completeness rate of the traditional questionnaire survey (86.0%, 301/350). This is because the web-based survey was set as an open and voluntary survey, resulting in 95 participants (12.9%, 95/735) who were not study participants. These findings suggest that a web-based survey could be a feasible tool in pediatrics. Most importantly, the different data sources did not impact the output of the web-based survey, indicating that the web-based survey had good internal consistency whether through web promotion or hospital publicity. However, despite revising the questions based on the pretests, 1.0% (5/515) of the cases reported difficulty with understanding the content. Thus, lack of investigator explanation might lead to confusion of the results due to misunderstanding among participants.

We next assessed whether the different sources of data could impact the results. Benedik et al [16] showed that both web-based and traditional questionnaire surveys demonstrated no differences in nutrient intake among pregnant women. Although the study ensured the consistency of participants, highly educated participants may make the conclusion unrepresentative. In addition, since nutrient intake represents objective data, it may not be easily affected by the survey methods. On the contrary, Milton et al [17] showed that the web-based survey had a higher reporting rate on sensitive topics in young people, suggesting that the results between the two methods of surveys might differ even in cases where the participants are the same. The effects of web-based and traditional survey methods among different populations have also been evaluated. Graefe et al [18] reported different demographic characteristics between web-based and mail-based surveys, but the participants of the web-based survey were younger and had better education. In our study, caregivers of children aged 2 to 3 years and grandparents were more likely to participate in the web-based survey, which is different from the finding of other studies [18,19] reporting that older people are less likely to participate in a web-based survey. This may be explained by the fact that today’s grandparents are not exactly the same as older people, and the widespread popularity of the internet in China [2] has undoubtedly changed how people use mobile phones. More importantly, different demographic characteristics might lead to different results. For instance, Cantuaria et al [19] demonstrated that participants in a mail-based survey were more likely to report health problems. In our study, we showed that not only the eating behaviors of children but also the feeding patterns of caregivers were inconsistent, even after adjusting for demographic characteristics, a phenomenon dominant in the younger age group. We associated these findings with the fact that our survey focused on children’s behavioral development, and the answers were more subjective and more dependent on the feelings of the parents. Thus, a web-based survey might be more prone to information bias in pediatrics when it comes to subjective topics. Besides, the interpretations of the investigators might play a certain role in the traditional questionnaire survey and the psychological effects of the participants may influence them to give a “good” answer [20]. In addition, although the web-based survey was meant to be a self-evaluation model to improve participation, the potential selection effect of its voluntary nature on participants could not be completely ruled out, which might lead to an increased proportion of problematic participants. Taken together, our data suggest that demographic characteristics and results would be different among different survey methods when used in pediatrics, and topics with strong subjectivity need more comprehensive consideration before adopting the study tools.

Limitations

Our study was limited by the convenience sample, as well as the fact that the interval between the two surveys was slightly long and the participants were not the same. Although stratified analysis was conducted according to demographic characteristics, changes in the feeding and eating behaviors of the participants and their effects on the results cannot be completely ruled out.

Conclusions

Our data demonstrated that a web-based survey could be a feasible tool in pediatrics. However, differences in demographic characteristics and their possible impacts on the results should be considered when interpreting data from a web-based survey.

Acknowledgments

We thank all the participants in the study and the members of the survey team, as well as the Children’s Hospital of Chongqing Medical University. We also thank Ms Zhiyi Chen for designing the paper poster.

Conflicts of Interest

None declared.

Multimedia Appendix 1

The paper poster used in the survey.

PNG File , 4969 KB

  1. Palmblad M, Tiplady B. Electronic diaries and questionnaires: Designing user interfaces that are easy for all patients to use. Qual Life Res 2004 Sep;13(7):1199-1207. [CrossRef] [Medline]
  2. The 47th China statistical report on internet development. China INIC.   URL: http://www.cac.gov.cn/2021-02/03/c_1613923423079314.htm [accessed 2021-02-03]
  3. Knitza J, Vossen D, Geffken I, Krusche M, Meyer M, Sewerin P, Arbeitskreis Junge Rheumatologen. [Use of medical apps and online platforms among German rheumatologists : Results of the 2016 and 2018 DGRh conference surveys and research conducted by rheumadocs]. Z Rheumatol 2019 Nov;78(9):839-846. [CrossRef] [Medline]
  4. Minto C, Vriz GB, Martinato M, Gregori D. Electronic Questionnaires Design and Implementation. Open Nurs J 2017;11:157-202 [FREE Full text] [CrossRef] [Medline]
  5. Braekman E, Charafeddine R, Demarest S, Drieskens S, Berete F, Gisle L, et al. Comparing web-based versus face-to-face and paper-and-pencil questionnaire data collected through two Belgian health surveys. Int J Public Health 2020 Jan;65(1):5-16. [CrossRef] [Medline]
  6. Ventegodt S, Merrick J, Andersen NJ. Measurement of quality of life II. From the philosophy of life to science. ScientificWorldJournal 2003 Oct 13;3:962-971 [FREE Full text] [CrossRef] [Medline]
  7. Lixia Z, Daoping S, Chulin C, Fangfang L, Daqiao Z. Reliability and validity of a Chinese version of Child Feeding Questionnaire among parents of preschoolers in Chinese. Chinese Journal of Child Health Care 2016 Oct;24(10):1019-1023. [CrossRef]
  8. Jing L, Yanhong H, Ying D, Chunlan C, Lili L, Xuejiao Z, et al. Intervention effects evaluation about the eating behavior problems of children by Identification and Management of Feeding Difficulties tools, China in Chinese. Chinese Journal of Child Health Care 2014 Mar;22(03):333-336 [FREE Full text]
  9. Biesanz JC, Human LJ, Paquin A, Chan M, Parisotto KL, Sarracino J, et al. Do We Know When Our Impressions of Others Are Valid? Evidence for Realistic Accuracy Awareness in First Impressions of Personality. Social Psychological and Personality Science 2011 Jan 19;2(5):452-459. [CrossRef]
  10. Pourhoseingholi MA, Vahedi M, Rahimzadeh M. Sample size calculation in medical studies. Gastroenterol Hepatol Bed Bench 2013;6(1):14-17 [FREE Full text] [Medline]
  11. Mingyue L. The effects of feeding styles on eating behavior and BMI of children aged 2 to 7. China National Knowledge Infrastructure (CNKI).   URL: https://kns.cnki.net/KCMS/detail/detail.aspx?dbname=CMFD201901&filename=1018864932.nh [accessed 2018-05-01]
  12. Bohannon J. SCIENTIFIC INTEGRITY. Survey fraud test sparks battle. Science 2016 Mar 04;351(6277):1014. [CrossRef] [Medline]
  13. Alessi EJ, Martin JI. Conducting an Internet-based Survey: Benefits, Pitfalls, and Lessons Learned. Social Work Research 2010 Jun 01;34(2):122-128. [CrossRef]
  14. Hohwü L, Lyshol H, Gissler M, Jonsson SH, Petzold M, Obel C. Web-based versus traditional paper questionnaires: a mixed-mode survey with a Nordic perspective. J Med Internet Res 2013 Aug 26;15(8):e173 [FREE Full text] [CrossRef] [Medline]
  15. Ball HL. Conducting Online Surveys. J Hum Lact 2019 Aug;35(3):413-417. [CrossRef] [Medline]
  16. Benedik E, Koroušić Seljak B, Simčič M, Rogelj I, Bratanič B, Ding EL, et al. Comparison of paper- and web-based dietary records: a pilot study. Ann Nutr Metab 2014;64(2):156-166. [CrossRef] [Medline]
  17. Milton AC, Ellis LA, Davenport TA, Burns JM, Hickie IB. Comparison of Self-Reported Telephone Interviewing and Web-Based Survey Responses: Findings From the Second Australian Young and Well National Survey. JMIR Ment Health 2017 Sep 26;4(3):e37 [FREE Full text] [CrossRef] [Medline]
  18. Graefe A, Mowen A, Covelli E, Trauntvein N. Recreation Participation and Conservation Attitudes: Differences Between Mail and Online Respondents in a Mixed-Mode Survey. Human Dimensions of Wildlife 2011 May 31;16(3):183-199. [CrossRef]
  19. Cantuaria ML, Blanes-Vidal V. Self-reported data in environmental health studies: mail vs. web-based surveys. BMC Med Res Methodol 2019 Dec 12;19(1):238 [FREE Full text] [CrossRef] [Medline]
  20. Phillips DL, Clancy KJ. Some Effects of "Social Desirability" in Survey Studies. American Journal of Sociology 1972 Mar;77(5):921-940. [CrossRef]

Edited by R Kukafka; submitted 01.06.21; peer-reviewed by M Hawkins, R Watson; comments to author 28.06.21; revised version received 18.07.21; accepted 27.07.21; published 26.08.21

Copyright

©Heping Fang, Ruoling Xian, Zhuoying Ma, Mingyue Lu, Yan Hu. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 26.08.2021.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.