This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.
The implementation of an Internet option in an existing public health interview survey using a mixed-mode design is attractive because of lower costs and faster data availability. Additionally, mixed-mode surveys can increase response rates and improve sample composition. However, mixed-mode designs can increase the risk of measurement error (mode effects).
This study aimed to determine whether the prevalence rates or mean values of self- and parent-reported health indicators for children and adolescents aged 0-17 years differ between self-administered paper-based questionnaires (SAQ-paper) and self-administered Web-based questionnaires (SAQ-Web), as well as between a single-mode control group and different mixed-mode groups.
Data were collected for a methodological pilot of the third wave of the "German Health Interview and Examination Survey for Children and Adolescents". Questionnaires were completed by parents or adolescents. A population-based sample of 11,140 children and adolescents aged 0-17 years was randomly allocated to 4 survey designs—a single-mode control group with paper-and-pencil questionnaires only (n=970 parents, n=343 adolescents)—and 3 mixed-mode designs, all of which offered Web-based questionnaire options. In the concurrent mixed-mode design, both questionnaires were offered at the same time (n=946 parents, n=290 adolescents); in the sequential mixed-mode design, the SAQ-Web was sent first, followed by the paper questionnaire along with a reminder (n=854 parents, n=269 adolescents); and in the preselect mixed-mode design, both options were offered and the respondents were asked to request the desired type of questionnaire (n=698 parents, n=292 adolescents). In total, 3468 questionnaires of parents of children aged 0-17 years (SAQ-Web: n=708; SAQ-paper: n=2760) and 1194 questionnaires of adolescents aged 11-17 years (SAQ-Web: n=299; SAQ-paper: n=895) were analyzed. Sociodemographic characteristics and a broad range of health indicators for children and adolescents were compared by survey design and data collection mode by calculating predictive margins from regression models.
There were no statistically significant differences in sociodemographic characteristics or health indicators between the single-mode control group and any of the mixed-mode survey designs. Differences in sociodemographic characteristics between SAQ-Web and SAQ-paper were found. Web respondents were more likely to be male, have higher levels of education, and higher household income compared with paper respondents. After adjusting for sociodemographic characteristics, only one of the 38 analyzed health indicators showed different prevalence rates between the data collection modes, with a higher prevalence rate for lifetime alcohol consumption among the online-responding adolescents (
These results suggest that mode bias is limited in health interview surveys for children and adolescents using a mixed-mode design with Web-based and paper questionnaires.
The assessment of population health using health interview surveys is an established method in many countries and is a cornerstone of health reporting, health policies, and health sciences. However, epidemiological studies have shown decreasing response rates since the 1990s [
Mode effects are systematic distortions caused by different survey modes or interview situations [
However, researchers have discussed mode effects for sensitive topics. Web-based responses are associated with both anonymity and greater individualization. Consequently, SAQ-Web participants are not affected by social desirability; rather, they are less orientated toward social norms. Therefore, SAQ-Web mode yields the most honest reports, especially compared with interview modes [
The German Health Interview and Examination Survey for Children and Adolescents (KiGGS) is a nationally representative health interview and examination survey of children and adolescents in Germany [
When planning population-based (health) studies like KiGGS, the survey design must minimize total survey error [
Are there any differences in the prevalence rates or mean values of core public health indicators for children and adolescents aged 0-17 years between the single-mode control group using only SAQ-paper and different mixed-mode groups that combine offers of SAQ-paper and SAQ-Web?
Are there any differences in prevalence rates or mean values of these indicators between the 2 data collection modes (SAQ-paper and SAQ-Web) if all online respondents are pooled and all paper-and-pencil respondents are pooled across all survey designs?
The methodological pilot study used a sample of children and adolescents registered in the local resident registries of 20 municipalities in 5 federal states of Germany, covering urban and rural areas as well as the eastern and western regions of the country.
Data were collected using SAQ-Web or SAQ-paper methods. All selected individuals were invited by mail to participate in the study. They were sent a cover letter with the invitation to participate, information about the study and data privacy, and an informed consent form. Depending on the allocated mode, the invitation comprised a username and password for participation through the Web option along with a paper questionnaire for those allocated to the concurrent mixed-mode design, only a paper questionnaire in the single-mode design, or only the access data for the online questionnaire in the sequential mixed-mode design. The SAQ-Web questionnaire was only optimized for desktop computers. A reminder was sent by mail to respondents who had not replied within 3 weeks of the initial invitation. Participants who did not respond to the reminder were telephoned up to 5 times 4 weeks after the initial invitation. As an additional motivation for prospective participants, each parent and adolescent who had completed a questionnaire received a shopping voucher to the value of €10. The methodological pilot study strictly adhered to the data protection regulations set out in the German Federal Data Protection Act. Participation in the study was voluntary. All parents and participating adolescents were informed about the study’s aims and content, as well as data protection, and they provided informed consent. Following the strict data privacy protocol, prospective participants between the ages of 11 and 17 years received their questionnaires only after their parents provided consent.
Different questionnaires were used for different age groups. Main health indicators were included on the health questionnaires for parents of all age groups (0-17 years), and self-report data for main health indicators were obtained from adolescents aged 11-17 years. To reduce the risk of mode effects, the 2 questionnaires were designed to be as similar as possible and contained the same wording for the questions and response categories. On the basis of the unified-mode design [
As shown in
A single-mode survey design as a control group—respondents were sent an invitation letter and paper-and-pencil questionnaires, followed by a reminder after 3 weeks
A sequential mixed-mode survey design—respondents were sent an invitation letter and an online access code, followed 3 weeks later with a reminder letter and a paper-based questionnaire
A concurrent mixed-mode survey design—respondents were sent an invitation letter, a paper-based questionnaire, and an online access code (a longer version of the questionnaire was tested with a subgroup of the concurrent mixed-mode design, but this subgroup was excluded from this study) and
A preselect mixed-mode design—respondents were sent the invitation along with a postcard asking participants to choose one of the 2 options (SAQ-Web or SAQ-paper), followed by a reminder with the same offer
There were no statically significant differences in the (gross) sample composition across the 4 design groups in terms of known sample characteristics, such as age, sex, municipality size, region, or respondent citizenship, which were obtained from local registries.
The combined response rate for all survey designs was 38.43% (n=4032), following the internationally used Standard Definitions of Outcome Rates for Surveys of the American Association for Public Opinion Research (AAPOR Response Rate 2) [
For this study, only survey design groups using the same version of the questionnaire were included, with 3468 completed parent-reported health questionnaires for children and adolescents aged 0-17 years and 1194 questionnaires completed by adolescents aged 11-17 years. A response was defined as one completed health questionnaire from either parents or children. Hence, a valid response did not require both parents and children to complete all requested questionnaires. To answer the first research question regarding mode equivalence across the different survey designs, we compared the single-mode control group with each of the 3 mixed-mode groups. To answer the second research question regarding mode equivalence between the 2 data collection modes, data from all survey designs were pooled (
The sample compositions of participating parents and adolescents were described by various sociodemographic characteristics separately by survey design and data collection mode. The variables examined included individual adolescent characteristics (age, sex, migration background, and highest level of education reached or aspired); parental characteristics (age, marital status, and participating parent); location (municipality size and region [East vs West Germany]); and household properties (education level and net household income). Household education level was measured using the Comparative Analysis of Social Mobility in Industrial Nations [
Study design of the methodological pilot study. SAQ-paper: self-administered paper-based questionnaire; SAQ-Web: self-administered Web-based questionnaire.
Cases used in this study.
Questionnaire type | Survey design | Mode group | |||||||||||
Single-mode design | Sequential mixed-mode design | Concurrent mixed-mode design | Preselect mixed-mode design | ||||||||||
SAQ-papera | SAQ-paper | SAQ-Webb | Total | SAQ-paper | SAQ-Web | Total | SAQ-paper | SAQ-Web | Total | SAQ-paper | SAQ-Web | Total | |
Parent-completed health questionnaires | 970 | 488 | 366 | 854 | 837 | 109 | 946 | 465 | 233 | 698 | 2760 | 708 | 3468 |
Adolescent-completed health questionnaires | 343 | 117 | 152 | 269 | 244 | 46 | 290 | 191 | 101 | 292 | 895 | 299 | 1194 |
aSAQ-paper: self-administered paper-based questionnaire.
bSAQ-Web: self-administered Web-based questionnaire.
Differences between the control group and the different mixed-mode groups and between the 2 data collection modes were tested using chi-squared tests.
A wide range of health status indicators and health behaviors for children and adolescents with high public health relevance were analyzed to identify differences between the mixed-mode designs and the single-mode control group, as well as mode differences between SAQ-paper and SAQ-Web.
Lifetime diagnoses of asthma, hay fever, atopic eczema, and attention-deficit hyperactivity disorder (ADHD) were indicated by parents. Recurrent pain during the last 3 months was measured using the adolescents’ self-reports. Self-rated health (SRH) and chronic diseases were evaluated by parental report using the Minimum European Health Module questions [
Child and adolescent mental health problems were evaluated using the parent- and self-report Strengths and Difficulties Questionnaire (SDQ) [
As indicators of health care use, pediatrician and orthodontist visits during the past 12 months for adolescents and parent-reported visits to any doctor for children under 11 years were analyzed [
Health-related quality of life (HRQoL) was measured using KIDSCREEN-27 for adolescents aged 11-17 years, with 5 subscores for physical and psychological well-being, relationships with peers and parents, and school well-being. Scores were summed and transformed into
Adolescents reported their current smoking status, water pipe consumption during the past 12 months, second-hand smoke exposure [
Following the recommendation of the World Health Organization [
We calculated prevalence rates for dichotomous health indicators and mean values for HRQoL (a scale outcome) by survey design and data collection mode. We compared these values using
Due to the different sample compositions of the SAQ-paper and SAQ-Web groups (see the Results), it was necessary to control for sociodemographic characteristics to identify possible mode effects. Survey modes can differ in selection (different population groups prefer different modes) and measurement (different answers are given by the same person under different modes of administration), so these differences are confounded [
For the survey design comparison, crude as well as adjusted prevalence rates and mean values were calculated. A statistical test for diversity was conducted between the single-mode control group design and each of the 3 mixed-mode designs. Because the survey design samples did not differ in sociodemographic characteristics (see the Results) and there were only marginal differences between the 2 approaches, only the results for crude prevalence rates or mean values without adjustment for sociodemographic characteristics to simplify the presentation of results are shown here.
In total, we analyzed 12 health indicators using the parental sample and 28 using the adolescent sample. For these health indicators, we tested each mixed-mode survey design against the control group. Additionally, we used 2 other statistical tests to identify differences between the data collection modes, using first the crude values and then the adjusted values.
Regarding the research questions, a sensitive approach to detect possible differences (ie, a higher probability of accepting the null hypothesis) is needed. Therefore, we decided to address the statistical problem of multiple testing by correcting the significance level only for the number of tests performed for each health indicator. This was done only for tests comparing the different survey designs. We used the Bonferroni correction method to neutralize the accumulation of α-error [
There were no statistically significant differences in sample composition between the mixed-mode survey designs and the single-mode control group for participating parents. However, the sample sociodemographic characteristics differed significantly between data collection modes (
For the responding children and adolescents (aged 11-17 years), there were no statistically significant differences in sociodemographic characteristics between the different survey designs, but adolescents responding online were more often male, had reached or aspired to reach higher levels of education, and were more likely to live in households with higher education and higher income, compared with adolescents who responded to the SAQ-paper (
The analyzed indicators of physical and mental health status showed no statistically significant differences by survey design or data collection mode (
No differences in the crude or adjusted prevalence rates were found in adolescent-reported 12-month use of pediatric or orthodontic services (
HRQoL, measured using the 5 dimensions of the KIDSCREEN-27 for adolescents, was the only indicator scale analyzed. Independent of adjustment, there were no significant differences between the 2 data collection modes (SAQ-paper and SAQ-Web) for any of the observed dimensions (
The crude prevalence of lifetime alcohol consumption (self-reported by adolescents aged 11-17 years), as well as hazardous consumption and binge drinking (based on AUDIT-C reports), showed significant differences between SAQ-paper and SAQ-Web, with higher levels of alcohol consumption reported by online participants (
There were no differences in other health behaviors assessed (tobacco consumption, physical activity, and media consumption) by survey design or data collection mode.
Physical and mental health status of children and adolescents aged 0-17 years by survey design and data collection mode (prevalence rates).
Physical and mental health status | Survey designa | Data collection modea | |||||||||||||||||||
Single-mode design | Sequential MMb design | Concurrent MMb design | Preselect MMb design | SAQ-Webc (crude) | SAQ-paperd (crude) | SAQ-Webc |
SAQ-paperd (adjustede) | ||||||||||||||
n (%) | n (%) | n (%) | n (%) | n (%) | n (%) | n (%) | n (%) | ||||||||||||||
Self-rated health (very good, good)g | 964 (97.6) | 848 (97.1) | .46 | 942 (97.3) | .71 | 693 (97.3) | .65 | 708 (97.0) | 2739 (97.41) | .60 | 708 (96.7) | 2739 (97.48) | .31 | ||||||||
Self-rated health (very good, good)h | 338 (90.2) | 267 (92.1) | .41 | 286 (87.1) | .22 | 290 (92.8) | .26 | 299 (90.6) | 881 (90.5) | .93 | 299 (90.1) | 881 (90.6) | .81 | ||||||||
Chronic disease (yes)g | 962 (10.2) | 849 (10.7) | .71 | 941 (7.4) | .03e | 691 (9.6) | .67 | 708 (10.2) | 2735 (9.25) | .47 | 708 (10.5) | 2735 (9.18) | .31 | ||||||||
Impairment owing to health problems (yes)g | 956 (3.2) | 851 (4.0) | .39 | 937 (3.4) | .83 | 692 (3.6) | .69 | 708 (4.1) | 2728 (3.41) | .40 | 708 (4.5) | 2728 (3.36) | .19 | ||||||||
Bronchial asthma (lifetime diagnosis)g | 951 (4.5) | 838 (6.1) | .14 | 928 (4.6) | .91 | 675 (5.2) | .54 | 701 (4.6) | 2691 (5.20) | .48 | 701 (5.0) | 2691 (5.10) | .91 | ||||||||
Neurodermatitis (lifetime diagnosis)g | 951 (15.0) | 839 (16.0) | .59 | 927 (13.8) | .45 | 678 (17.4) | .20 | 704 (17.0) | 2691 (14.98) | .19 | 704 (17.5) | 2691 (14.95) | .12 | ||||||||
Hay fever (lifetime diagnosis)g | 955 (11.2) | 831 (11.2) | .99 | 931 (10.7) | .75 | 680 (11.5) | .87 | 702 (11.3) | 2695 (11.09) | .91 | 702 (12.0) | 2695 (10.96) | .50 | ||||||||
Headache (recurrent during the last 3 months)h | 331 (35.6) | 260 (37.7) | .61 | 274 (33.9) | .66 | 282 (31.2) | .24 | 299 (32.1) | 847 (35.4) | .30 | 299 (33.8) | 847 (34.8) | .77 | ||||||||
Dorsal pain (recurrent during the last 3 months)h | 321 (25.9) | 257 (23.0) | .42 | 265 (24.2) | .64 | 276 (27.5) | .64 | 298 (22.5) | 820 (26.2) | .19 | 298 (23.6) | 820 (25.8) | .45 | ||||||||
Any pain (recurrent during the last 3 months)h | 343 (84.3) | 265 (83.0) | .68 | 288 (81.9) | .44 | 291 (81.1) | .30 | 294 (82.3) | 892 (82.7) | .87 | 294 (82.6) | 892 (82.6) | .99 | ||||||||
Attention-deficit hyperactivity disorder (lifetime diagnosis)g | 824 (5.1) | 716 (4.1) | .33 | 794 (5.0) | .96 | 593 (3.2) | .07 | 595 (3.5) | 2332 (4.67) | .19 | 595 (3.9) | 2332 (4.56) | .45 | ||||||||
At risk for emotional and behavioral symptomsg | 788 (13.6) | 695 (15.5) | .29 | 754 (13.7) | .96 | 577 (12.7) | .62 | 598 (13.0) | 2216 (14.12) | .49 | 598 (13.9) | 2216 (13.89) | .97 | ||||||||
At risk for emotional and behavioral symptomsh | 331 (9.7) | 255 (12.2) | .34 | 278 (14.0) | .10 | 283 (14.1) | .09 | 293 (13.7) | 853 (12.0) | .46 | 293 (13.) | 853 (11.9) | .41 | ||||||||
At risk for impairment following psychosocial problemsg | 820 (18.5) | 719 (17.4) | .56 | 789 (17.2) | .50 | 592 (16.6) | .33 | 600 (16.3) | 2320 (17.80) | .39 | 600 (17.5) | 2320 (17.48) | .99 | ||||||||
At risk for impairment following psychosocial problemsh | 339 (16.5) | 263 (19.0) | .43 | 286 (16.4) | .98 | 288 (18.1) | .61 | 298 (19.8) | 877 (16.5) | .21 | 298 (19.5) | 877 (16.6) | .30 | ||||||||
Obesity of children (aged 0-10 years)g | 401 (2.7) | 407 (2.7) | .97 | 423 (2.4) | .73 | 302 (4.0) | .38 | 370 (1.9) | 1163 (3.18) | .14 | 370 (2.7) | 1163 (3.49) | .46 | ||||||||
Obesity of adolescents (aged 11-17 years)h | 330 (5.2) | 260 (4.6) | .76 | 275 (4.4) | .65 | 275 (3.6) | .36 | 291 (3.4) | 848 (4.8) | .28 | 291 (4.1) | 848 (4.6) | .78 |
aSample sizes are shown in
bMM: mixed-mode.
cSAQ-Web: self-administered Web-based questionnaire.
dSAQ-paper: self-administered paper-based questionnaire.
eAdjusted for age of the child (adolescent, parent); sex of the child (adolescent); relationship to the child (parent); household income (adolescent, parent); parental education (adolescent, parent); adolescent education (adolescent); region (adolescent, parent); municipality size (adolescent, parent); and parental marital status (parent).
fTested against single-mode control group.
gProxy-reported by parents of children and adolescents aged 0-17 years.
hSelf-reported by adolescents aged 11-17 years.
Health care utilization among children and adolescents aged 0-17 years by survey design and data collection mode (prevalence rates).
Health care utilization | Survey designa | Data collection modea | ||||||||||||
Single-mode design | Sequential MMbdesign | Concurrent MMbdesign | Preselect MMbdesign | SAQ-Webc (crude) | SAQ-paperd (crude) | SAQ-Webc |
SAQ-paperd |
|||||||
n (%) | n (%) | n (%) | n (%) | n (%) | n (%) | n (%) | n (%) | |||||||
Any doctor (children aged 0-13 years; past 12 months)g | 970 (91.3) | 804 (92.7) | .31 | 935 (91.8) | .74 | 645 (91.5) | .93 | 594 (95.3) | 2760 (91.05) | <.001 | 594 (92.2) | 2760 (91.69) | .75 | |
Pediatric services (children aged 0-13 years; past 12 months)g | 970 (70.0) | 803 (71.6) | .46 | 935 (72.0) | .34 | 645 (71.6) | .48 | 593 (80.9) | 2760 (69.16) | <.001 | 593 (70.9) | 2760 (71.47) | .75 | |
Pediatric services (adolescents aged 14-17 years; past 12 months)h | 205 (35.6) | 147 (35.4) | .96 | 155 (29.0) | .18 | 159 (35.8) | .96 | 164 (34.1) | 502 (34.1) | .99 | 164 (33.8) | 502 (34.2) | .92 | |
Orthodontic services (adolescents aged 14-17 years; past 12 months)h | 332 (40.4) | 260 (37.3) | .45 | 281 (39.5) | .83 | 288 (44.4) | .31 | 295 (39.7) | 865 (40.7) | .75 | 295 (40.3) | 865 (40.5) | .96 |
aSample sizes are shown in
bMM: mixed-mode.
cSAQ-Web: self-administered Web-based questionnaire.
dSAQ-paper: self-administered paper-based questionnaire.
eAdjusted for age of the child (adolescent, parent); sex of the child (adolescent); relationship to the child (parent); household income (adolescent, parent); parental education (adolescent, parent); adolescent education (adolescent); region (adolescent, parent); municipality size (adolescent, parent); and parental marital status (parent).
fTested against single-mode control group.
gProxy-reported by parents of children and adolescents aged 0-17 years.
hSelf-reported by adolescents aged 11-17 years.
Health-related quality of life of adolescents aged 11-17 years by survey design and data collection mode (mean values).
Health related quality of life | Survey designa | Data collection modea | |||||||||||||||
Single-mode design | Sequential MMbdesign | Concurrent MMbdesign | Preselect MMbdesign | SAQ-Webc (crude) | SAQ-paperd (crude) | SAQ-Webc |
SAQ-paperd |
||||||||||
n (%) | n (%) | n (%) | n (%) | n (%) | n (%) | n (%) | n (%) | ||||||||||
Physical well-being (mean)f | 337 (49.9) | 263 (49.9) | .98 | 282 (49.2) | .38 | 290 (49.0) | .21 | 297 (49.5) | 874 (49.5) | .95 | 297 (49.0) | 874 (49.7) | .26 | ||||
Psychological well-being (mean)f | 336 (51.2) | 265 (50.0) | .14 | 283 (49.6) | .05g | 291 (49.8) | .07 | 297 (49.6) | 877 (50.4) | .20 | 297 (49.3) | 877 (50.5) | .07 | ||||
Relations with parents (mean)f | 331 (53.5) | 263 (53.0) | .52 | 283 (53.3) | .74 | 290 (52.0) | .04g | 296 (52.5) | 871 (53.1) | .31 | 296 (52.3) | 871 (53.2) | .18 | ||||
Relations with peers (mean)f | 340 (51.2) | 266 (50.1) | .14 | 288 (50.5) | .37 | 292 (50.1) | .13 | 297 (49.8) | 888 (50.8) | .12 | 297 (49.9) | 888 (50.7) | .18 | ||||
Well-being in school (mean)f | 335 (51.9) | 265 (51.6) | .66 | 283 (50.8) | .08 | 285 (51.4) | .43 | 294 (51.1) | 873 (51.60) | .31 | 294 (50.9) | 873 (51.70) | .17 |
aSample sizes are shown in
bMM: mixed-mode.
cSAQ-paper: self-administered paper-based questionnaire.
dSAQ-Web: self-administered Web-based questionnaire.
eAdjusted for age of the child (adolescent, parent); sex of the child (adolescent); relationship to the child (parent); household income (adolescent, parent); parental education (adolescent, parent); adolescent education (adolescent); region (adolescent, parent); municipality size (adolescent, parent); and parental marital status (parent).
fSelf-reported by adolescents aged 11-17 years.
gNot significant,
hTested against single-mode control group.
Health behaviors of adolescents aged 11-17 years by survey design and data collection mode (prevalence-rates).
Health related quality of life | Survey designa | Data collection modea | |||||||||||||
Single-mode design | Sequential MMb design | Concurrent MMb design | Preselect MMb design | SAQ-Webc (crude) | SAQ-paperd (crude) | SAQ-Webc (adjustede) | SAQ-paperd (adjustede) | ||||||||
n (%) | n (%) | n (%) | n (%) | n (%) | n (%) | n (%) | n (%) | ||||||||
Current smoking status (yes)f | 342 (8.8) | 264 (8.3) | .85 | 288 (11.5) | .27 | 291 (10.7) | .43 | 295 (11.5) | 889 (9.2) | .27 | 295 (11.6) | 889 (9.2) | .25 | ||
Water pipe consumption (past 12 months, yes)f | 339 (18.0) | 262 (17.9) | .99 | 286 (18.2) | .95 | 291 (16.2) | .54 | 295 (20.3) | 882 (16.7) | .17 | 295 (19.1) | 882 (17.1) | .38 | ||
Second-hand smoke exposure (yes)f | 308 (13.6) | 239 (15.5) | .55 | 249 (10.8) | .31 | 259 (14.3) | .82 | 261 (11.1) | 793 (14.4) | .16 | 261 (12.9) | 793 (13.8) | .74 | ||
Lifetime consumption of alcohol (yes)f | 343 (51.0) | 262 (56.1) | .21 | 290 (50.7) | .93 | 292 (54.8) | .34 | 295 (61.4) | 891 (50.3) | .001 | 295 (60.1) | 891 (50.7) | <.001 | ||
Hazardous alcohol consumption (based on AUDIT-Cg)f | 328 (12.2) | 255 (12.5) | .90 | 284 (10.6) | .53 | 284 (12.7) | .86 | 293 (17.1) | 857 (10.3) | .01 | 293 (14.9) | 857 (11.9) | .16 | ||
Binge drinking (based on AUDIT-C)f | 339 (7.7) | 262 (7.7) | .99 | 288 (5.6) | .29 | 291 (8.9) | .57 | 295 (10.5) | 884 (6.5) | .04 | 295 (10.0) | 884 (7.2) | .13 | ||
Physical activity consistent with WHOh guidelinesf | 343 (6.1) | 265 (5.3) | .66 | 288 (3.1) | .07 | 292 (3.8) | .17 | 295 (4.1) | 892 (4.8) | .58 | 295 (3.3) | 892 (5.2) | .16 | ||
Low physical activityf | 343 (14.9) | 265 (12.5) | .39 | 288 (15.3) | .89 | 292 (17.1) | .44 | 295 (13.9) | 892 (15.2) | .57 | 295 (14.5) | 892 (15.0) | .82 | ||
Currently doing sportsf | 342 (79.5) | 265 (81.9) | .46 | 286 (78.0) | .63 | 291 (77.0) | .44 | 295 (80.3) | 888 (78.6) | .52 | 295 (79.3) | 888 (78.9) | .89 | ||
Social media (>2 hours/day)f | 341 (20.2) | 262 (19.1) | .72 | 289 (17.3) | .35 | 290 (19.7) | .86 | 295 (20.0) | 886 (18.8) | .67 | 295 (21.3) | 886 (18.4) | .30 | ||
TV (>2 hours/day)f | 341 (42.2) | 262 (41.6) | .88 | 289 (41.2) | .79 | 292 (38.0) | .28 | 294 (39.1) | 889 (41.4) | .49 | 294 (41.5) | 889 (40.6) | .79 | ||
Game console (>2 hours/day)f | 339 (18.9) | 261 (19.9) | .75 | 287 (18.5) | .90 | 290 (12.8) | .03i | 295 (20.0) | 881 (16.7) | .21 | 295 (19.9) | 881 (16.7) | .22 |
aSample sizes are shown in
bMM: mixed-mode.
cSAQ-Web: self-administered Web-based questionnaire.
dSAQ-paper: self-administered paper-based questionnaire.
eAdjusted for age of the child (adolescent, parent); sex of the child (adolescent); relationship to the child (parent); household income (adolescent, parent); parental education (adolescent, parent); adolescent education (adolescent); region (adolescent, parent); municipality size (adolescent, parent); and parental marital status (parent).
fSelf-reported by adolescents aged 11-17 years.
gAUDIT-C: Alcohol Use Disorders Identification Test.
hWHO: World Health Organization; sample sizes are shown in
iNot significant,
The main aim of this study was to examine the risk of mode effects in a mixed-mode health interview survey for children and adolescents that combined paper-and-pencil questionnaires and Web-based questionnaires. Therefore, we compared prevalence rates and mean values of a broad range of health indicators from 3 alternative mixed-mode designs (all combining paper-and-pencil and Web-based questionnaires) with a single-mode control group (paper-and-pencil only). We also compared results between online respondents and paper-and-pencil respondents regardless of the survey design. First, we examined differences in sociodemographic characteristics by survey design and data collection mode, as it is well documented that sociodemographic characteristics are associated with health status and health behavior [
Consistent with previous findings, the sample composition of responding parents and of responding adolescents differed by data collection mode. We confirmed the so-called “digital divide” [
Prevalence rates of health complaints, such as diagnosed allergies, diagnosed ADHD, obesity, and chronic pain, were equivalent between the modes, as previous studies of adults [
We found similar prevalence rates for SRH, chronic diseases, and impairment owing to health problems between SAQ-paper and SAQ-Web respondents. The 2 previous studies examining these health indicators among adults in general [
For mental and psychosocial problems, we calculated risk groups for emotional and behavioral problems and for impairment owing to psychosocial problems based on SDQ scores [
In their review of 55 studies investigating 79 instruments, Campbell et al [
All reports of health care utilization were equivalent between the self-administered modes; this is consistent with prior empirical results, including studies of adult vaccination use [
Most of the analyzed adolescent health behaviors (current smoking, 12-month water pipe consumption, second-hand smoke exposure, physical activity, and screen-based media use) showed comparable results and no differences between the 2 modes. These results are consistent with the results of other studies on adolescents [
Considering alcohol consumption, the crude and adjusted prevalence rates for lifetime consumption were significantly higher among SAQ-Web-responding adolescents. After adjusting for sociodemographic characteristics, the difference decreased but could not be explained by the sociodemographic differences between the 2 groups of respondents. The prevalence of hazardous consumption and binge drinking were comparable between data collection modes after controlling for sample composition.
Most previous studies have reported no statistically significant differences in alcohol consumption among adolescents or young adults by these 2 data collection modes [
The higher rate of reported lifetime alcohol consumption among SAQ-Web-responding adolescents, in the absence of frequently reported hazardous consumption or binge drinking, may be interpreted in multiple ways. For example, this may be a result of different sample properties, such as SAQ-Web-preferring adolescents being more likely to experiment with alcohol consumption. However, it is also possible that this result is a mode effect based on the assumption of identical alcohol consumption in both groups. Web-based questionnaires afford greater privacy because there is no risk of parents checking the responses. Another possible explanation is the lower social orientation in the Internet mode [
Other empirical comparisons of measurement results between different mixed-mode survey designs are rare. In accord with one other result for the adult population [
The strengths of the methodological pilot study are the randomized study design, the population-based sample, and the inclusion of a single-mode control group as a reference to interpret the results. However, there are also some limitations, predominantly the relatively small size of the net samples of the analyzed groups. Each survey design had a relatively low number of cases, so interpretations of the results based on the net samples must be made with caution. Possible differences across the 4 survey designs or between the 2 data collection modes could have been overlooked because of a lack of statistical power, particularly regarding the need for correction for multiple testing. Other limitations concern the external validity of the results; the study was conducted in a German setting using register-based samples of children and adolescents, so the results are difficult to generalize to other countries, settings, or populations.
Our results are consistent with those of most previous studies. We found comparable results between the 2 self-administered modes (SAQ-Web and SAQ-paper) for almost all analyzed health indicators, except for lifetime consumption of alcohol among adolescents aged 11-17 years. Thus, no differences were found between the single-mode control group design and 3 mixed-mode survey designs that combined the 2 data collection modes.
These results suggest that it is possible to measure health indicators for children and adolescents using a mixed-mode design combining SAQ-Web and SAQ-paper methods, with a low risk of mode effects and high comparability across different mixed-mode survey designs combining these 2 data collection modes [
Checklist for Reporting Results of Internet E-Surveys (CHERRIES).
Sociodemographic characteristics of responding parents of children aged 0-17 years by survey design and data collection mode mode.
Sociodemographic characteristics of responding adolescents aged 11-17 years by survey design and data collection mode.
attention-deficit hyperactivity disorder
Alcohol Use Disorders Identification Test
health-related quality of life
The German Health Interview and Examination Survey for Children and Adolescents
mixed-mode
self-administered paper-based questionnaires
self-administered Web-based questionnaires
Strengths and Difficulties Questionnaire
self-rated health
This study was funded by the Robert Koch Institute and the German Federal Ministry of Health within the German Health Monitoring System. The authors thank Diane Williams, PhD, from Edanz Group for editing a draft of this manuscript.
None declared.