Published on 18.09.20 in Vol 22, No 9 (2020): September
Primary Care Pre-Visit Electronic Patient Questionnaire for Asthma: Uptake Analysis and Predictor Modeling
Background: mHealth tablet-based interventions are increasingly being studied and deployed in various health care settings, yet little knowledge exists regarding patient uptake and acceptance or how patient demographics influence these important implementation metrics.
Objective: To determine which factors influence the uptake and successful completion of an mHealth tablet questionnaire by analyzing its implementation in a primary care setting.
Methods: We prospectively studied a patient-facing electronic touch-tablet asthma questionnaire deployed as part of the Electronic Asthma Management System. We describe tablet uptake and completion rates and corresponding predictor models for these behaviors.
Results: The tablet was offered to and accepted by patients in 891/1715 (52.0%) visits. Patients refused the tablet in 33.0% (439/1330) visits in which it was successfully offered. Patients aged older than 65 years of age (odds ratio [OR] 2.30, 95% CI 1.33-3.95) and with concurrent chronic obstructive pulmonary disease (OR 2.22, 95% CI 1.05-4.67) were more likely to refuse the tablet, and those on an asthma medication (OR 0.55, 95% CI 0.30-0.99) were less likely to refuse it. Once accepted, the questionnaire was completed in 784/891 (88.0%) instances, with those on an asthma medication (OR 0.53, 95% CI 0.32-0.88) being less likely to leave it incomplete.
Conclusions: Older age predicted initial tablet refusal but not tablet questionnaire completion, suggesting that perceptions of mHealth among older adults may negatively impact uptake, independent of usability. The influence of being on an asthma medication suggests that disease severity may also mediate mHealth acceptance. Although use of mHealth questionnaires is growing rapidly across health care settings and diseases, few studies describe their real-world acceptance and its predictors. Our results should be complemented by qualitative methods to identify barriers and enablers to uptake and may inform technological and implementation strategies to drive successful usage.
J Med Internet Res 2020;22(9):e19358
The use of information technology in health care has proliferated rapidly over the past 20 years. For example, patient-facing handheld touchscreen computers (tablets)—a form of mobile health (mHealth)—are increasingly being used and studied across a range of clinical settings [- ]. Tablets are particularly well suited for collecting information directly from patients and have several potential advantages over traditional paper-based methods, including increased efficiency, facilitated data entry, greater data security, and ease of data transport [ , ]. Tablet-based patient questionnaires are also perceived favorably by patients, can promote greater patient-centered care, and are cost efficient [ , ]. Given these advantages, many clinicians and health care organizations are now integrating tablets into care. In Canada, 3500 physicians are already using electronic medical record–integrated tablet-based mHealth questionnaires, and the Canadian Medical Association has partnered with technology providers to further promote the use of tablets [ ].
Patient uptake is critical to realizing the health system benefits of tablet-based questionnaires. However, despite broad interest and increasing use of tablets, there is limited research evaluating real-world mHealth tablet uptake and acceptability. Certain groups, such as older patients and those with lower educational attainment, may have disproportional barriers to accepting and using tablets [, ] and a higher risk of experiencing technology-induced medical errors [ ]. Accordingly, further research is required to understand the differences in uptake across demographics and the reasons for these differences, with a view toward improving implementation processes and technologies to enable broad acceptance.
We recently completed a prospective study of a chronic disease management tool called the Electronic Asthma Management System (eAMS) , which consisted of a patient-facing electronic touch-tablet asthma questionnaire for use in waiting rooms, that directly informed a provider-facing computerized clinical decision support system. Herein, we report on the implementation of this tablet-based questionnaire, including patient uptake, completion rates, and predictors of these metrics, in a real-world primary care setting.
The eAMS questionnaire was created as a web-based app for Apple iPad and requires approximately 10 minutes to complete . Questions were designed to (1) document current asthma control criteria, (2) identify current asthma medications used and adherence to these medications, and (3) collect information required to create a personalized self-management asthma action plan. Questionnaire data were transferred to a computerized clinical decision support system, which provided primary care clinicians with patients’ asthma control status, evidence-based recommendations for medication adjustments, and a prepopulated asthma action plan, integrated into the electronic medical record in real time [ ]. A questionnaire prototype was developed by asthma and knowledge translation experts in our research group by applying evidence-based electronic questionnaire design principles [ ]. After pilot testing in a convenience sample of clinicians and researchers, the prototype’s content and usability were iteratively refined using a rapid-cycle design process with serial focus groups (in total, 20 patients with asthma), followed by summative qualitative and quantitative analyses [ , ]. The questionnaire was designed and tested for use across a range of ages (with 30% of included participants aged older than 60 years of age), disease severities, and technological experience [ , ]. The system had a final System Usability Scale score of 84.2. The reported mean System Usability Scale score for web-based systems is 68, with a score of 84.2 corresponding to a subjective usability rating of excellent and a 95th percentile rank. The high System Usability Scale score was supported by favorable Likert-scale question responses and qualitative findings [ ].
Study Design and Population
The eAMS patient questionnaire and clinical decision support system were tested in a 1-year prospective interrupted time-series design study across 3 primary care sites in Ontario, Canada . Detailed questionnaire uptake and completion data were captured at 2 of these clinics. We included patients aged older than 16 years, with a physician diagnosis of asthma (these patients were identified according to an electronic medical record search algorithm validated in this population and with the same electronic medical records used in study sites [ ]), and with an asthma medication prescription in the prior year (but not necessarily using asthma medication at the time of the study). We excluded patients who were pregnant (as the questionnaire was designed in part to inform creation of a physician-delivered asthma action plan, and there is limited evidence for asthma action plan use during pregnancy [ ]), had cognitive or language difficulties, had a life expectancy of <1 year, or had been on a medication for chronic obstructive pulmonary disease in the prior year [ ]. The study was approved by St. Michael’s Hospital and Hamilton Integrated research ethics boards.
Each day, an on-site research assistant at each site used a database query to identify all eligible patients who were booked to see a physician or nurse practitioner (excluding visits exclusively for administration of injections such as the flu vaccine). The research assistant was tasked with identifying patients for questionnaire completion in the clinic waiting room and inviting them to complete the questionnaire on 1 of 2 available tablet devices, prior to their appointment. Patients who had completed the questionnaire within the prior 28 days were not asked to repeat it.
Research assistants recorded tablet uptake and completion information for each eligible patient in a standardized Excel (Microsoft Inc) spreadsheet. This information included whether or not the tablet was given to the patient and the reason it was not given (if applicable) and whether the questionnaire was fully completed and reasons for noncompletion (if applicable, though patients were not required to provide a specific reason if they refused to accept the tablet or complete the questionnaire). We also collected demographic and clinical data including age, sex, smoking status, asthma medication prescriptions, history of emergency room visits or hospitalizations for asthma (since 2003), the presence or absence of prior physician documentation of a diagnosis of asthma or chronic obstructive pulmonary disease, and whether the patient’s asthma was currently under control through an electronic chart audit.
The primary outcomes were tablet questionnaire uptake—the proportion of visits in which the tablet questionnaire was offered to and accepted by patients—and tablet questionnaire completion—the proportion of visits in which the questionnaire was completed by patients when uptake was successful. Secondary outcomes were the reasons for failed uptake and the reasons for failed completion. Among cases where the tablet questionnaire was offered to patients, we also sought to determine if tablet refusal and questionnaire noncompletion were influenced by the following patient-level variables (established a priori, based on clinical relevance): age, sex, smoking status, asthma medication prescription, physician-documented diagnosis of asthma or chronic obstructive pulmonary disease, history of emergency room visit or hospitalization related to asthma, and current asthma control status. Finally, we evaluated the proportion of patients who accepted the tablet questionnaire at least once who were willing to accept it on a subsequent occasion, as well as predictors of accepting it on a subsequent occasion. We compared key outcomes between clinical sites.
We present summary statistics for tablet uptake and completion rates (by patient visit), and reasons for uptake and completion failures. We used chi-square and Fisher exact tests to compare results between sites. We then constructed a logistic regression model to evaluate the influence of patient characteristics on the odds of tablet refusal and questionnaire noncompletion. In order to account for repeated measures at the patient level and clustering at the clinic level, generalized estimating equations were used . Though the patient visits were not predetermined by the study investigators in this real-world study and follow-up was irregular, generalized estimating equations remain appropriate for analysis as the outcomes of interest were not clinical in nature, thus outcome-dependent follow-up was unlikely [ , ]. In order to ensure that this assumption was valid, another logistic regression model was constructed as a sensitivity analysis, modeling tablet refusal on only first patient visits, removing any potential residual confounding effects of repeated visits. Model performance was assessed by examining the concordance (C) statistic. At the patient level, we present summary statistics for the number of times each patient accepted the tablet questionnaire. We constructed a logistic regression model identifying predictors of refusing the tablet on the second occasion after acceptance on the first occasion. Additional available predictors for this patient-level analysis were time between completion of the first questionnaire and next time being offered it, and whether or not the questionnaire was fully completed on the first occasion it was accepted. All statistical analyses were performed using SAS (university edition; SAS Institute). A P value<.05 was statistically significant.
Primary Outcome: Tablet Questionnaire Uptake
There were 612 eligible patients who made 1715 eligible visits during the study period (visits per patient: median 2, IQR 1-4) (). The tablet questionnaire was successfully offered and accepted in 891/1715 visits, resulting in an overall uptake of 52.0%. Failed uptake was due to the tablet not being offered in 385/1715 (22.5%) visits and not being accepted in 439/1330 (33.0%) visits in which it was offered. The tablet questionnaire was completed in 784/891 (88.0%) visits in which uptake (tablet offered and accepted) was successful ( ).
|All, n (%)||Clinic 1, n (%)||Clinic 2, n (%)|
|Tablet offered to patient||1715 (100)||1348 (100)||367 (100)|
|Yes||1330 (77.6)||1084 (80.4)||246 (67.0)|
|No||385 (22.5)||264 (19.6)||121 (33.0)|
|Tablet accepted by patient after being offered||1330 (100)||1084 (100)||246 (100)|
|Yes||891 (67.0)||699 (64.5)||192 (78.0)|
|No||439 (33.0)||385 (35.5)||54 (22.0)|
|Questionnaire completed after tablet offered and accepted by patient||891 (100)||699 (100)||192 (100)|
|Yes||784 (88.0)||618 (88.4)||166 (86.5)|
|No||107 (12.0)||81 (11.6)||26 (13.5)|
Secondary Outcomes: Reasons for Tablet Uptake and Completion Failures
Both technology-related and logistical issues prevented the tablet from being offered to patients. Most patients who refused the tablet did not provide a reason. Once accepted, patients failed to complete the questionnaire for a variety of reasons, including aborting the questionnaire due to difficulties in completing it, technology-related failures, or clinicians calling patients in from the waiting room ().
|Reasons||All visits, n (%)||Clinic 1 visits, n (%)||Clinic 2 visits, n (%)||P value|
|Reasons tablet not offered to patients||385 (100)||264 (100)||121 (100)||<.001a|
|Patient missed by research assistant||307 (79.7)||215 (81.4)||92 (76.0)|
|Technology-related issueb||48 (12.5)||38 (14.4)||10 (8.3)|
|Both tablets already in use||12 (3.1)||7 (2.7)||5 (4.1)|
|Not documented||18 (4.7)||4 (1.5)||14 (11.6)|
|Reasons tablet not accepted by patients when offered||439 (100)||385 (100)||54 (100)||.70c|
|Patient refusal (denies asthma diagnosis)||54 (12.3)||46 (12.0)||8 (14.8)|
|Patient refusal (indicates too difficult to use)||24 (5.5)||22 (5.7)||2 (3.7)|
|Patient refusal (no reason provided)||351 (80.0)||307 (79.7)||44 (81.5)|
|Caregiver refusal||10 (2.3)||10 (2.6)||0 (0.0)|
|Reasons questionnaire not completed when tablet accepted||107 (100)||81 (100)||26 (100)||<.001c|
|Patient aborted (no reason provided)||46 (43.0)||45 (55.5)||1 (3.9)|
|Patient aborted (indicates too difficult to complete)||17 (15.9)||8 (9.9)||9 (34.6)|
|Patient aborted (started, then denied asthma diagnosis)||11 (10.3)||6 (7.4)||5 (19.2)|
|Technology-related issueb||5 (4.7)||2 (2.5)||3 (11.5)|
|Clinician aborted (called patient in prior to finishing)||28 (26.1)||20 (24.7)||8 (30.8)|
aChi-square test was used.
bTechnology-related issues included poor Wi-Fi or network connectivity and tablet software or hardware malfunctions.
cFisher exact test was used.
Predictors of Tablet Questionnaire Refusal
Among the 1330 visits in which the tablet was successfully offered, we were able to match 1026 visits (77.1%) to patient characteristics for modeling. Among these 1026 visits, the tablet was accepted on 679 (66.2%) occasions and refused on 347 (33.8%) occasions. In the multivariable model, being 65 years and older was associated with the greatest odds of tablet questionnaire refusal (odds ratio [OR] 2.30, 95% CI 1.34-3.95). Having a previously documented diagnosis of chronic obstructive pulmonary disease was also associated with increased odds of refusal (OR 2.22, 95% CI 1.05-4.67). Conversely, a current prescription for an asthma medication was associated with lower odds of refusal (OR 0.55, 95% CI 0.30-0.99). The model demonstrated good predictive ability, with a C statistic of 0.67. All predictors are shown in. Patient asthma control level was missing in 40% of visits (a known care gap [ ]) and was therefore excluded from the model.
In the multivariable model constructed with only the data from 451 first patient visits, the tablet was accepted on 407 (90.2%) occasions and refused on 44 (9.8%) occasions. In order to limit the potential for overspecification, only the covariates meeting or approaching statistical significance in the original model or those that were thought to be clinically important were included in the model, which included age ≥65 years, sex, current asthma medication prescription, physician-documented diagnosis of chronic obstructive pulmonary disease, and clinic site. In this sensitivity analysis, being 65 years of age or older was again associated with the greatest odds of tablet questionnaire refusal (OR 2.79, 95% CI 1.39-5.59), controlling for other confounders. Prescription of an asthma medication and physician diagnosis of chronic obstructive pulmonary disease also remained statistically significant predictors. Full results from the sensitivity analysis can be found in.
|Patient-level descriptors||Visits (n=1026), n (%)||Tablet refused (proportion of visits), n (%)||Odds ratio (95% CI)|
|<65 years old||750 (73.1)||208 (27.7)||reference|
|≥65 years old||276 (26.9)||139 (50.4)||2.30 (1.33-3.95)|
|Male||263 (25.6)||81 (30.8)||reference|
|Female||763 (74.4)||266 (34.9)||1.18 (0.68-2.06)|
|Current asthma medication prescription|
|No asthma medications||218 (21.3)||90 (41.3)||reference|
|Any asthma medication||808 (78.7)||257 (31.8)||0.55 (0.30-0.99)|
|Lifelong nonsmoker||489 (47.7)||161 (32.9)||reference|
|Current or former smoker||432 (42.1)||136 (31.5)||0.84 (0.48-1.46)|
|Smoking history not documented||105 (10.2)||50 (47.6)||1.74 (0.79-3.83)|
|Physician-documented diagnosis of COPDa|
|Absent||887 (86.5)||274 (30.9)||reference|
|Present||139 (13.5)||73 (52.5)||2.22 (1.05-4.67)|
|Physician-documented diagnosis of asthma|
|Absent||310 (30.2)||106 (34.2)||reference|
|Present||716 (69.8)||241 (33.7)||1.13 (0.68-1.89)|
|Prior ERb visit or hospitalization for asthma|
|No||951 (92.7)||330 (34.7)||reference|
|Yes||75 (7.3)||17 (22.7)||0.59 (0.15-2.32)|
|Site 2||169 (16.5)||35 (20.7)||reference|
|Site 1||857 (83.5)||312 (36.4)||1.63 (0.79-3.35)|
aCOPD: chronic obstructive pulmonary disease.
bER: emergency room.
Predictors of Failure to Complete Questionnaire After Accepting Tablet
Among the 891 visits in which patients accepted the tablet, we were able to match 706 visits (79.2%) to patient characteristics for modeling. After removing 27 visits where the questionnaire was not completed for reasons outside of the patient’s control (such as the patient being called in prior to finishing or technology-related issues), 679 visits remained, among which the questionnaire was completed on 605 (89.1%) occasions and not completed on 74 (10.9%) occasions. In the multivariable model considering these visits, use of asthma medications was the only statistically significant predictor and was associated with lower odds of failing to complete the questionnaire (OR 0.53, 95% CI 0.32-0.88). The logistic model demonstrated adequate predictive ability with a C statistic of 0.64. All predictors are shown in. Asthma control level was missing in 40% of visits and was therefore excluded from the model.
|Patient-level descriptors||Visits (n=1026), n (%)||Tablet refused (proportion of visits), n (%)||Odds ratio (95% CI)|
|<65 years old||542 (79.8)||53 (9.8)||reference|
|≥65 years old||137 (20.2)||21 (15.3)||1.48 (0.88-2.49)|
|Male||182 (26.8)||24 (13.2)||reference|
|Female||497 (73.2)||50 (10.1)||0.77 (0.45-1.31)|
|Current asthma medication prescription|
|No asthma medications||128 (18.8)||22 (17.2)||reference|
|Any asthma medication||551 (81.2)||52 (9.4)||0.53 (0.32-0.88)|
|Lifelong nonsmoker||328 (48.3)||35 (10.7)||reference|
|Current or former smoker||296 (43.6)||36 (12.2)||1.00 (0.60-1.67)|
|Smoking history not documented||55 (8.1)||2 (3.6)||0.48 (0.15-1.62)|
|Physician-documented diagnosis of COPDa|
|Absent||613 (90.3)||62 (10.1)||reference|
|Present||66 (9.7)||12 (18.2)||1.90 (0.96-3.75)|
|Physician-documented diagnosis of asthma|
|Absent||204 (30.0)||30 (14.7)||reference|
|Present||475 (70.0)||44 (9.3)||0.79 (0.47-1.34)|
|Prior ERb visit or hospitalization for asthma|
|No||621 (91.5)||70 (11.3)||reference|
|Yes||58 (8.5)||4 (6.9)||0.66 (0.24-1.81)|
|Site 2||134 (19.7)||15 (11.2)||reference|
|Site 1||545 (80.3)||59 (10.8)||0.88 (0.47-1.67)|
aCOPD: chronic obstructive pulmonary disease.
bER: emergency room.
Willingness to Accept the Tablet on More Than One Visit
Of the 612 eligible patients in the study, 507 (82.8%) accepted the tablet at least once, 208 (34.0%) accepted the tablet at least twice, 84 (13.7%) accepted the tablet at least three times, and 45 (7.4%) accepted it four or more times.
Among the 507 patients who accepted the tablet at least once, 248 (48.9%) were offered the tablet a second time, among whom 208 (83.9%) accepted it.
In a multivariable model including age older than 65 years, sex, study site, time between accepting the first questionnaire and the next time it was offered, and whether or not the patient completed the questionnaire on the occasion in which it was first accepted, the only statistically significant predictor of refusing to accept the questionnaire a second time after accepting it the first time was noncompletion during the first attempt. Not having completed the questionnaire on the first instance had an odds ratio of 5.73 (95% CI 2.31-14.21) for refusing the questionnaire on the second eligible opportunity (for full details of model, see).
Our analysis revealed that a simple tablet-based questionnaire for use in waiting rooms was successfully offered and accepted by patients in only 52.0% (891/1715) and completed in only 45.7% (784/1715) of eligible visits in primary care over a period of 1 year. Although use of tablet-based questionnaires is growing rapidly in health care, few studies have explored their use in primary care settings, and knowledge about predictors of their real-world uptake is still limited. Although other investigators have reported on patient surveys of tablet acceptability [, , - ], to our knowledge, ours is the first study to examine tablet questionnaire uptake and completion rates in a primary care setting and to quantitatively identify predictors of uptake and successful completion.
The first barrier encountered to tablet questionnaire uptake was failure of the study site research assistants to offer the tablet to eligible patients. This accounted for 17.9% of instances of failed uptake (307/1715). The main reason for this was research assistants missing an eligible patient in the busy clinic environments. The number of missed patients differed between the 2 clinic sites, suggesting that clinic-specific factors such as patient flow, clinic staff engagement, and physical layout may impact successful tablet implementation. This finding aligns with the results of previous systematic reviews  of eHealth implementation in general, which have highlighted the importance of good fit between eHealth systems and clinic workflows. One strategy to mitigate this workflow barrier could be to tie tablet distribution to the patient registration process upon arrival, such that the same personnel member who registers the patient (eg, a clinic receptionist) distributes the tablet. However, identifying appropriate patients, distributing, and managing tablets, including collecting and disinfecting between patients [ ], may not be feasible for a clinic receptionist. Overall, our findings support the notion that strategies to successfully deliver tablets need tailoring to each clinic environment and that prelaunch workflow analysis [ ] and pilot testing may be beneficial.
Of note, technology issues such as Wi-Fi or network failures and hardware or software issues prevented tablet uptake in only 48/1715 (2.8%) cases and completion in only 5/891 (1.0%) cases (). Though technology failures are recognized barriers to eHealth implementation success [ ], they were only minor contributors to failed uptake in our study. This trend is likely to continue as smartphone use becomes more ubiquitous [ ], and electronic questionnaires can instead be offered on patients’ personal devices.
The second major barrier to uptake was patient refusal to accept the tablet questionnaire, which occurred in 33.0% (439/1330) of visits (). Refusal levels did not differ between clinics, suggesting that this barrier is mediated mostly by patient-level factors ( ). In our multivariable model, patient age older than 65 years of age was the strongest predictor of tablet refusal ( ). This finding could be related to known decreased acceptance and comfort-levels with mobile technology among older adults [ , ] and may also reflect a decreased willingness among older adults to adopt newer paradigms of care in which patients are expected to play a more active role [ - ]. Previous researchers have shown somewhat conflicting results with regard to the influence of older age on tablet usage. Some authors have shown that increasing age is associated with decreased preference for tablet questionnaires over paper version [ ], increased time needed to complete tablet questionnaires [ ], and increased difficulty using tablets [ , , , , ]. However, others have reported no significant influence of increasing age on tablet questionnaire usability, feasibility, and preference over paper versions [ , , , ]. Interestingly, being older than 65 years of age was not associated with failure to complete the tablet questionnaire in patients who accepted it. This finding may reflect the fact that our questionnaire development process included older patients (30% were 60 years of age or older) [ ], and thus if the initial acceptance barrier was overcome, the electronic questionnaire had good usability across age ranges. These findings underscore the importance of including older individuals in the development process of mHealth tools but also that optimal tool usability across age ranges alone is not sufficient to drive uptake, and other implementation strategies to engage older individuals need to be considered as well.
Patients with a physician diagnosis of chronic obstructive pulmonary disease were also more likely to refuse the tablet questionnaire (). There is often a degree of overlap between asthma and chronic obstructive pulmonary disease in patients with airways disease, and some patients who experience this overlap may identify more as having chronic obstructive pulmonary disease than asthma [ , ]. It isn’t clear currently if and how patients with asthma–chronic obstructive pulmonary disease overlap engage differently with mHealth interventions targeted at either condition, as this group is poorly studied, but our findings suggest that there may be barriers to mHealth acceptance targeted specifically at one of their codiagnoses.
The only statistically significant predictor in both models was having a prescription for an asthma medication, which was associated with lower odds of both tablet questionnaire refusal and noncompletion (refusal: OR 0.55, 95% CI 0.30-0.99; noncompletion: OR 0.53, 95% CI 0.32-0.88). The fact that patients using asthma medications were less likely to refuse and more likely to complete the tablet questionnaire may reflect a greater interest in and willingness to invest in disease management among patients with more active disease (ie therapy-meriting disease). It is also of note that a fair proportion of both tablet refusals (54/439, 12.3%) and failed questionnaire completions (11/107, 10.3%) were due to patients denying that they had asthma (). Although use of a probabilistic electronic medical record search algorithm may have incorrectly identified some patients as having asthma [ ], it is also likely that some patients with asthma who are not actively taking asthma medications do not self-identify as currently having asthma. Although health status–based differences in mHealth uptake have not previously been reported, research into how an individual’s health status influences desire for autonomy and preference for actively participating in care provides conflicting evidence across different patient populations and care settings [ , , ]. In asthma in particular, there is recent evidence to suggest that there is an overall high level of desire among patients for autonomy and collaborative decision making regardless of disease severity [ , ]. However, how this relates to mHealth and tablet use is not clear and warrants further study.
The barriers and predictors discussed above illuminate important practical factors requiring consideration when implementing a tablet-based questionnaire. Many are likely relevant to any mHealth questionnaire, including smartphone questionnaires. Ideally, strategies to optimize usage of such tools will involve formal measurement of barriers and enablers and a theory-based approach to overcoming and leveraging these, respectively. To this end, we used complementary quantitative and qualitative research methods to specifically identify barriers and enablers to mHealth questionnaire uptake, through a Theoretical Domains Framework analysis . We will use these findings to match specific behavior-change strategies to the identified barriers and enablers, in order to inform system changes and implementation strategies that will drive usage.
Encouragingly, we found that when the tablet was successfully offered to and accepted by patients, overall, questionnaire completion occurred in 88% (784/891) of cases. This finding corresponds with our previous study’s findings [, ] of high questionnaire usability and further validates the importance of patient engagement and iterative questionnaire design with a focus on both content and usability. The unexpected finding that older patient age did not negatively influence tablet questionnaire completion underscores this importance further and supports our decision to deliberately include patients from a wide age range in our tablet questionnaire development process. This finding provides an important addition to the literature, as it implies that provided initial acceptance barriers can be overcome, optimizing mobile interface usability across patient demographics can lead to successful engagement in patient populations that are traditionally thought to be more difficult to reach (for example, older adults). This finding has implications for the design of future implementation strategies for mobile technology in older adults.
Finally, we assessed patients’ willingness to complete tablet questionnaires on more than one occasion. The vast majority (208/248, 83.9%) of patients who accepted the tablet the first time accepted it again on the second opportunity. This suggests that once initial buy-in is achieved with patients, the majority see sufficient value in the process to continue to participate. The only significant predictor of refusing the tablet on the second occasion after having accepted it on the first was not having completed the tablet fully on the first attempt (OR 5.73, 95% CI 2.31-14.21). Programs considering routine use of tablet questionnaires should ensure that patients have adequate time and support to complete the questionnaire on the first attempt, as our findings suggest that a failed first attempt significantly deters future participation.
Our study has some limitations. Although asthma patients were identified by a validated electronic medical record–based search algorithm  and were further required to have been prescribed an asthma medication in the prior year, some patients without asthma were likely erroneously identified due to imperfect algorithm specificity. Patients denied an asthma diagnosis in 65 visits, but this represented only 3.8% of the total 1715 visits and is unlikely to have influenced our main conclusions. We were unable to include about 20% of visits in our predictive models due to an inability to match those visits to patient characteristics. This resulted from the fact that patient characteristics were collected independently, and at a separate point in the study, whereby certain patients who were offered the questionnaire did not have their clinical characteristics recorded. This process can be assumed to have occurred at random; there was no difference in the tablet questionnaire acceptance level between patients who had full data and the whole cohort. We were also unable to obtain real-time qualitative patient feedback on tablet use during the study, as we designed our study to not interfere with clinic workflow as much as possible. However, we have completed an accompanying qualitative analysis of barriers and enablers to mobile health questionnaire acceptance and uptake in patients with asthma using this tablet questionnaire. It should also be noted that tablet questionnaires can be delivered via various operating systems and tablet sizes. Although we would not expect metrics related to offering the patients the tablet, or willingness of patients to accept the tablet to be affected by such differences, our findings regarding tablet completion rates may not be generalizable to other hardware and operating system types. Finally, as the denominator for our primary analysis was patient visits, there is the possibility that repeated measures from patients being offered the tablet on multiple occasions might have biased our analysis. To account for this, we used generalized estimating equations, which enable a population average model that adjusts for the correlated nature of repeated measurements. We also replicated our main results in a sensitivity analysis which included only a single visit for each patient.
In summary, we found that an electronic tablet-based asthma questionnaire implemented in a primary care waiting room setting had an overall uptake of only 52.0% (891/1715), even with involvement of dedicated research personnel. Patients refused to accept or to complete the tablet for various reasons. In particular, older age was an important predictor of tablet refusal, despite older patients having been included during tool development. Given that older adults, particularly those with chronic diseases that require close monitoring, likely stand to benefit the most from improvements in quality of care that may be realized through such mHealth questionnaires, our findings suggest that future research is required to specifically identify and address age-related barriers to successful uptake of such technology. The practical barriers and statistical predictors identified in our study may be relevant for electronic questionnaire usage across settings, diseases, and patient demographics. These factors, particularly if complemented by qualitative methods to identify barriers and enablers to uptake, can be used by clinics and programs employing such questionnaires, to target improvements to the intervention and its implementation strategy in order to drive usage.
This work was supported by the Keenan Research Summer Student Program, the Canadian Respiratory Research Network and the Canadian Institutes of Health Research. AK is supported by the Thorpe Family Fellowship in Respiratory Medicine at St. Michael’s Hospital. JY is supported by the Lung Association Provincial Grant-in-Aid/National Grant Review competition. SG is supported by the Michael Locke Term Chair in Knowledge Translation and Rare Lung Disease Research. Funders had no role in the design of the study; in the collection, analysis, or interpretation of data; or in manuscript preparation.
Conflicts of Interest
SG owns the intellectual property associated with eAMS and the eAMS questionnaire and would have an ownership interest in any commercial enterprise derived from this system. Other authors report no potential conflicts of interest.
Supplementary tables for regression analyses of tablet refusal at first patient visit only, and refusal of the tablet after having accepted it on first occasion.DOC File , 20 KB
- Freynhagen R, Baron R, Tölle T, Stemmler E, Gockel U, Stevens M, et al. Screening of neuropathic pain components in patients with chronic back pain associated with nerve root compression: a prospective observational pilot study (MIPORT). Curr Med Res Opin 2006 Mar;22(3):529-537. [CrossRef] [Medline]
- O'Leary KJ, Lohman ME, Culver E, Killarney A, Randy SG, Liebovitz DM. The effect of tablet computers with a mobile patient portal application on hospitalized patients' knowledge and activation. J Am Med Inform Assoc 2016 Jan;23(1):159-165. [CrossRef] [Medline]
- Richter JG, Becker A, Koch T, Nixdorf M, Willers R, Monser R, et al. Self-assessments of patients via Tablet PC in routine patient care: comparison with standardised paper questionnaires. Ann Rheum Dis 2008 Dec;67(12):1739-1741. [CrossRef] [Medline]
- Salaffi F, Gasparini S, Ciapetti A, Gutierrez M, Grassi W. Usability of an innovative and interactive electronic system for collection of patient-reported data in axial spondyloarthritis: comparison with the traditional paper-administered format. Rheumatology (Oxford) 2013 Nov;52(11):2062-2070 [FREE Full text] [CrossRef] [Medline]
- Kim H, Park HC, Yoon SM, Kim TH, Kim J, Kang MK, et al. Evaluation of quality of life using a tablet PC-based survey in cancer patients treated with radiotherapy: a multi-institutional prospective randomized crossover comparison of paper and tablet PC-based questionnaires (KROG 12-01). Support Care Cancer 2016 Oct;24(10):4399-4406. [CrossRef] [Medline]
- Suzuki E, Mackenzie L, Sanson-Fisher R, Carey M, D'Este C, Asada H, et al. Acceptability of a Touch Screen Tablet Psychosocial Survey Administered to Radiation Therapy Patients in Japan. Int J Behav Med 2016 Aug;23(4):485-491. [CrossRef] [Medline]
- Abernethy AP, Herndon JE, Wheeler JL, Patwardhan M, Shaw H, Lyerly HK, et al. Improving health care efficiency and quality using tablet personal computers to collect research-quality, patient-reported data. Health Serv Res 2008 Dec;43(6):1975-1991 [FREE Full text] [CrossRef] [Medline]
- Newell SM, Logan HL, Guo Y, Marks JG, Shepperd JA. Evaluating tablet computers as a survey tool in rural communities. J Rural Health 2015;31(1):108-117 [FREE Full text] [CrossRef] [Medline]
- Fritz F, Balhorn S, Riek M, Breil B, Dugas M. Qualitative and quantitative evaluation of EHR-integrated mobile patient questionnaires regarding usability and cost-efficiency. Int J Med Inform 2012 May;81(5):303-313. [CrossRef] [Medline]
- Joule Partners with CognisantMD. CognisantMD. URL: https://www.cognisantmd.com/joule-partners-cognisantmd/ [accessed 2019-07-04]
- Zarghom S, Di Fonzo D, Leung F. Does Socioeconomic Status Affect Patients’ Ease of Use of a Touch-Screen (iPad) Patient Survey? Interact J Med Res 2013 Jan 11;2(1):e1. [CrossRef]
- Kastner M, Lottridge D, Marquez C, Newton D, Straus SE. Usability evaluation of a clinical decision support tool for osteoporosis disease management. Implementation Sci 2010 Dec 10;5(1):96. [CrossRef]
- Gupta S, Price C, Agarwal G, Chan D, Goel S, Boulet L, et al. The Electronic Asthma Management System (eAMS) improves primary care asthma management. Eur Respir J 2019 Apr;53(4):1802241. [CrossRef] [Medline]
- Lam Shin Cheung V, Kastner M, Sale JE, Straus S, Kaplan A, Boulet L, et al. Development process and patient usability preferences for a touch screen tablet–based questionnaire. Health Informatics J 2019 Jan 23:146045821882474. [CrossRef]
- Gupta S, Lam Shin Cheung V, Kastner M, Straus S, Kaplan A, Boulet L, et al. Patient preferences for a touch screen tablet-based asthma questionnaire. J Asthma 2019 Jul;56(7):771-781. [CrossRef] [Medline]
- Palmblad M, Tiplady B. Electronic diaries and questionnaires: designing user interfaces that are easy for all patients to use. Qual Life Res 2004 Sep;13(7):1199-1207. [CrossRef] [Medline]
- Xi N, Wallace R, Agarwal G, Chan D, Gershon A, Gupta S. Identifying patients with asthma in primary care electronic medical record systems Chart analysis-based electronic algorithm validation study. Can Fam Physician 2015 Oct;61(10):e474-e483 [FREE Full text] [Medline]
- Hanley J, Negassa A, Edwardes MD, Forrester J. Statistical analysis of correlated data using generalized estimating equations: an orientation. Am J Epidemiol 2003 Feb 15;157(4):364-375. [CrossRef] [Medline]
- Pullenayegum EM, Lim LS. Longitudinal data subject to irregular observation: A review of methods with a focus on visit processes, assumptions, and study design. Stat Methods Med Res 2016 Jul 11;25(6):2992-3014. [CrossRef]
- Lin H, Scharfstein DO, Rosenheck RA. Analysis of longitudinal data with irregular, outcome-dependent follow-up. J Royal Statistical Soc B 2004 Aug;66(3):791-813. [CrossRef]
- Price C, Agarwal G, Chan D, Goel S, Kaplan AG, Boulet L, et al. Large care gaps in primary care management of asthma: a longitudinal practice audit. BMJ Open 2019 Jan 29;9(1):e022506 [FREE Full text] [CrossRef] [Medline]
- Miller Jr DP, Weaver KE, Case LD, Babcock D, Lawler D, Denizard-Thompson N, et al. Usability of a Novel Mobile Health iPad App by Vulnerable Populations. JMIR Mhealth Uhealth 2017 Apr 11;5(4):e43. [CrossRef]
- Hobden B, Bryant J, Carey M, Sanson-Fisher R, Oldmeadow C. Computer tablet or telephone? A randomised controlled trial exploring two methods of collecting data from drug and alcohol outpatients. Addictive Behaviors 2017 Aug;71:111-117. [CrossRef]
- Brinker TJ, Brieske CM, Esser S, Klode J, Mons U, Batra A, et al. A Face-Aging App for Smoking Cessation in a Waiting Room Setting: Pilot Study in an HIV Outpatient Clinic. J Med Internet Res 2018 Aug 15;20(8):e10976. [CrossRef]
- Moura LM, Schwamm E, Moura Junior V, Seitz MP, Hsu J, Cole AJ, et al. Feasibility of the collection of patient-reported outcomes in an ambulatory neurology clinic. Neurology 2016 Nov 04;87(23):2435-2442. [CrossRef]
- Carey M, Noble N, Mansfield E, Waller A, Henskens F, Sanson-Fisher R. The Role of eHealth in Optimizing Preventive Care in the Primary Care Setting. J Med Internet Res 2015 May;17(5):e126 [FREE Full text] [CrossRef] [Medline]
- Barentsz MW, Wessels H, van Diest PJ, Pijnappel RM, Haaring C, van der Pol CC, et al. Tablet, Web-Based, or Paper Questionnaires for Measuring Anxiety in Patients Suspected of Breast Cancer: Patients' Preferences and Quality of Collected Data. J Med Internet Res 2014 Oct 31;16(10):e239. [CrossRef]
- Hess R, Santucci A, McTigue K, Fischer G, Kapoor W. Patient difficulty using tablet computers to screen in primary care. J Gen Intern Med 2008 Apr;23(4):476-480 [FREE Full text] [CrossRef] [Medline]
- Aiello EJ, Taplin S, Reid R, Hobbs M, Seger D, Kamel H, et al. In a randomized controlled trial, patients preferred electronic data collection of breast cancer risk-factor information in a mammography setting. J Clin Epidemiol 2006 Jan;59(1):77-81. [CrossRef] [Medline]
- Ross J, Stevenson F, Lau R, Murray E. Factors that influence the implementation of e-health: a systematic review of systematic reviews (an update). Implement Sci 2016 Oct 26;11(1):146. [CrossRef] [Medline]
- Sheehan B, Bakken S. Approaches to workflow analysis in healthcare settings. NI 2012 (2012) 2012;2012:371 [FREE Full text] [Medline]
- The Internet and Digital Technology Internet. Statistics Canada. 2019 Jul 04. URL: https://www150.statcan.gc.ca/n1/pub/11-627-m/11-627-m2017032-eng.htm [accessed 2019-07-04]
- Wildenbos GA, Peute L, Jaspers M. Aging barriers influencing mobile health usability for older adults: A literature based framework (MOLD-US). Int J Med Inform 2018 Jun;114:66-75. [CrossRef] [Medline]
- Harrington CN, Ruzic L, Sanford JA. Universally Accessible mHealth Apps for Older Adults: Towards Increasing Adoption and Sustained Engagement. In: Antona M, Stephanidis C, editors. Universal Access in Human–Computer Interaction. Human and Technological Environments. UAHCI 2017. Lecture Notes in Computer Science, vol 10279. Cham: Springer; 2017.
- Bastiaens H, Van Royen P, Pavlic DR, Raposo V, Baker R. Older people's preferences for involvement in their own care: A qualitative study in primary health care in 11 European countries. Patient Education and Counseling 2007 Sep;68(1):33-42. [CrossRef]
- Belcher VN, Fried TR, Agostini JV, Tinetti ME. Views of older adults on patient participation in medication-related decision making. J Gen Intern Med 2006 Apr;21(4):298-303. [CrossRef]
- Levinson W, Kao A, Kuby A, Thisted RA. Not all patients want to participate in decision making. A national study of public preferences. J Gen Intern Med 2005 Jun;20(6):531-535 [FREE Full text] [CrossRef] [Medline]
- Pollom EL, Wang E, Bui TT, Ognibene G, von Eyben R, Divi V, et al. A prospective study of electronic quality of life assessment using tablet devices during and after treatment of head and neck cancers. Oral Oncology 2015 Dec;51(12):1132-1137. [CrossRef]
- McCleary NJ, Wigler D, Berry D, Sato K, Abrams T, Chan J, et al. Feasibility of computer-based self-administered cancer-specific geriatric assessment in older patients with gastrointestinal malignancy. Oncologist 2013;18(1):64-72 [FREE Full text] [CrossRef] [Medline]
- Gibson PG, Simpson JL. The overlap syndrome of asthma and COPD: what are its features and how important is it? Thorax 2009 Aug;64(8):728-735. [CrossRef] [Medline]
- Zeki A, Schivo M, Chan A, Albertson T, Louie S. The Asthma-COPD Overlap Syndrome: A Common Clinical Problem in the Elderly. J Allergy (Cairo) 2011;2011:861926 [FREE Full text] [CrossRef] [Medline]
- Cullati S, Courvoisier DS, Charvet-Bérard AI, Perneger TV. Desire for autonomy in health care decisions: A general population survey. Patient Education and Counseling 2011 Apr;83(1):134-138. [CrossRef]
- Chung GS, Lawrence RE, Curlin FA, Arora V, Meltzer DO. Predictors of hospitalised patients' preferences for physician-directed medical decision-making. J Med Ethics 2012 Feb;38(2):77-82. [CrossRef] [Medline]
- Caress A, Beaver K, Luker K, Campbell M, Woodcock A. Involvement in treatment decisions: what do adults with asthma want and what do they get? Results of a cross sectional survey. Thorax 2005 Mar;60(3):199-205 [FREE Full text] [CrossRef] [Medline]
- Adams RJ, Smith BJ, Ruffin RE. Patient preferences for autonomy in decision making in asthma management. Thorax 2001 Feb;56(2):126-132 [FREE Full text] [Medline]
- Atkins L, Francis J, Islam R, O’Connor D, Patey A, Ivers N, et al. A guide to using the Theoretical Domains Framework of behaviour change to investigate implementation problems. Implementation Sci 2017 Jun 21;12(1):77. [CrossRef]
|eAMS: Electronic Asthma Management System|
|mHealth: mobile health|
|OR: odds ratio|
Edited by G Eysenbach; submitted 21.04.20; peer-reviewed by D Dalan, D Ryan; comments to author 29.06.20; revised version received 22.07.20; accepted 26.07.20; published 18.09.20
©Andrew Kouri, Janet Yamada, Joanna E M Sale, Sharon E Straus, Samir Gupta. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 18.09.2020.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.