This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.
Nurses providing home health care services are dependent on access to patient information and communicating with general practitioners (GPs) to deliver safe and effective health care to patients. Information and communication technology (ICT) systems are viewed as powerful tools for this purpose. In Norway, a standardized electronic messaging (e-messaging) system is currently being established in health care.
The aim of this study was to explore home health care nurses’ assessments of the utility of the e-messaging system for communicating with GPs and identify elements that influence the assessment of e-messaging as a useful communication tool.
The data were collected using a self-developed questionnaire based on variables identified by focus group interviews with home health care nurses (n=425) who used e-messaging and existing research. Data were analyzed using logistic regression analyses.
Over two-thirds (425/632, 67.2%) of the home health care nurses returned the questionnaire. A high proportion (388/399, 97.2%) of the home health care nurses who returned the questionnaire found the e-messaging system to be a useful tool for communication with GPs. The odds of reporting that e-messaging was a useful tool were over five times higher (OR 5.1, CI 2.489-10.631,
This study has shown that technical (ease of use and system functionality), organizational (training), and individual (full-time equivalent percentage) elements had an impact on home health care nurses’ assessments of using e-messaging to communicate with GPs. By identifying these elements, it is easier to determine which interventions are the most important for the development and implementation of ICT systems in home health care services.
Health care services are characterized as fragmented and dispersed [
The current study was conducted in Norway, where home health care service is managed by municipalities; furthermore, GPs are self-employed [
A sociotechnical perspective aims to understand how ICT systems are developed, implemented, and become a part of standard work practices [
Previous studies have shown that several elements impact health care professionals’ assessments of the usability of ICT systems. These elements include the health professionals’ participation in the development of the ICT system [
In Norway, a standardized electronic messaging (e-messaging) system is currently being implemented in primary health services [
The development of the e-messaging system was initiated and managed by health care professionals in close cooperation with vendors [
The aim of this paper was to describe home health care nurses’ assessments of using the e-messaging system to communicate with GPs and identify which elements predict that e-messaging will be considered a useful tool.
The study used a cross-sectional approach with a questionnaire administered between November 2011 and February 2012.
We searched the literature but did not find any questionnaires that measured the use of e-messaging between home health care services and GPs. Therefore, we developed a questionnaire in two phases [
The second phase consisted of the questionnaire development. The questionnaire was reviewed by 6 registered nurses with clinical, ICT, and research skills, and it was pilot tested by 15 registered nurses who worked in home health care services. These steps provided significant and valuable information that was used to revise the questionnaire. The final version of the questionnaire contained 62 items in six sections: (1) Demographic information, eight items, (2) Procedures for and amount of information exchange and communication with GPs, 19 items, (3) Information content, 12 items, (4) Expectations for electronic communication with GPs, 10 items, (5) Electronic communication, three items, and (6) Experiences of the use of electronic communication, 10 items (see
The items selected for this study have been previously documented as important factors for health care providers’ assessments of health information systems [
The questionnaire items were rated on a 5-point Likert scale as follows: (1) strongly disagree, (2) disagree, (3) neither agree or disagree, (4) agree, and (5) strongly agree. The Cronbach alpha exceeded .7, indicating acceptable reliability [
To test the validity of the questionnaire, a selection of the items from the main study was summarized and correlated with the item “useful tool for communication with GPs”, which is the questionnaire’s concluding question. Pearson’s
The questionnaire data were collected from home health care nurses in 12 out of 428 municipalities in Norway. The inclusion criterion for the municipalities required that the e-messaging system had been in use for more than 6 months. We assumed that system usage would be stable after this time frame [
The home health care nurses who were invited to answer the questionnaire received written information explaining that their participation was anonymous and that returning the questionnaire meant that they agreed to participate in the study. The methods for data collection and handling the interviews and the questionnaire were approved by the Norwegian Social Science Data Services, which is the official data protection agency for research at the University of Oslo (reference no. 26230).
We used a descriptive analysis to determine the characteristics of the sample and the nurses’ assessments of using e-messaging. Scores for negatively worded items were reversed. Two logistic regression models were developed to determine the extent to which specific elements influenced the odds that the nurses would report that e-messaging was a useful and easy tool for communication with GPs.
In the first logistic regression model, “the useful tool model”, we chose to include the independent variables of “easy to use” and the home health care nurses’ demographic characteristics of “age”, “gender”, “full-time equivalent percentages”, “years of experience as a registered nurse (RN)”, “years in current position”, and “number of months using e-messaging”. The reason for choosing these variables is based on results from previous research [
In the second logistic regression model, “the ease-of-use model”, the dependent variable of “easy to use” was combined with the independent variables of “received training”, “access to user support”, “hindered by poor functionality”, “hindered by low system performance”, and “hindered by software error”, and the home health care nurses’ characteristics. These variables have been shown to have an impact of the assessment and adoption of ICT systems [
In the pre-analysis of the data, we searched for outliers in continuous variables. The results showed that it was not necessary to transform any of the data. We used ±3.30 standard deviations (SD) to check for outliers, as suggested by Altman [
Homoscedasticity in both logistic regression models was assessed by a one-way analysis of variance of the standardized residuals to establish that their variance was approximately the same for all values of the predictor variables. No homoscedasticity was found. The first model was used to predict if the e-messaging system was a useful tool for communication with GPs. The item “easy to use” did not show statistically significant differences (
We tested for possible multicollinearity among the independent variables in both models using the Pearson correlation to exclude that the independent variables were highly correlated. The highest correlation was .76 for age and years of experience as an RN; therefore, no significant multicollinearity was found [
In the logistic regression analysis, we dichotomized the variables because of their skewed distribution. The variable “useful tool” was dichotomized as strongly disagree, disagree, neutral, and agree (0=1-4) and strongly agree (1=5). The rest of the ordinal variables were dichotomized as strongly disagree, disagree, neutral (0=1-3), agree, and strongly agree (1=4-5).
The
A total of 425 (67.2%) of the 632 home health care nurses who received the questionnaire completed it. The demographic information is presented in
Demographic information of home health care nurses (N=425).
Demographic characteristics | Mean | n (%) | |
|
|||
|
Female |
|
383 (90.4) |
|
Male |
|
41 (9.6) |
Age, mean (SD) | 39.6 (10.1) | 424 (99.8) | |
Years of experience, mean (SD) | 11.5 (9.1) | 421 (99.1) | |
Full-time equivalent percentage, mean (SD) | 90.9 (14.2) | 425 (100.0) | |
Years in current position, mean (SD) | 5.8 (5.6) | 422 (99.3) | |
Number of months using e-messaging, mean (SD) | 10.21 (7.5) | 382 (89.9) |
The vast majority of the responding home health care nurses (388/399, 97.2%) agreed or strongly agreed that e-messaging was a useful tool for communication with GPs.
For the “easy to use e-messaging” item, the majority of home health care nurses agreed or strongly agreed (357/398, 89.7%) that e-messaging was easy to use. A high proportion of the home health care nurses agreed or strongly agreed that they had received training (333/399, 83.5%), while the remaining nurses were neutral or disagreed. The home health care nurses agreed to a lower degree that they had access to user support (256/389, 65.8%).
A relatively high proportion of the home health care nurses were neutral or agreed that they were hindered when using e-messaging because of poor functionality (123/395, 31.1%), low system performance (123/396, 31.1%), or software errors (129/395, 32.7%).
Assessments of using e-messaging (N=425).
Item | Strongly disagree, |
Disagree, |
Neutral, |
Agree, |
Strongly agree, |
Missing, |
Received training | 1.7 (7) | 6.0 (24) | 8.8 (35) | 50.9 (203) | 32.6 (130) | 26 |
Access to user support | 1.2 (5) | 3.6 (14) | 29.3(114) | 52.9 (206) | 12.9 (50) | 36 |
Hindered by poor functionality | 18.7 (74) | 50.1 (198) | 23.3 (92) | 6.6 (26) | 1.2 (5) | 30 |
Hindered by low system performance | 19.4 (77) | 49.5 (196) | 23.5 (93) | 6.3 (25) | 1.2 (5) | 29 |
Hindered by software error | 21.5 (85) | 45.8 (181) | 17.2 (68) | 13.1 (52) | 2.3 (9) | 30 |
Easy to use | 0.2 (1) | 1.5 (6) | 8.5 (34) | 52.8 (210) | 36.9 (147) | 27 |
Useful tool | - | 0.7 (3) | 2 (8) | 31.8 (127) | 65.4 (261) | 26 |
The logistic regression model showed that the “easy to use” item gave a statistically significant contribution to the model (
The odds of reporting that e-messaging was a useful tool were over five times higher if the nurses agreed or strongly agreed that e-messaging was easy to use. The “useful tool model” was controlled for the demographic variables of gender, age, years of experience as an RN, full-time equivalent percentage, and years in current position.
Logistic regression model examining predictors affecting the assessment of e-messaging as a useful tool (N=391).
|
B (SE) |
|
Odds ratio (95% CI) |
Gender | .041 (0.402) | .920 | 1.041 (0.473-2.291) |
Age | .009 (0.018) | .621 | 1.009 (0.974-1.045) |
Years of experience as an RN | -.014 (0.021) | .505 | 0.986 (0.948-1.027) |
Full-time equivalent percentage | .006 (0.008) | .505 | 1.006 (0.989-1.022) |
Years in current position | .001 (0.024) | .973 | 1.1001 (0.955-1.049) |
Easy to use | 1.638 (0.370) | .000 | 5.144 (2.489-10.631) |
Constant | -1.555 | .134 | 0.211 |
The “ease-of-use model” determined which elements influenced the assessment of e-messaging as easy to use. The results from the logistic regression model (
The odds that a home health care nurse would experience e-messaging as easy to use increased as the full-time equivalent percentage of the nurses increased. If the home health care nurses agreed that they were not hindered by poor e-messaging functionality, the odds of agreeing that e-messaging was easy to use were over 6.9 times higher than if they did not agree. The home health care nurses who agreed that they had received training had odds over 6.6 times higher of agreeing that e-messaging was easy to use. The home health care nurses’ gender, number of months using e-messaging, being hindered by low system performance or software errors, or having access to user support were not statistically significant predictors for the assessment of e-messaging as easy to use.
Logistic regression model examining predictors affecting assessments of e-messaging as easy to use (N=364).
|
B (SE) |
|
Odds ratio (95% CI) |
Gender | -.561 (0.796) | .480 | 0.570 (0.120-2.713) |
Age | -.049 (0.038) | .193 | 0.952 (0.884-1.025) |
Years of experience | -.015 (0.039) | .701 | 0.985 (0.913-1.063) |
Full-time equivalent percentage | .031 (0.016) | .045 | 1.032 (1.001-1.064) |
Years in current position | -.030 (0.043) | .488 | 0.971 (0.893-1.056) |
Number of months using e-messaging | -.030 (0.030) | .318 | 0.971 (0.915-1.029) |
Hindered by low system performance | -.633 (0.731) | .387 | 0.531 (0.127-2.226) |
Hindered by software error | -.094 (0.548) | .863 | 0.910 (0.311-2.664) |
Hindered by poor functionality | 1.933 (0.712) | .007 | 6.914 (1.713-27.899) |
Received training | 1.890 (0.494) | <.001 | 6.622 (2.515-17.437) |
Having access to user support | .170 (0.479) | .722 | 1.186 (.464-3.031) |
Constant | 1.053 (2.001) | .599 | 2.867 |
The results from this study showed that individual, organizational, and technological elements are interrelated and affect home health care nurses’ assessments of using e-messaging to communicate with GPs. This is in line with a sociotechnical perspective [
Several home health care nurses cooperated with the vendors in the development and implementation of the e-messaging system. The realization of an ICT system relies on the participation of the people who will ultimately use it [
The only statistically significant predictor of e-messaging as a useful tool for communication with GPs was that nurses assessed e-messaging as easy to use. This is supported by a study of health care professionals’ adoption and use of a clinical information system that found that the ease of use of the system was required for it to be considered as a beneficial tool for their clinical practice [
Ease of use is one of the most frequent elements reported among studies of facilitators and barriers of the adoption of ICT systems in health care [
Tools/technical concerns, such as functionality and system design, are important elements that can act as barriers and facilitators for the implementation and use of ICT systems in health care [
Assessments of ICT systems are affected by organizational aspects, such as offering user support and training [
By applying a sociotechnical perspective, we were able to detect and predict which elements were important for the development and implementation of e-messaging. All of these elements are interrelated and shape the sociotechnical system; furthermore, these elements should be considered when developing and implementing new technology in home health care services [
This study has several limitations. The best methodology to develop a questionnaire is to include a pilot study with a smaller sample using all possible variables and subsequently identifying the relevant ones. However, we did not complete this type of study because of our limited time frame. As an alternative, we based the questionnaire on explanatory variables identified by the focus group interviews and existing research.
The questionnaire was not assessed across all dimensions of validity and reliability. However, the aim of the study did not include a complete psychometric testing of the questionnaire.
The mean full-time equivalent percentage may have been falsely high because one of the inclusion criteria for participation in the study was that the nurses had to be engaged in a 50% or greater full-time equivalent position. This was necessary to ensure that the nurses had enough experience in using e-messaging. According to the Norwegian Nurses Organization in 2011, 56% of the nurses in primary care in Norway did not work in a full-time positon, and 15% of those had less than a 50% full-time equivalent position [
The strength of this study is that we questioned home health care nurses in all of the municipalities that had used the e-messaging system for at least 6 months at the time of the study. The response rate of the study was 67.2%, which can be considered fairly good for a questionnaire study [
By identifying elements that affect home health care nurses’ assessments of e-messaging for communication with GPs, we are better able to determine which interventions are most important for the development and implementation of e-messaging. This study demonstrated that home health care nurses assessed e-messaging as a useful tool for communication with GPs. It also shows that ICT systems must be easy to use to be assessed as a useful tool and that a higher full-time equivalent percentage, having received training, and not being hindered by poor functionality are important predictors for the assessment of the e-messaging system as easy to use. Our results imply that users should be actively involved in the development and implementation of ICT systems. Future studies should use a sociotechnical approach to consideration the complete range of elements that can affect working practices and the outcomes for the patients and organizations involved. These insights may help to increase the understanding of effective strategies for developing and implementing ICT systems in home health care services.
The questionnaire.
general practitioner
information and communication technology
registered nurse
The project was funded by Grant 196365/V50 awarded by the Research Council of Norway. We would like to thank the municipalities that helped us organize the data collection and the home health care nurses who participated in our study. We would also like to thank Dr Line Melby for participating in the collection of data used to develop the questionnaire; Dr Maren Kristine Raknes Sogstad; the RNs, Edith Gjevjon, Torunn Wibe, Sissel Skarsgaard, Irene Henriksen Aune, and Astrid Simonsen; the Centre for the Development of Home Healthcare Services in Oslo for their contribution to the development of the questionnaire; and research assistant, Linda Aasvangen, for preparing the SPSS file.
ML designed the work, conducted the analysis, and wrote the first draft of the paper. DH and AG contributed to the design of the study and the statistical analysis. RH contributed to the design of the study, the data collection, and the statistical analysis. All of the authors revised the manuscript and read and approved the final version.
None declared.