Background: Support interventions for caregivers can reduce their stress, possibly improving the quality of patients’ care while reducing care costs. Technological solutions have been designed to cover their needs, but there are some challenges in making them truly functional for end users. Co-design approaches present important opportunities for engaging diverse populations to help ensure that technological solutions are inclusive and accessible.
Objective: This study aimed to identify co-created technological solutions, as well as the process followed for their co-creation, in the field of health for caregivers.
Methods: The literature review was conducted in the Medline, Web of Science, Scopus, Science Direct, Scielo, and IEEE Xplore databases. The inclusion criteria were studies written in English or Spanish and with a publication date until May 2021. The content had to specify that the caregivers actively participated in the co-creation process, which covered until the development phase of the technological solution (prototype). The level of evidence and the methodological quality were analyzed when possible, using the Scottish Intercollegiate Guidelines Network criteria and the Mixed Methods Appraisal Tool, version 2018, respectively.
Results: In total, 410 papers were identified, and 11 met the eligibility criteria. The most predominant articles were mixed methods studies and qualitative studies. The technology used in the analyzed articles were mobile or web applications (9 studies) and specific devices such as sensors, cameras, or alarm systems (2 studies) to support the health and social aspects of caregivers and improve their education in care. The most common patient profile was older people (7 studies); 6 studies used co-creation in the requirements phase, 6 studies detailed the design phase. In 9 studies, the prototype was iteratively refined in the development phase, and the validation phase was performed in 5 of the reviewed studies.
Conclusions: This systematic review suggests that existing co-created technological solutions in the field of health for caregivers are mostly mobile or web applications to support caregivers’ social health and well-being and improve their health knowledge when delivering care to patients, especially older people. As for the co-creation process, caregivers are particularly involved during development and in the design. The scarce literature found indicates that further research with higher methodological quality is needed.
The steady increase in the number of people with acute and chronic diseases and increasing life expectancy place new demands on the health systems . This increase is linked to a higher demand for care, representing a significant overload on public health resources [ ]. At the same time, the demand for informal care is likely to increase over the coming decades [ ]. In fact, caregivers have a ubiquitous and very substantial presence throughout the world, including the following countries: 43.5 million caregivers in the United States, 8.1 million caregivers in Canada, 6.5 million caregivers in the United Kingdom, and more than 8 million caregivers in France [ ]. It is estimated that 10%-25% of Europe’s population regularly provides informal care, even though one’s identification as a caregiver and the definitions of caregivers vary in different contexts [ ].
Informal caregivers usually are family members, neighbors, close acquaintances, or other significant individuals who provide unpaid daily assistance to a family member or dependent older adult who cannot care for themselves . They play a strategic role in the daily activities of their dependent care recipients; however, informal care negatively affects the caregiver’s work productivity and their health, leading to a gradual worsening of caregivers’ quality of life. A survey conducted in the United States demonstrated that 32% have high caregiver burden and 19% have medium caregiver burden based on a measurement of time spent providing care and the care recipient’s degree of dependency [ ]. Considering this, it is clear that many caregivers need support services to improve their health and quality of life [ ].
Lack of support is a significant problem , and caregivers’ demand for education related to functional care is high [ ]. A current systematic review concluded that support interventions for caregivers can reduce their stress, possibly improving the quality of patients’ care while reducing care costs [ ]. Consequently, it is essential to introduce user-friendly and time-effective educational and supportive interventions.
Technological health solutions, especially in the form of assistive technologies, create significant opportunities to optimize both health and social care delivery. In this paper, we consider technological health solutions for caregivers as those that can transform and complement current care such as web or mobile applications, artificial intelligence, or virtual or augmented reality that can be used for medication management, community support, cognitive stimulation, or emotional support; nevertheless, we do not consider static repositories of information (such as static web pages or blogs) as a technological solution. Many studies support the idea that technological solutions can support conventional health care provision methods, thereby reducing demand for local services . Today, technological solutions are popular [ ], and they have the potential to provide personalized health care and disease management strategies and services to patients and their family members, as well as offer a flexible mode of communication between health workers and their consumers [ , ].
Lately, there has been a shift in the development of new products, first from a supplier-centered design (ie, service providers design a product) to user-centered design (ie, based on the user’s needs) and now to co-design, also called co-creation. In co-creation, designers, service providers or suppliers, and consumers work together to identify the problems and design solutions . To achieve better outcomes, all parties have an active role [ ], contributing and working together by using their knowledge and resources [ ]. Co-creation in health interventions involves the equal partnership of the people who engage in a health intervention, such as service suppliers (ie, health staff), end users (ie, patients, families, and caregivers), and intervention developers (eg, information technology experts) [ ]. Recently, this method has been widely used to develop health interventions [ ], more in the preparation of recommendations [ ], evaluation frameworks [ ], or the creation of new knowledge [ ] and not so much in the technological field.
The concerns of caregivers have been reported from many points of view, from sociological issues, national regulation, and stakeholders’ views, to caring activities to avoid the negative effect of losing control due to informal care and the so-called caregiver burden [, ]. Technological solutions have been designed consistent with the current context, which cover some of these needs and present some challenges to make them truly functional for end users and caregivers. Co-creation might be a good option [ ], but there is still a gap in identifying the type of technological solutions developed and their characteristics and in measuring their co-creation development.
This systematic review aimed to identify co-created technological solutions, as well as the process followed for their co-creation, in the field of health for caregivers.
This systematic review was carried out following the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analysis) guideline . The research questions from which this systematic review started were the following: What co-created technological solutions exist for caregivers in the field of health? What co-creation process is followed for the design and development of this technology?
The literature review was conducted in the Medline, Web of Science, Scopus, Science Direct, Scielo, and IEEE Xplore databases. The main search terms used to carry out this work were co-creation AND technology AND health AND caregiver. The full search string is available in.
The inclusion criteria were studies written in English or Spanish and with publication dates until May 2021. No starting year was established for the search in order to cover all existing evidence published by journals over all time. The included studies had to specify that the caregivers actively participated in the co-creation process, which covered until the development phase of the technological solution (prototype). The publications that were exclusively aimed at the people being cared for were excluded; only publications aimed at the population of caregivers were included. All publications that did not appear in peer-reviewed journals were excluded, except those extracted from the IEEE Xplore database, due to the relevance and impact of contributions to conferences in the field of technological engineering.
Study Selection and Data Extraction
Initially, the search was carried out in the different databases by 2 researchers from the group (CBA and MRM). Subsequently, an independent blinded review process in which the different researchers of the team participated was carried out. The screening phase began with an independent blinded review of the previously identified studies by 2 other investigators. First, 2 researchers (SGC and EMM) evaluated the titles and the abstracts of the studies to assess their eligibility. Second, the remaining article’s full texts were assessed by the other 2 researchers (ARA and JAMB). In the 3 phases, the disagreements in selecting the studies between the 2 researchers were resolved in consensus by consulting the full text again. When the disagreements persisted, a third reviewer of the team assessed the eligibility of the research. Finally, in those studies in which the design could be evaluated, the level of evidence and methodological quality were independently analyzed using the Scottish Intercollegiate Guidelines Network (SIGN) criteria  and the Mixed Methods Appraisal Tool [ ], version 2018, respectively. In both cases, disagreements were also resolved by consensus.
Independently and in pairs, using a template, the researchers extracted the following descriptive information from the articles included in the systematic review: authors and location, design, level of evidence, methodological quality, patient profile, type of technology designed, and objective of the designed technology. In addition, detailed information was extracted related to the co-creation process (name of the phase, description of the phase, agents involved, and result variables), divided into 4 phases: (1) requirements, (2) design, (3) development, and (4) validation. These phases were extracted from the studies included in the systematic review as common points between the different frameworks used in them . The requirements phase consisted of identifying the problem and setting the objectives of the process. The design phase consisted of the creation of a solution. The development phase consisted of the implementation of a functional prototype. Finally, the validation phase consisted of evaluating the co-creation process and the effectiveness of the proposed solution.
Based on these categories to extract the information, 2 tables of results were created to subsequently analyze the content with the aim of answering the research questions initially raised and, therefore, the objectives of this systematic review.
The search produced 410 papers. We removed duplicates, leaving 279 papers. Titles and abstracts were screened to ensure alignment with the inclusion criteria, and 218 were eliminated from the study, thus leaving 61 eligible papers for further scrutiny. We read the entire text of 61 papers to assess eligibility in line with the inclusion and exclusion criteria. We eliminated 50 of the publications mainly because the studies were not consistent with our inclusion criteria; the reasons included not developing the technology (reason 1), not including caregivers as their target group (reason 2), not having a participatory design (reason 3), being part of another study already included (reason 4), or not having enough information to determine compliance with the inclusion criteria (reason 5).indicates the process of searching and identifying the papers through the PRISMA flow diagram.
Authors and location, design, level of evidence, patient profile, type of technology designed, and objective of the designed technology are detailed in.
|Author(s), year (location)||Design||Level of evidencea||Patient profile (age)||Type of technology designed||Purpose of the designed technology|
|Backman et al, 2020 (Canada) ||Mixed methods study||—b||Older adults leaving hospital (67-97 years)||Mobile/web application||To manage the personalized needs of geriatric rehabilitation patients during their transition from the hospital to home|
|Egan et al, 2021 (United Kingdom) ||Mixed methods study||—||Caregivers (NRc)||Mobile/web application||To educate and support caregivers in the undertaking of regular physical activity at home during and beyond COVID-19 restrictions|
|de la Harpe and van Zyl, 2011 (South Africa) ||Mixed methods study||—||Population from under-resourced communities (NR)||Mobile/web application||To participate as simultaneous producers and consumers of information, relating to their own experiences, and to contribute to a joint repository of information and educational material in their own "idiom"|
|Guerrero et al, 2019 (Sweden) ||Case study||3||Older adults (57-67 years)||Specific devices||To assist with medication management|
|Harding et al, 2021 (India, Uganda, Zimbabwe) ||Mixed methods study||2||Palliative patients (NR)||Mobile/web application||To improve access to palliative care|
|Latulippe et al, 2020 (Canada) ||Qualitative study||—||Functionally dependent older persons (NR)||Mobile/web application||To facilitate the process of help-seeking for caregivers of functionally dependent older persons|
|Lemetyinen et al, 2018 (United Kingdom) ||Randomized controlled trial||1+||African Caribbean persons diagnosed with schizophrenia or other nonaffective psychosis (NR)||Mobile/web application||To improve knowledge about and attitudes toward schizophrenia in African Caribbean families|
|Meiland et al, 2014 (Netherlands and Germany) ||Qualitative study||—||People with mild cognitive impairment and dementia (NR)||Specific devices||To support community-dwelling people with mild cognitive impairment and dementia in daily functioning, monitor (deviations from) patterns in daily behavior, and automatically detect emergency situations|
|O\'Connor, 2020 (United Kingdom) ||Qualitative study||—||People with dementia (NR)||Mobile/web application||To stimulate memory and communication by sharing memories together|
|Rathnayake et al, 2020 (Australia) ||Mixed methods study||—||People with dementia (NR)||Mobile/web application||To address functional disability care needs|
|Sin et al, 2019 (United Kingdom) ||Qualitative study||—||People with psychosis (NR)||Mobile/web application||To provide carers with psychoeducation and emotional support using health care professional contribution and peer support|
aScottish Intercollegiate Guidelines Network (SIGN) criteria: 1+ well-conducted meta-analyses, systematic reviews, or randomized controlled trials (RCTs) with a low risk of bias; 1- meta-analyses, systematic reviews, or RCTs with a high risk of bias; 2- case control or cohort studies with a high risk of confounding or bias and a significant risk that the relationship is not causal; 3 nonanalytic studies (eg, case reports, case series).
bNot able to be assessed.
cNR: not reported.
Design, Level of Evidence, and Methodological Quality
Regarding the designs of the included articles, 5 mixed methods studies [- , , ] and 4 qualitative studies [ , , , ] were the most predominant, followed by 1 randomized controlled trial (RCT) [ ] and 1 case study [ ]. The level of evidence in only 3 of the 11 articles could be assessed using the SIGN criteria [ ], with the following results: 1 level 1+ for the RCT study [ ], 1 level 2- for a mixed methods study [ ], and 1 level 3 for a case study [ ]. The methodological quality assessment indicates that only 3 studies fully met the quality criteria. The results are available in .
Type of Technology
The technology used in the analyzed articles can be classified into mobile or web applications [- , - , - ] and specific devices such as sensors, cameras, or alarm systems [ , ]. Mobile applications are applications that live and run on the device itself. Web applications are accessible through websites using a web browser and have functionality and interactive elements.
Purpose of the Technological Solutions
The aims can be classified into education and information in health [, , , , ], social [ , , ], and well-being [ , , ] aspects; assistive systems [ , , , ]; and cognitive rehabilitation [ ].
The main strategy for education and information is through content and resources (text, videos, storyboards) from the company’s or external sources’ solutions. Some of them use e-learning environments to introduce interactive resources such as quizzes [, ]. They also use social interactions with experts or caregivers through forums, blogs, or peer support. The assistive systems aim to monitor and automatically detect emergencies or to improve patients’ autonomy. These systems use mobile or web applications in combination with specific devices. The data reported by the systems are shared with patients, family or community caregivers, or the patient’s care provider. Finally, 1 tool [ ] provides a digital profile of the person with dementia to stimulate the person’s memory and improve communication with caregivers.
Most patient profiles in the articles were that of an elderly population, either being directly targeted [, , ] or including patients with health conditions that mostly affect older adults like dementia [ - ] or require palliative care [ ]. The rest of the articles included patients with mental health conditions such as psychosis or schizophrenia [ , ] or other circumstances such as being from under-resourced communities [ ].
Co-creation — Agents Involved
The co-creation process involved a wide variety of agents. Caregivers, as a target group, were involved in all the studies (see). Other individuals involved in most studies were health professionals, who were included in 9 studies [ , , - , - ]; the research team (health, social, education, and technology researchers), who appeared in 8 studies [ - , , , , ]; patients, who participated in 6 studies [ , , - , ]; and technology professionals, who participated in 6 studies [ - , - ]. Other agents who were involved in the co-creation process were community members in 3 studies [ , , ] and students [ ], social professionals [ ], a volunteer [ ], a museum manager [ ], and a translator [ ] in 1 study each. In addition, some heterogeneous groups also participated, such as an advisory committee [ , ] made up of community workers, health and social professionals, and caregivers; a young, mixed group [ ] of relatives and patients; and a focus group [ ] of relatives, caregivers, and patients. On average, between 4 and 5 different types of actors were consulted for each study.
|Author(s), year (location)||Phase 1: requirements||Phase 2: design||Phase 3: development||Phase 4: validation|
|Backman et al, 2020 (Canada) ||—a||Xb||X||X|
|Egan et al, 2021(United Kingdom) ||—||X||X||X|
|de la Harpe and van Zyl, 2011 (South Africa) ||—||X||—||—|
|Guerrero et al, 2019 (Sweden) ||X||—||—||X|
|Harding et al, 2021 (India, Uganda, Zimbabwe) ||—||X||X||X|
|Latulippe et al, 2020 (Canada) ||X||X||X||X|
|Lemetyinen et al, 2018 (United Kingdom) ||X||—||—||—|
|Meiland et al, 2014 (Netherlands and Germany) ||X||—||X||—|
|O\'Connor, 2020 (United Kingdom) ||—||—||X||X|
|Rathnayake et al, 2020 (Australia) ||X||—||X||—|
|Sin et al, 2019 (United Kingdom) ||—||—||—||X|
With co-creation, users have an active role from the beginning of the creation process . There are different frameworks for co-creation, but they have common phases [ ]: requirements, design, development, and validation. The analysis of the process of creating the technological solution carried out in this systematic review is based and organized on these phases ( ).
Co-creation — Phase 1: Requirements
The first phase of the co-creation process involves the requirements. Of the articles, 5 did not use co-creation in this phase [, , , , ]. The rest of the papers tackled this phase with a needs assessment [ , , - , ]; 1 of these studies added a data gathering phase after the needs assessment [ ]. Of the 6 articles that used a needs assessment, 5 specified the concrete methods used for the co-creation, while 1 only indicated that it was an ethnographic study [ ]. Of the articles, 3 used interviews [ , , ]. However, only 1 of these articles proceeded exclusively with interviews [ ], while 1 study combined interviews with workshops, expert meetings, and consultation with partners [ ] and the other study combined interviews with an online survey [ ]. In addition, 1 more article adopted workshops [ ], and another study used focus groups [ ]. Finally, the article that added a data gathering phase introduced questionnaires for this purpose [ ].
When we analyzed the participation of the agents by phase, 5 of the 11 selected articles incorporated caregivers [, - , ], followed by health professionals [ , , ], patients [ , , ], and community members [ , , ]. The research team was consulted in 2 of the 11 studies [ , ]. Only 1 of the articles included technology professionals [ ] and students [ ], and 5 articles did not mention involving any agent at this stage [ , , , , ].
Co-creation — Phase 2: Design
The design phase of the co-creation process, the second of the 4 phases, was described in 6 of the 11 studies included in this systematic review. All stood out for the application of technology design methods, such as a modern agile and iterative co-design , the MoSCoW methodology [ ], or the “keep, lose, change” approach [ ], and specific design tools, such as service design methods [ ], focusing on users’ needs during development. For this, different teamwork techniques were carried out, such as interviews [ , ], analysis [ ], brainstorming [ ], workshops [ , ], and debriefing [ ], to reach agreement by consensus on aspects related to the technological solution such as the content, functionality, or appearance that would allow covering the previously identified needs.
When we analyzed the agents involved in this phase, the studies most frequently involved caregivers (6 studies) [- , , , ] and the research team [ - , , ] (5 studies), followed by health professionals in 4 studies [ , , , ]; patients in 2 studies [ , ]; and community members [ ], technology professionals [ ], and social professionals [ ] in only 1 study each. In addition, 4 articles did not mention involving any agent at this stage [ , - ].
Co-creation — Phase 3: Development
The development phase consists of the creation of a functional prototype. Unlike the proof of concept in the design phase, creation of the functional prototype is not only to answer technical and design questions. It must be functional and usable to test the fundamental hypotheses of the proposal. It consists of gradual and iterative development, alternating phases of creation, and testing for subsequent refinements. End users and co-creation participants generally participate in the testing phases, and the prototype is refined based on their feedback. In some cases, they may also participate in content development, design, and feature selection phases. They also usually participate by conducting a usability test, and the final prototype is refined based on their feedback.
In 1 study , the prototype was developed according to the results from the needs assessment and design phases, and the co-creation participants did not intervene; in another study [ ], there was no description of the development. In 9 of the 11 studies, the prototype was iteratively refined, and co-creation participants were involved in all iterations [ , , - , ,- ]. In 4 of these studies, caregivers’ and patients’ participation was limited to testing and feedback sessions, whereas in 4 of the studies, they were also involved in workshops on creating and selecting content and functionality [ , , , ]. In 1 study, a nurse was involved in gradual and iterative development [ ]. Usability testing to refine the final prototype was conducted in 4 studies [ , , , ]. The number of iterations of the prototype development in the studies varied from 2 to 4 and was not detailed in 2 studies.
Analyzing agents involved in the 11 articles, caregivers were the most predominantly consulted, in 8 articles [, , , , - ], followed by the research team in 7 articles [ - , , , ]; health professionals in 7 articles [ , , , , - ]; technology professionals in 6 articles [ - , - ]; patients in 4 articles [ , , , ]; and community members [ ], social professionals [ ], students [ ], volunteers [ ], and translators [ ] in at least 1 article each. In addition, 1 article did not mention involving anyone at this stage [ ].
Co-creation — Phase 4: Validation
The validation phase was performed in 5 of the 11 studies included in this systematic review [, , , , ]. Of the remaining studies, there was no validation phase in 2 studies [ , ]. In 2 studies [ , ], the final prototype was tested with a controlled trial, but the trial was not described. In 1 study, a basic evaluation of the prototype was done by the co-creation group, and a real-world study was planned but not performed [ ]. In the other study [ ], the protocol included a usability and user satisfaction study through a think-aloud method [ ] and the IBM usability questionnaire [ , ] with 5 caregivers and 5 community workers, but this study was not completed.
A qualitative test of the prototype with a small sample of participants (2 or 3 patients and 2 caregivers) was included in 2 studies [, ]. In 1 study [ ], the experience and efficacy of the tool were analyzed, and in the other study [ ], the authors used the Assessment of Autonomy in Internet-Mediated Activity tool [ ] for activity performance analysis and breakdown detection, the Raw NASA Task Load Index questionnaire [ ] to explore the task load of using the tool, and the System Usability Scale questionnaire [ ] to analyze the user experience.
The other 3 studies performed a pilot test with a significant sample size [, , ]. In 1 study [ ], the pilot was done in parallel in 3 sites with a sample size of 25 family caregivers and 25 community caregivers at each site, and the study explored application usage activity, acceptability, views on the app processes, and future refinements through qualitative data collection obtained in semistructured individual interviews. The last 2 studies performed a pilot to test feasibility and acceptability. One of the studies [ ] was a pilot RCT with 20 participants (relatives and caregivers) in each arm examining the feasibility of recruiting and retaining participants; collecting relevant outcome data; and evaluating the intervention's acceptability, accessibility, and utility. Standardized questionnaires (Culturally adapted Knowledge About Psychosis questionnaire [ ], Attitudes to Severe Mental Illness scale [ ], SF-12 quality of life questionnaire [ ]) were used for quantitative outcome measures, and semistructured individual interviews were used to collect qualitative acceptability data and to explore accessibility, usefulness, impact on attitudes and beliefs about schizophrenia, and feasibility aspects. The second study [ ] was a single-arm feasibility pilot test with a sample of 30 patients, 18 caregivers, and 20 health professionals to test feasibility and acceptability and to refine the method for a larger study. Application usage was measured, baseline and follow-up surveys were used, semistructured interviews were conducted, and the results were summarized using a standardized index (Technology Readiness Index: A Multiple-Item Scale to Measure Readiness to Embrace New Technologies [ ]) and content analysis.
Regarding agents involved in this phase, caregivers were involved, with the difference compared with the other agents, in 6 of the 11 articles [, , - , ], followed by health professionals [ , , , ] in 4 of the 11 articles; patients [ , , ] in 3 of the 11 articles; and technology professionals [ ], the research team [ ], community members [ ], social professionals [ ], and museum managers [ ] in at least 1 article each. In addition, 2 articles did not mention involving agents in this phase [ , ].
The objective of this systematic review was to identify co-created technological solutions, as well as the process or methodology followed for their co-creation, in the field of health for caregivers. The interventions in the included studies highlighted that, despite the rapid increase in interest in technological support for caregivers, very few studies included caregivers in the co-design process. In other reviews for specific populations such as older adults , none of the identified studies involved co-design or patient-oriented research approaches. The lack of standardized methodologies and the diversity of frameworks used in the co-creation process of technology solutions make it difficult to analyze and compare. Most of the technological solutions proposed in these studies were mobile or web applications, and all studies included caregivers, at least during design or development, and reached the prototyping phase of the technological solution.
Regarding the type of technological solution designed and developed in the studies included in the review, in 9 of 11 articles, the proposal was a mobile or web application [- , - ]. The proposals in the remaining 2 studies were based on high-tech products that involve more complex technologies such as augmented reality, robotics, or innovative assistive technologies (eg, sensor-based surveillance and monitoring systems, mobile technology such as wearable fall detectors) [ , ]. This could be due to the accessibility of mobile devices, mobile applications, and web applications and their ability to provide interventions instantly to promote health [ ]. This allows and facilitates, in turn, intervening and interacting with users in their daily life and context, known as an ecological momentary intervention (EMI) [ - ]. EMI consists of momentary treatments provided via mobile technologies while people are engaged in their typical daily life routines [ ]. EMI can be a useful add-on to traditional treatment, thanks to the 24-hour availability, low cost, and possibility of continuing follow-up in a nonpresential manner [ ].
In turn, mobile or web applications have proven to be one of the most feasible technological solutions in digital health interventions with different population profiles [- ]. In addition, previous studies suggest that mobile applications have the potential to have a greater positive impact on caregivers by providing support, communication, and facilitation of care, reducing the burden and positively impacting caregiver health outcomes [ ].
On the other hand, the more significant presence of the design and development of mobile or web applications in this systematic review, compared with other types of technological solutions such as virtual reality, augmented reality, or robotics, could be due to the presence of this technology in people’s daily lives and, therefore, their knowledge, experience, and familiarity with it to actively participate in the process of co-creation .
Previous literature provides evidence that technology offers a cost-effective and practical method for delivering interventions to caregivers . Nevertheless, the relevance of barriers to high-tech products suggests that external constraints impair consumers’ participation in complex or technologically advanced products [ ]. Caregivers, possibly due to older age or little experience with these more complex technologies, do not realize how they can benefit personally from this kind of technology [ ]. High-tech products require effort to obtain new specialized knowledge and skills. For this reason, we think that, in this case, it is even more important to involve caregivers from the beginning of the co-creation process in order to fulfill their needs and preferences. For example, robots have the potential to help with the caregiving and domestic needs of the growing aging population [ ], and it is important to ensure the participation of caregivers in the creation of the solutions.
As some studies identified, almost one-half of the caregivers providing substantial help with health care assisted an older adult with dementia . In this review, we identified that the needs of caregivers being tackled by technological solutions are very diverse. Even though there is research on the most common needs for caregivers of people with Alzheimer disease and dementia, as an example [ ], their needs were related to personal health and receiving help from others, as well as information gaps and their education or learning needs. Although we can see how the articles identified in this study aimed to cover the needs related to information gaps and the education or learning needs [ - , , , ], they also had other objectives that may not be a priority need for caregivers. Caregivers still have needs related to the action of caring, but none of the studies focused on one of the most important needs, such as caring for their health.
This review also analyzed the co-creation process of solutions in which at least a functional prototype had been developed. Most of the existing published literature on this topic, including the recent , did not go beyond the design phase, which makes it difficult to understand the implementation and evaluation of co-creation processes and the effectiveness of the proposed solution.
The lack of standardized methodologies and diversity of frameworks used in the co-creation process of technology solutions make it difficult to analyze and classify the studies. The same occurred for other related, excluded studies, which used multimethod designs . Also, the lack of information makes it impossible to replicate some of this work. In some studies, the project was developed following the waterfall model, which consists of linear-sequential phases that each depends on the completion of the previous phase. Currently, this model has also been used in other health care solutions focused on patients rather than caregivers [ , ]. However, in most of the studies, the execution of these phases is not sequential, and the development phase involves refining the previous phases. Moreover, 1 study followed an agile scrum co-design methodology [ ] based on the prototype model; in this methodology, development occurs through short and fast iterations involving both co-design and development. Some authors conclude that the use of the scrum framework in health solutions efficiently helps to carry out activities by allowing careful analysis of each stage with regard to quality, technology, and implementation [ ].
To conduct this review, we analyzed the studies through the 4 common phases of a development process: obtention of the requirements, design of the proposal, development of the prototype or product, and validation . Regarding the requirements phase, some of the included studies detailed how the co-creation process was carried out. The main methods were interviews, workshops, expert meetings, surveys, and focus groups. The current literature indicates that qualitative studies using a focus group are a methodology widely used to engage stakeholders during co-creation [ ]. Regarding the design phase, in 2 studies, a design phase was not detailed, but caregivers participated in workshops to co-create content and functionality in the development phase. The involvement of end users in the design process is known to be immensely valuable and facilitates a design process that is intuitive and attuned to the end users’ needs [ ]. The development phase was detailed in most studies, and caregivers (and other participants) were involved. The involvement of co-creation agents varied from testing feedback sessions to participating in workshops to create and select content and functionality. In the studies in which validation was detailed, a pilot with a significant sample of participants and a validation of the prototype with a small sample were conducted. Few of these studies use standardized questionnaires. The case study methodology is another tool found in the literature to understand which co-created solutions work effectively [ ].
In a co-creation process in which caregivers are the target population, it would be expected that they would be involved in all 4 phases of the process. However, of the 11 studies analyzed, in only 2 cases [, ], caregivers were involved in all phases, although the studies did not describe the validation phase or its results but simply stated that a usability test or a pilot was carried out at a later stage. Nevertheless, the current literature indicates that end users should be extensively involved, in many different roles, which will give them the opportunity to not only evaluate each phase but also enlarge their involvement in the eventual product [ ].
Regarding the analysis of the effectiveness of the use of co-created technological solutions by caregivers, with this systematic review, we could not determine the long-term outcomes of the identified projects. Most of the articles were published less than 2 years before the search for this study was carried out, so this could be the main reason why experimental studies were not found to analyze the impact of the use of the proposed technological solution. In addition, 2 of the included studies were published in 2011 [, ]. In these cases, it is possible that experimental studies used co-created technology solutions, but they were not returned in the search because the co-creation process was not the focus of the study. The study by Lemetyinen et al [ ] is the only experimental study that met the requirements, but they did not explain their co-creation process, and the previous study that detailed the process was not found in our search. The web application on which the intervention was based is not currently operational. Considering the timeliness of the studies included in this systematic review, it could be of great interest in the future to carry out a study that allows determining the success rate of the development and implementation of the technology, the analysis of its effectiveness, and if the technology was offered to the general public. Since this is the purpose of the design and development of technological solutions, it is well known that numerous factors can intercede in this process, starting with economic factors.
After analyzing the 11 studies, the future of technology solutions for caregivers need to focus on (1) identifying the common needs of caregivers, regardless of for whom they care, to be able to create specific solutions with them; (2) providing more detailed information on the creation process, because if caregivers were involved, this can add value for its use; (3) involve caregivers more actively at all stages of the creative process, as this can substantially increase the usefulness of the created technologies; and (4) use validated tools regularly and to increase the scientific evidence on the impact of the technological solutions created.
Strengths and Limitations
The main strengths of this review are based on the multiple steps performed to achieve methodological rigor. The review was guided by PRISMA, and the database searches, screening, data extraction, methodological assessment, and level of evidence evaluations were conducted in duplicate, with strong agreement between reviewers. In addition, the search was conducted in 2 languages, English and Spanish, which allowed for a broader review of the literature. Finally, as far as we know, this is the first review focused on co-created technological solutions for caregivers in health care.
Regarding the limitations, although appropriate keywords were used, there may be a certain word from a specific area that has not been checked. Consultation with a librarian could have helped. Another limitation of the systematic review is that a meta-analysis could not be performed since heterogeneous studies with poor methodological quality and limited results emerged. Future research using validated tools is needed to evaluate the technological solutions for a more in-depth analysis.
In summary, the current systematic review suggests that, despite the increasing need to provide technological support for informal caregivers, very few studies included them in the co-creation process. The existing co-created technological solutions in the health field for caregivers are mostly mobile or web applications aimed at supporting caregivers' social health and well-being and improve their health knowledge when delivering care to patients, most commonly older people. As for the co-creation process, caregivers are more likely to be involved at the time of development and in the design. Future research should include the following criteria: detailed reporting on the co-creation process, involving caregivers more actively in all phases of the process, and using validated tools to evaluate the impact of the technological solutions created. Scientific evidence could help informal caregivers in their caregiving tasks.
This study received support from the Erasmus+ programme of the European Union (project reference: 612532-EPP-1-2019-1-ES-EPPKA2-KA).
Conflicts of Interest
Search terms.DOCX File , 19 KB
Methodological quality assessment of the included articles.XLSX File (Microsoft Excel File), 53 KB
Detailed phases of the co-creation process in the included articles.XLSX File (Microsoft Excel File), 30 KB
- Adelman RD, Tmanova LL, Delgado D, Dion S, Lachs MS. Caregiver burden: a clinical review. JAMA 2014 Mar 12;311(10):1052-1060. [CrossRef] [Medline]
- Healthcare expenditure, UK Health Accounts: 2020. Office for National Statistics. 2022 May 09. URL: https://www.ons.gov.uk/peoplepopulationandcommunity/healthandsocialcare/healthcaresystem/bulletins/ukhealthaccounts/2020 [accessed 2023-04-07]
- Directorate-General for Employment, Social Affairs and Inclusion (European Commission), London School of Economics and Political Science (LSE), Zigante V. Informal care in Europe: exploring formalisation, availability and quality. Publications Office of the European Union. 2018 Jun 11. URL: https://op.europa.eu/en/publication-detail/-/publication/96d27995-6dee-11e8-9483-01aa75ed71a1/language-en [accessed 2023-04-07]
- Disability and Carers. Australian Government Department of Social Services. URL: https://www.dss.gov.au/disability-and-carers/carers [accessed 2023-04-07]
- Besnard X, Brunel M, Couvert N, Roy D. Caregivers of seniors and their feelings about the help provided - Results of the "CARE" surveys of caregivers (2015-2016). Directorate of Research, Studies, Evaluation and Statistics. 2015. URL: https://drees.solidarites-sante.gouv.fr/publications/les-dossiers-de-la-drees/les-proches-aidants-des-seniors-et-leur-ressenti-sur-laide [accessed 2023-04-07]
- Caregiving in the U.S. 2020: A Focused Look at Family Caregivers of Adults Age 18 to 49. The National Alliance for Caregiving. 2020 Dec. URL: https://www.caregiving.org/wp-content/uploads/2021/05/AARP1339_RR_Caregiving18to49_5081.pdf [accessed 2023-04-07]
- Lopez-Hartmann M, Wens J, Verhoeven V, Remmen R. The effect of caregiver support interventions for informal caregivers of community-dwelling frail elderly: a systematic review. Int J Integr Care 2012 Aug 10;12(5):e133 [FREE Full text] [CrossRef] [Medline]
- Brown EL, Ruggiano N, Page TF, Roberts L, Hristidis V, Whiteman KL, et al. CareHeroes web and Android™ apps for dementia caregivers: a feasibility study. Res Gerontol Nurs 2016;9(4):193-203 [FREE Full text] [CrossRef] [Medline]
- Vaingankar JA, Chong SA, Abdin E, Picco L, Jeyagurunathan A, Zhang Y, et al. Care participation and burden among informal caregivers of older adults with care needs and associations with dementia. Int Psychogeriatr 2016 Feb;28(2):221-231 [FREE Full text] [CrossRef] [Medline]
- Aksoydan E, Aytar A, Blazeviciene A, van Bruchem-Visser RL, Vaskelyte A, Mattace-Raso F, et al. Is training for informal caregivers and their older persons helpful? A systematic review. Arch Gerontol Geriatr 2019;83:66-74. [CrossRef] [Medline]
- Mortenson WB, Demers L, Fuhrer MJ, Jutai JW, Lenker J, DeRuyter F. How assistive technology use by individuals with disabilities impacts their caregivers: a systematic review of the research evidence. Am J Phys Med Rehabil 2012 Nov;91(11):984-998 [FREE Full text] [CrossRef] [Medline]
- Broderick J, Devine T, Lemerise A, Lier S, Harris L. Designing health literate mobile apps. NAM Perspectives 2014 Jan 28;4(1):1 [FREE Full text] [CrossRef]
- Hochstenbach LMJ, Courtens AM, Zwakhalen SMG, Vermeulen J, van Kleef M, de Witte LP. Co-creative development of an eHealth nursing intervention: Self-management support for outpatients with cancer pain. Appl Nurs Res 2017 Aug;36:1-8 [FREE Full text] [CrossRef] [Medline]
- Kalem G, Turhan. Mobile technology applications in the healthcare industry for disease management and wellness. Procedia - Social and Behavioral Sciences 2015 Jul;195:2014-2018. [CrossRef]
- Ward ME, De Brún A, Beirne D, Conway C, Cunningham U, English A, et al. Using co-design to develop a collective leadership intervention for healthcare teams to improve safety culture. Int J Environ Res Public Health 2018 Jun 05;15(6):1 [FREE Full text] [CrossRef] [Medline]
- Bate P, Robert G. Experience-based design: from redesigning the system around the patient to co-designing services with the patient. Qual Saf Health Care 2006 Oct;15(5):307-310 [FREE Full text] [CrossRef] [Medline]
- Bovaird T, Loeffler E. From engagement to co-production: the contribution of users and communities to outcomes and public value. Voluntas 2012 Aug 4;23(4):1119-1138. [CrossRef]
- Cho HJ, Smith D, Hart A, Prasad R, Sata SS, Clarke K, et al. Choosing wisely in adult hospital medicine: co-creation of new recommendations for improved healthcare value by clinicians and patient advocates. J Gen Intern Med 2022 Aug 06;37(10):2454-2461. [CrossRef] [Medline]
- Ghosh S, Struminger BB, Singla N, Roth BM, Kumar A, Anand S, et al. Appreciative inquiry and the co-creation of an evaluation framework for Extension for Community Healthcare Outcomes (ECHO) implementation: a two-country experience. Eval Program Plann 2022 Jun;92:102067 [FREE Full text] [CrossRef] [Medline]
- Langley J, Wolstenholme D, Cooke J. 'Collective making' as knowledge mobilisation: the contribution of participatory design in the co-creation of knowledge in healthcare. BMC Health Serv Res 2018 Jul 25;18(1):585 [FREE Full text] [CrossRef] [Medline]
- Bruvik FK, Ulstein ID, Ranhoff AH, Engedal K. The quality of life of people with dementia and their family carers. Dement Geriatr Cogn Disord 2012;34(1):7-14 [FREE Full text] [CrossRef] [Medline]
- Zacharopoulou G, Zacharopoulou V, Lazakidou A. Quality of life for caregivers of elderly patients with dementia and measurement tools: a review. International Journal Of Health Research and Innovation 2015;3(1):49-64 [FREE Full text]
- Boyd H, McKernon S, Mullin B, Old A. Improving healthcare through the use of co-design. N Z Med J 2012 Jun 29;125(1357):76-87. [Medline]
- Dickson K, Yeung CA. PRISMA 2020 updated guideline. Br Dent J 2022 Jun 10;232(11):760-761. [CrossRef] [Medline]
- Harbour R, Miller J. A new system for grading recommendations in evidence based guidelines. BMJ 2001 Aug 11;323(7308):334-336 [FREE Full text] [CrossRef] [Medline]
- Pace R, Pluye P, Bartlett G, Macaulay AC, Salsberg J, Jagosh J, et al. Testing the reliability and efficiency of the pilot Mixed Methods Appraisal Tool (MMAT) for systematic mixed studies review. Int J Nurs Stud 2012 Jan;49(1):47-53. [CrossRef] [Medline]
- Leask CF, Sandlund M, Skelton DA, Altenburg TM, Cardon G, Chinapaw MJM, GrandStand, Safe Step and Teenage Girls on the Move Research Groups. Framework, principles and recommendations for utilising participatory methodologies in the co-creation and evaluation of public health interventions. Res Involv Engagem 2019 Jan 9;5(1):2 [FREE Full text] [CrossRef] [Medline]
- Backman C, Harley A, Kuziemsky C, Mercer J, Peyton L. MyPath to Home web-based application for the geriatric rehabilitation program at Bruyère Continuing Care: user-centered design and feasibility testing study. JMIR Form Res 2020 Sep 14;4(9):e18169 [FREE Full text] [CrossRef] [Medline]
- Egan K, Hodgson W, Dunlop M, Imperatore G, Kirk A, Maguire R. A novel mobile app ("CareFit") to support informal caregivers to undertake regular physical activity from home during and beyond COVID-19 restrictions: co-design and prototype development study. JMIR Form Res 2021 Oct 01;5(10):e27358 [FREE Full text] [CrossRef] [Medline]
- de la Harpe R, van Zyl I. Community 2.0: A collaborative university-community network to inform and educate caregivers in home-based healthcare in South Africa. 2011 Presented at: 6th International Conference on Pervasive Computing and Applications; October 26-28, 2011; Port Elizabeth, South Africa. [CrossRef]
- Guerrero E, Lu M, Yueh H, Lindgren H. Designing and evaluating an intelligent augmented reality system for assisting older adults’ medication management. Cognitive Systems Research 2019 Dec;58:278-291. [CrossRef]
- Harding R, Carrasco JM, Serrano-Pons J, Lemaire J, Namisango E, Luyirika E, et al. Design and evaluation of a novel mobile phone application to improve palliative home-care in resource-limited settings. J Pain Symptom Manage 2021 Jul;62(1):1-9. [CrossRef] [Medline]
- Latulippe K, Hamel C, Giroux D. Integration of conversion factors for the development of an inclusive eHealth tool with caregivers of functionally dependent older persons: social justice design. JMIR Hum Factors 2020 Aug 26;7(3):e18120 [FREE Full text] [CrossRef] [Medline]
- Lemetyinen H, Onwumere J, Drake R, Abel K, Haigh C, Moulton G, et al. Co-production and evaluation of an e-learning resource to improve African-Caribbean families' knowledge about schizophrenia and engagement with services: a pilot randomised controlled trial protocol. Pilot Feasibility Stud 2018;4:174 [FREE Full text] [CrossRef] [Medline]
- Meiland FJM, Hattink BJJ, Overmars-Marx T, de Boer ME, Jedlitschka A, Ebben PWG, et al. Participation of end users in the design of assistive technology for people with mild to severe cognitive problems; the European Rosetta project. Int Psychogeriatr 2014 May;26(5):769-779. [CrossRef] [Medline]
- O'Connor S. Co-designing technology with people with dementia and their carers: Exploring user perspectives when co-creating a mobile health application. Int J Older People Nurs 2020 Sep;15(3):e12288. [CrossRef] [Medline]
- Rathnayake S, Moyle W, Jones C, Calleja P. Co-design of an mHealth application for family caregivers of people with dementia to address functional disability care needs. Inform Health Soc Care 2021 Mar 02;46(1):1-17. [CrossRef] [Medline]
- Sin J, Henderson C, Woodham LA, Sesé Hernández A, Gillard S. A multicomponent eHealth intervention for family carers for people affected by psychosis: a coproduced design and build study. J Med Internet Res 2019 Aug 06;21(8):e14374 [FREE Full text] [CrossRef] [Medline]
- Vargo SL, Lusch RF. Evolving to a new dominant logic for marketing. Journal of Marketing 2018 Oct 10;68(1):1-17. [CrossRef]
- Jaspers MWM, Steen T, van den Bos C, Geenen M. The think aloud method: a guide to user interface design. Int J Med Inform 2004 Nov;73(11-12):781-795. [CrossRef] [Medline]
- Lewis JR. IBM computer usability satisfaction questionnaires: Psychometric evaluation and instructions for use. International Journal of Human-Computer Interaction 1995 Jan;7(1):57-78. [CrossRef]
- Yen P, Bakken S. Review of health information technology usability study methodologies. J Am Med Inform Assoc 2012;19(3):413-422 [FREE Full text] [CrossRef] [Medline]
- Lindgren H. Personalisation of internet-mediated activity support systems in the rehabilitation of older adults: a pilot study. AIME 2009 International Workshop on Personalisation for e-Health 2009:20-27 [FREE Full text]
- Grier RA. How High is High? A Meta-Analysis of NASA-TLX Global Workload Scores. Proceedings of the Human Factors and Ergonomics Society Annual Meeting 2016 Dec 20;59(1):1727-1731 [FREE Full text] [CrossRef]
- Brooke J. SUS - a quick and dirty usability scale. In: Jordan PW, Thomas B, McClelland IL, Weerdmeester B, editors. Usability Evaluation In Industry. Boca Raton, FL: CRC Press; 1996.
- Degnan A, Berry K, James S, Edge D. Development, validation and cultural-adaptation of the knowledge about psychosis questionnaire for African-Caribbean people in the UK. Psychiatry Res 2018 May;263:199-206. [CrossRef] [Medline]
- Madianos M, Economou M, Peppou LE, Kallergis G, Rogakou E, Alevizopoulos G. Measuring public attitudes to severe mental illness in Greece: Development of a new scale. Eur. J. Psychiat 2012 Mar;26(1):55-67. [CrossRef]
- Ware J, Kosinski M, Keller SD. A 12-Item Short-Form Health Survey: construction of scales and preliminary tests of reliability and validity. Med Care 1996 Mar;34(3):220-233. [CrossRef] [Medline]
- Parasuraman A, Colby CL. An updated and streamlined technology readiness index. Journal of Service Research 2014 Jun 22;18(1):59-74. [CrossRef]
- Garnett A, Northwood M, Ting J, Sangrar R. mHealth interventions to support caregivers of older adults: equity-focused systematic review. JMIR Aging 2022 Jul 08;5(3):e33085 [FREE Full text] [CrossRef] [Medline]
- McKay FH, Wright A, Shill J, Stephens H, Uccellini M. Using health and well-being apps for behavior change: a systematic search and rating of apps. JMIR Mhealth Uhealth 2019 Jul 04;7(7):e11926 [FREE Full text] [CrossRef] [Medline]
- Hawker CO, Merkouris SS, Youssef GJ, Dowling NA. A smartphone-delivered ecological momentary intervention for problem gambling (GamblingLess: Curb Your Urge): single-arm acceptability and feasibility trial. J Med Internet Res 2021 Mar 26;23(3):e25786 [FREE Full text] [CrossRef] [Medline]
- Hanssen E, Balvert S, Oorschot M, Borkelmans K, van Os J, Delespaul P, et al. An ecological momentary intervention incorporating personalised feedback to improve symptoms and social functioning in schizophrenia spectrum disorders. Psychiatry Res 2020 Feb;284:112695 [FREE Full text] [CrossRef] [Medline]
- Smith KE, Juarascio A. From ecological momentary assessment (EMA) to ecological momentary intervention (EMI): past and future directions for ambulatory assessment and interventions in eating disorders. Curr Psychiatry Rep 2019 Jun 04;21(7):53. [CrossRef] [Medline]
- Dias N, Boring E, Johnson LA, Grossoehme DH, Murphy S, Friebert S. Developing a theoretically grounded, digital, ecological momentary intervention for parental bereavement care using the ORBIT model-Phase 1. Death Stud 2021 Apr 29:1-10. [CrossRef] [Medline]
- Barrigon ML, Porras-Segovia A, Courtet P, Lopez-Castroman J, Berrouiguet S, Pérez-Rodríguez MM, MEmind Study Group, et al. Smartphone-based Ecological Momentary Intervention for secondary prevention of suicidal thoughts and behaviour: protocol for the SmartCrisis V.2.0 randomised clinical trial. BMJ Open 2022 Sep 20;12(9):e051807 [FREE Full text] [CrossRef] [Medline]
- Nesvåg S, McKay JR. Feasibility and effects of digital interventions to support people in recovery from substance use disorders: systematic review. J Med Internet Res 2018 Aug 23;20(8):e255 [FREE Full text] [CrossRef] [Medline]
- Saad A, Bruno D, Camara B, D'Agostino J, Bolea-Alamanac B. Self-directed technology-based therapeutic methods for adult patients receiving mental health services: systematic review. JMIR Ment Health 2021 Nov 26;8(11):e27404 [FREE Full text] [CrossRef] [Medline]
- Stokes K, Oronti B, Cappuccio FP, Pecchia L. Use of technology to prevent, detect, manage and control hypertension in sub-Saharan Africa: a systematic review. BMJ Open 2022 Apr 05;12(4):e058840 [FREE Full text] [CrossRef] [Medline]
- Berkanish P, Pan S, Viola A, Rademaker Q, Devine KA. Technology-based peer support interventions for adolescents with chronic illness: a systematic review. J Clin Psychol Med Settings 2022 Dec;29(4):911-942 [FREE Full text] [CrossRef] [Medline]
- Lavallee DC, Lee JR, Semple JL, Lober WB, Evans HL. Engaging patients in co-design of mobile health tools for surgical site infection surveillance: implications for research and implementation. Surg Infect (Larchmt) 2019 Oct;20(7):535-540 [FREE Full text] [CrossRef] [Medline]
- Finkel S, Czaja SJ, Schulz R, Martinovich Z, Harris C, Pezzuto D. E-care: a telecommunications technology intervention for family caregivers of dementia patients. Am J Geriatr Psychiatry 2007 May;15(5):443-448. [CrossRef] [Medline]
- Mandolfo M, Chen S, Noci G. Co-creation in new product development: Which drivers of consumer participation? International Journal of Engineering Business Management 2020 Mar 24;12:184797902091376. [CrossRef]
- Thordardottir B, Malmgren Fänge A, Lethin C, Rodriguez Gatta D, Chiatti C. Acceptance and use of innovative assistive technologies among people with cognitive impairment and their caregivers: a systematic review. Biomed Res Int 2019 Mar 06;2019:9196729-9196718 [FREE Full text] [CrossRef] [Medline]
- Hall AK, Backonja U, Painter I, Cakmak M, Sung M, Lau T, et al. Acceptance and perceived usefulness of robots to assist with activities of daily living and healthcare tasks. Assist Technol 2019;31(3):133-140. [CrossRef] [Medline]
- Wolff JL, Spillman BC, Freedman VA, Kasper JD. A national profile of family and unpaid caregivers who assist older adults with health care activities. JAMA Intern Med 2016 Mar;176(3):372-379 [FREE Full text] [CrossRef] [Medline]
- Queluz FNFR, Kervin E, Wozney L, Fancey P, McGrath PJ, Keefe J. Understanding the needs of caregivers of persons with dementia: a scoping review. Int Psychogeriatr 2020 Jan;32(1):35-52. [CrossRef] [Medline]
- Tiersen F, Batey P, Harrison MJC, Naar L, Serban A, Daniels SJC, et al. Smart home sensing and monitoring in households with dementia: user-centered design approach. JMIR Aging 2021 Aug 11;4(3):e27047 [FREE Full text] [CrossRef] [Medline]
- Strudwick G, McLay D, Lo B, Shin HD, Currie L, Thomson N, et al. Development of a resource guide to support the engagement of mental health providers and patients with digital health tools: multimethod study. J Med Internet Res 2021 Apr 22;23(4):e25773 [FREE Full text] [CrossRef] [Medline]
- Abba S, Wadumi Namkusong J, Lee JA, Liz Crespo M. Design and performance evaluation of a low-cost autonomous sensor interface for a smart IoT-based irrigation monitoring and control system. Sensors (Basel) 2019 Aug 21;19(17):3643 [FREE Full text] [CrossRef] [Medline]
- Pedraza LL, Moraes JRWD, Rabelo-Silva ER. Development and testing of a text messaging (SMS) monitoring software application for acute decompensated heart failure patients. Rev Lat Am Enfermagem 2020 Sep 07;28:e3301 [FREE Full text] [CrossRef] [Medline]
- Klein L. Agile Development. Interaction Design Foundation. URL: https://www.interaction-design.org/literature/topics/agile-development [accessed 2022-10-06]
- Torrente G, de Souza TQ, Tonaki L, Cardoso AP, Manickchand Junior L, da Silva GO. Scrum framework and health solutions: management and results. Stud Health Technol Inform 2021 Dec 15;284:290-294. [CrossRef] [Medline]
- Ariza-Vega P, Prieto-Moreno R, Mora-Traverso M, Molina-García P, Ashe MC, Martín-Matillas M. Co-creation of mHealth intervention for older adults with hip fracture and family caregivers: a qualitative study. Disabil Rehabil Assist Technol 2022 Oct 29:1-10. [CrossRef] [Medline]
- Korving H, Sterkenburg PS, Barakova EI, Feijs LMG. Designing pain visualisation for caregivers of people with special needs: A co-creation approach. Heliyon 2022 Dec;8(12):e11975 [FREE Full text] [CrossRef] [Medline]
- Smith G, Dixon C, Neiva Ganga R, Greenop D. How do we know co-created solutions work effectively within the real world of people living with dementia? Learning methodological lessons from a co-creation-to-Evaluation Case Study. Int J Environ Res Public Health 2022 Nov 02;19(21):14317 [FREE Full text] [CrossRef] [Medline]
|EMI: ecological momentary intervention|
|PRSIMA: Preferred Reporting Items for Systematic Reviews and Meta-Analysis|
|RCT: randomized controlled trial|
|SIGN: Scottish Intercollegiate Guidelines Network|
Edited by T Leung; submitted 20.07.22; peer-reviewed by M Northwood, A Garnett; comments to author 10.09.22; revised version received 20.10.22; accepted 14.03.23; published 01.05.23Copyright
©Jose Antonio Merchán-Baeza, Cristina Borralleras Andreu, Eduard Minobes-Molina, Sergi Grau Carrión, Montse Romero-Mas, Anna Ramon-Aribau. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 01.05.2023.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.