This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.
Artificial intelligence (AI) and digital health technological innovations from startup companies used in clinical practice can yield better health outcomes, reduce health care costs, and improve patients' experience. However, the integration, translation, and adoption of these technologies into clinical practice are plagued with many challenges and are lagging. Furthermore, explanations of the impediments to clinical translation are largely unknown and have not been systematically studied from the perspective of AI and digital health care startup founders and executives.
The aim of this paper is to describe the barriers to integrating early-stage technologies in clinical practice and health care systems from the perspectives of digital health and health care AI founders and executives.
A stakeholder focus group workshop was conducted with a sample of 10 early-stage digital health and health care AI founders and executives. Digital health, health care AI, digital health–focused venture capitalists, and physician executives were represented. Using an inductive thematic analysis approach, transcripts were organized, queried, and analyzed for thematic convergence.
We identified the following four categories of barriers in the integration of early-stage digital health innovations into clinical practice and health care systems: (1) lack of knowledge of health system technology procurement protocols and best practices, (2) demanding regulatory and validation requirements, (3) challenges within the health system technology procurement process, and (4) disadvantages of early-stage digital health companies compared to large technology conglomerates. Recommendations from the study participants were also synthesized to create a road map to mitigate the barriers to integrating early-stage or novel digital health technologies in clinical practice.
Early-stage digital health and health care AI entrepreneurs identified numerous barriers to integrating digital health solutions into clinical practice. Mitigation initiatives should create opportunities for early-stage digital health technology companies and health care providers to interact, develop relationships, and use evidence-based research and best practices during health care technology procurement and evaluation processes.
Emergent technologies in health care, such as digital health care technologies and artificial intelligence, have changed and shaped clinical care for patients by filling gaps in the current health care delivery system. One primary driver for the digitization of health care is early-stage digital health care startups [
Early-stage digital health companies, often referred to as “startups,” which produce advanced technological solutions such as artificial intelligence (AI), machine learning, and digital health interventions, pass through different critical development stages [
Despite the importance of health care systems' relationships to early-stage digital health companies, partnerships integrating machine learning, AI or digital health care technologies into clinical practice are plagued with many challenges and thus are generally slow, preventing implementation in places where it could be most beneficial [
Progression of early-stage digital health and health care artificial intelligence startup funding, regulation, and integration into clinical practice. FDA: Food and Drug Administration.
The aim of the study was to describe the barriers to integrating early-stage digital health technologies in clinical practice and to suggest intervention and education strategies to mitigate the barriers. This is a pressing issue, as many patients cannot wait long periods for the necessary technologies to be adopted in their city or town. Although many early-stage digital health technology companies bring exciting innovations to clinical care, there are significant barriers to integrating these technologies into clinical practice and health care systems [
Previous reviews and studies highlighted the limitations in the quality of literature on the consumer aspect of digital health technologies, insights into entrepreneurial orientation and motivation, and the perspectives of founders or entrepreneurs [
An exploratory, descriptive, qualitative study was conducted using a stakeholder focus group workshop design on early-stage digital health and health care AI entrepreneurs who are leaders of companies that develop digital health technologies [
Thus, a stakeholder focus group workshop is a practical way to collect information from stakeholders, encouraging group interactions. This design provides in-depth insight into under-researched areas by interviewing stakeholders and experts to give an in-depth description.
We conducted a stakeholder focus group workshop facilitated by the principal investigator and two trained assistants, which lasted approximately 65 minutes [
The participants were recruited for the workshops using snowball and convenience sampling [
Participants who confirmed participation in the workshop had to sign an informed consent form. At the start of the workshop, the principal investigator gave an evidence presentation to ensure that the stakeholders were familiarized with consistent information about the workshop. Evidence presentations are used in expert elicitations to capture and present all pertinent information that stakeholders rely on to formulate their opinion [
We aimed to optimize internal validity by providing access to the current data so the stakeholder could form opinions based on their different expertise and experiences. Optimizing internal validity was necessary to maintain the rigor of qualitative research and ensure the research results were trustworthy and credible. The presentation offered basic definitions of health care innovation integration, product definitions, summarized statistics, and provided an overview of the current state of the early-stage innovations integration into clinical practice [
Data collection consisted of the participants listing barriers to integrating their products into clinical practice in a written notebook and the audio recording of the discussion. Using Braun and Clarke's inductive thematic analysis approach, we analyzed the transcripts and used NVivo 12 (QSR International) to organize, query, and explore data for thematic convergence [
The stakeholders were informed orally during the workshop and in writing about the study's objective, privacy considerations, and voluntary participation. The participants' notebooks and audio and transcribed discussions were kept private, and their documents remained anonymous. The research group collected written informed consent.
To provide insight into the unique challenges faced by early-stage digital health and health care AI entrepreneurs, we conducted a multidisciplinary stakeholder workshop with 10 participants (n=4, 40% female and n=6, 60% male participants) representing the following groups of stakeholders: digital health entrepreneurs (4/10, 40%), health care AI entrepreneurs (4/10, 40%), a digital health entrepreneur turned venture capitalist (1/10, 10%), and a physician and digital health entrepreneur (1/10, 10%).
Description of stakeholder participants in the workshop focus group (n=10).
Stakeholder | Description of participants | Value, n (%) |
Digital health entrepreneurs | Founder or chief executive officer of a preprofit company with a digital health solution for cardiovascular medicine | 4 (40) |
Health care AIa entrepreneurs | Founder or chief executive officer of the preprofit company with an AI health solution for cardiovascular medicine | 4 (40) |
Digital health entrepreneur turned venture capitalist | Previous founder of preprofit company with a digital health solution for cardiovascular medicine turned venture capitalist (funder) | 1 (10) |
Physician and digital health entrepreneur | Physician and current founder of a company with an AI health solution for cardiovascular medicine | 1 (10) |
aAI: artificial intelligence.
Based on the written list in the notebooks and the transcribed discussion, we identified the following four overarching categories, as well as numerous sub-barriers, in the integration of early-stage or novel digital health technologies in clinical practice and health care systems: (1) lack of knowledge on health care system technology procurement protocols and best practices, (2) demanding regulatory and validation requirements, (3) challenges within the health care system technology procurement process, and (4) disadvantages of early-stage digital health companies compared to large technology conglomerates, as displayed in
List of barriers of digital health and health care AIa in integration of their innovations into clinical health care systems and operation from the workshop.
Themes | Barriers |
Knowledge on health care systems' technology procurement process |
Lack of knowledge on health care systems' technology procurement protocols Limited access to best practices and strategies for successful technology procurement Venture funding leads more companies to sale directly to employers Lack of awareness on how to reach and educate providers on product offerings |
Digital health innovations from large technology companies |
Competing with large technology companies Lack of large marketing departments Lack of broad network of connections in comparison to larger companies Lack of networking and financial resources in comparison to larger companies |
Demanding regulatory and validation requirements |
Strenuous regulatory, validation, and technology evaluation evidence required from health care systems Lack of funding for randomized controlled trials Inappropriate existing study design to evaluate digital health innovations Inability to publish study results in academic journals and other peer review mediums due to proprietary concerns Lack of ability to explain AI algorithms |
Success in health care systems' technology procurement |
Limited information and uniformity on the health care procurement process Lengthy sales cycle Strenuous marketing and networking process Lack of transparency on who the decision maker is Lack of funding to attend conference trade shows Limited resources to support a health care pilot that demonstrates financial and clinical ROIb |
aAI: artificial intelligence.
bROI: return on investment.
The stakeholders expressed deficiencies in their knowledge of the health care sales cycle and implementation process of digital technologies into clinical practice. All members of the group mentioned there is limited information and uniformity on the health care procurement process. They argued that each client and clinical health care organization, system, or physician has a unique process for vendor purchasing and selection. Furthermore, they stated the knowledge on how to sell into a particular health organization is derived from best practices of colleagues and members of their professional network. Overall, all members of the group highlighted the difficulty of selling health innovations due to the lack of knowledge on the process.
It is an extremely complicated and frustrating process. Each client and clinical healthcare organization, system and/or physician has a unique process for vendor purchasing and selection. We have to learn them all.
The participants raised concerns about the strenuous regulatory, validation, and technology evaluation evidence that is required for their products to be used in clinical settings. Randomized controlled trials are the golden standard and oftentimes are the inappropriate study design for the evaluation of digital health innovations. When asked about their preferred methodology and validation processes and procedures for evaluating their technologies, the participants' answers varied. All of them shared and expressed confusion and frustration toward the challenges in the evaluation of digital health solutions. Only 2 participants stated they published study results in academic journals and other peer-reviewed mediums. The participants with AI solutions shared that potential clients require extensive detailed information on the back end of how their technology functions, especially products that have AI-driven decision-making capabilities. Explainability versus accuracy is a debate the entrepreneurs have with their teams constantly. To summarize, they are not sure if health care systems would prefer simpler AI innovations that are less accurate or complex AI innovations with high accuracy and low explainability.
Early-stage startups are severely disadvantaged. We have research that supports our product's usability, effectiveness, and safety, but it seems that everyone wants RCTs (randomized controlled trials). RCTs are extremely expensive. It is like a cost-benefit. If the benefit cannot keep up with the cost, our products will not be implemented into the practice.
I've chaired medical device committees in various healthcare entities for many years. It sadly depends on the system. My current hospital prefers results from randomized control trials. My previous hospital relied on patient/provider testimonies and user research feedback to evaluate if we should buy a product. Large tech companies have 200 people in their sales, research and product development teams, who can find this info out.
The participants were asked to name the top 3 health care system technology procurement barriers experienced by early-stage health care technology entrepreneurs. The overall response to this question was remarkable. Six (60%) participants commented on the length of the sales cycle. The average length of the sale cycle in the group was 13 months. A small minority of the participants indicated the marketing and networking process as one of the biggest hurdles. Surprisingly, all participants mentioned that the top barrier was lack of information on the appropriate decision maker and process. In the group, the most used strategy to connect with decision makers at hospital and health care systems was to cold-call each department and ask for a referral and contact information of specific personnel.
It's hard to find the right person to talk to. We have limited resources and, frankly, time. It is important to speak directly with a decisionmaker. The problem, though, is without a connection from my network, it is tough to reach out to them. The decision-maker varies depending on the organization.
All participants were confident in large health care technology companies' role in the unique challenges of integrating their products into clinical practice. The participants identified numerous barriers, listed in
Because of the unique nature of healthcare sales cycles, [digital health entrepreneurs] are recommended to raise more funding dollars at the early stage than other venture-backed technology companies. Basically, successfully integrating new technologies relies on the early-stage startup's availability to segment sufficient marketing dollars for the entire length of the sale process.
The results of our study highlighted the barriers facing the integration of early-stage digital health innovations into clinical practice and health care systems. We identified 4 areas of barriers, as follows: lack of knowledge on health care systems' technology procurement protocols and best practices, demanding regulatory and validation requirements, challenges within the health care systems' technology procurement process, and disadvantages of early-stage digital health companies compared to large technology conglomerates.
It remains an industry-wide challenge to evaluate digital health solutions and provide credible evidence, hindering adoption and widespread use in health care [
Digital health innovations and artificial intelligence in health care are financed by venture capital and debt. Venture capital firms and startup friendly banks provide the majority of the financing for early-stage startup companies [
All the early-stage health care technology entrepreneurs in the workshop alluded to their current funding status or aspiration of funding as the key driver to their choice to integrate their products into health care systems. Helminen et al [
Early-stage companies need more financial resources for robust health care enterprise sales and marketing departments [
Due to the unique nature of health care sales cycles, digital health entrepreneurs are recommended to raise more funding dollars than other venture-backed technology companies. In other words, successfully integrating new technologies into clinical practice relies on the early-stage startup's availability to segment sufficient marketing dollars for the entire length of the sale process [
To mitigate the barriers early-stage digital health and health care AI entrepreneurs experience when integrating technologies, we provide the following road map and recommendations (
Recommendations and road map to mitigate the barriers to integrating early-stage or novel digital health technologies in clinical practice.
Themes and barriers | Recommendations | ||
|
|||
|
Lack of knowledge on health care system technology procurement protocols and best practices |
Continuing education for digital health entrepreneurs, health care providers, and systems on innovations and the health care technology procurement process |
|
|
|||
|
Disadvantages of early-stage digital health companies compared to large technology conglomerates |
Special activation and initiatives to support early-stage startup health care procurement processes |
|
|
|||
|
Demanding regulatory and validation requirements |
Improving research and best practices on integrating digital health technologies into clinical practice |
|
|
|||
|
Challenges within the health care system technology procurement process |
Creation of opportunities for early digital health technology companies, venture capitalists, as well as health care providers and systems to interact and develop relationships |
|
|
Lack of bandwidth at health care systems to properly evaluate digital innovations | Creation of new health technology departments in medical systems with the following roles: Chief Information Officer Chief Research Information Officer Chief Clinical Information Officer |
Second is the creation of opportunities for early digital health technology companies, venture capitalists, health care providers, health care systems, regulatory boards, and insurance companies to interact and develop relationships. As more early-stage companies decide to raise external funding, and as the number and size of digital health deals increase yearly, it is possible that uneven funding and regulatory patterns can alter the development of digital innovations for clinical use [
It is clear from our investigation that there is a critical need for investors, payers, health care organizations, regulatory boards, and technology stakeholders to collaborate and strategize on the best processes to improve digital health care technology integration into clinical practice. An example of this type of collaboration can be found in Germany through the Digital Health Applications (DiGA) Act. Insured patients are entitled to be provided with DiGA, which can be prescribed by doctors and psychotherapists and are reimbursed by the health insurance fund [
While our study provided valuable insight into early-stage technology integration in clinical practice, this study has several limitations. The small sample size in comparison to other study designs such as surveys might prevent generalizability of the study results into other contexts. Furthermore, the sampling method of targeting leaders of preprofit companies with a digital health solution for cardiovascular medicine may offer limited generalizability to the entire AI and digital health care technology community. Additionally, it is possible that some early-stage digital health and the health care AI entrepreneur groups may have created an atmosphere where participants did not feel comfortable expressing proprietary information such as sales strategies focusing on the barriers. Different entrepreneur stakeholders might have produced the same or different themes, although thematic saturation was noted and reached [
We found that the barriers to implementing health care technologies into clinical practice are vast. Based on the narratives of early-stage digital health and health care AI entrepreneur stakeholders, it is apparent that these barriers prevent patients and providers from having access to the newest technologies in clinical practice. Health care and clinical care structures are failing to catch up with the rapid progress of the health care AI and digital medicine technology industry [
artificial intelligence
Digital Health Applications
IMO thanks Iyore N. Olaye, for her helpful feedback on the Early-Stage Digital Health Startup Financing section based on her significant expertise as an executive in various startups and investment firms. This research was supported by funding from the National Institutes of Health (T32-HL135465-03, K01HL135452, and R01HL152453). The funding sources had no role in the study's design, conduct, or analysis, or in the decision to submit the manuscript for publication
IMO: ideation, design, conceptualization, methodology, analysis, and interpretation. All authors met the ICMJE criteria for authorship, contributed equally to the subsequent preparation of the manuscript, wrote the initial draft, critically reviewed the manuscript, and reviewed and accepted the final version of the manuscript.
None declared.