Mobile health (mHealth) and related digital health interventions in the past decade have not always scaled globally as anticipated earlier despite large investments by governments and philanthropic foundations. The implementation of digital health tools has suffered from 2 limitations: (1) the interventions commonly ignore the “law of amplification” that states that technology is most likely to succeed when it seeks to augment and not alter human behavior; and (2) end-user needs and clinical gaps are often poorly understood while designing solutions, contributing to a substantial decrease in usage, referred to as the “law of attrition” in eHealth. The COVID-19 pandemic has addressed the first of the 2 problems—technology solutions, such as telemedicine, that were struggling to find traction are now closely aligned with health-seeking behavior. The second problem (poorly designed solutions) persists, as demonstrated by a plethora of poorly designed epidemic prediction tools and digital contact-tracing apps, which were deployed at scale, around the world, with little validation. The pandemic has accelerated the Indian state’s desire to build the nation’s digital health ecosystem. We call for the inclusion of regulatory sandboxes, as successfully done in the fintech sector, to provide a real-world testing environment for mHealth solutions before deploying them at scale.J Med Internet Res 2020;22(9):e21276
Millions of dollars have been spent on digital health interventions in the global south in the past few years, often with the generous support of influential global philanthropic organizations . Many have not had the anticipated impact at scale despite successful pilot studies [ , ]. The COVID-19 pandemic has amplified and accelerated this mobile health (mHealth) pilotitis with unvalidated and untested social distancing scoreboards, apps, and forecasting models proliferating around the world [ ]. As communities and businesses struggle with the uncertainties associated with reopening society, a wide range of technological solutions are being proposed—including syndromic surveillance trackers, electronic passes controlling access to building and transport hubs, and contact-tracing apps and devices [ ]. These unvetted tools risk detracting policymakers from focusing on the basics of pandemic response [ ].
Over the past decade, the implementation of digital health tools has largely suffered from 2 limitations. First, technocentric interventions commonly ignore what Toyama refers to as the law of amplification in his book Geek Heresy . Technology, he postulates, is most likely to succeed when it seeks to augment and not alter human behavior. Secondly, end-user needs are poorly understood in the interest of creating new markets, leading to a significant decrease in usage over time. This attrition is seldom studied but is important while evaluating digital health implementation, as described by Eysenbach [ ].
The minimum viable product—the norm in the start-up industry—is anathema in medicine and public health. Speed need not supplant rigor. In this article, we propose a measured iterative approach to developing, validating, and scaling digital health interventions, using illustrative examples from India.
The Law of Amplification
Before the pandemic, there were over already 350,000 apps in the mHealth category across app stores, though few have had any longevity . Most have relied on misguided assumptions of how humans consume and interact with technology and provided false reassurances to users and policymakers [ ]. Those that have been somewhat successful and had impact at scale have demonstrated a commitment to understanding the context and have adopted key design thinking principles, including an iterative and inclusive development process [ , ]. They recognize Toyama’s law of amplification [ ] that states that technology by itself will not fundamentally change behavior; it will merely amplify it.
The motion picture, radio, television, and finally, massive open online courses (MOOCs) were all expected to fundamentally change how we educated students. They did not [- ]. While the internet substantially eased the pursuit of knowledge, it had, for the most part, augmented but not entirely replaced classrooms—until the pandemic fundamentally modified human behavior. How we seek education and health has now changed; human mobility is restricted, and because students cannot travel to campuses, we are witnessing an explosive growth in online learning tools as technology responds to amplify human behavior [ ].
Telemedicine, which was expected to substantially improve access to general and specialized care, has also not scaled as anticipated. Legal, financial, behavioral, and infrastructural limitations were hard to overcome in both high- and low-income settings . The pandemic has, however, disrupted primary and chronic care globally with particularly devastating consequences in resource-poor settings. In the United States, for example, this public health emergency has accelerated negotiations about regulation and remuneration in telemedicine that had otherwise stalled for years [ ]. The 3-month-long nationwide lockdown in India precluded hundreds of thousands of patients from receiving routine home care by community health workers or periodic checks, for example, by their endocrinologists and oncologists [ ]. While physicians in the developing world have routinely been using phone-based apps, especially WhatsApp, to communicate with patients [ ], the clinical and financial imperatives posed by the pandemic prompted many hospitals, both private and public, to adopt telemedicine solutions expeditiously [ ]. The government of India released Telemedicine Practice Guidelines on March 25, 2020, allowing teleconsultation for essential non-COVID services, further embedding telemedicine in routine health care delivery [ ].
In contrast, COVID-19 mobility dashboards and contact-tracing apps promoted by governments, technology companies, consulting companies, and academics largely failed to alter the course of the epidemic . Decision rooms of health departments were inundated with presentations of prediction models, despite the unavailability of data required to parametrize these models adequately [ ]. Policymakers are weighed down by the low signal-to-noise ratio in the data generated by the health system along with the digital solutions presented to them, compelling the WHO director-general to label the phenomenon an “infodemic” [ , ]. These solutions could not amplify what did not exist—the capacity to test, isolate, or treat patients or to provide for citizens to safely stay home [ , ].
The Tyranny of “Tablets”
The discordance between what is needed and what is supplied by the digital health tool ecosystem is not new. A closer look at the national programs digitizing their health data ecosystem in the countries with low- and middle-income reveals common patterns: overburdened healthcare providers enter large volumes of data into tablet computers or web apps in survey forms riddled with radio buttons, small fonts, and hard-to-navigate dropdown menus that are all anachronistic . These tools reflect regulatory reporting and operational requirements originally designed for an analog world. In India, for example, individual programs must collect demographic data de novo, instead of relying on an application programming interface (API)-enabled import from an interoperable sister program. These data are then transferred with varying compliance and quality to dashboards with scant detail on how the summary statistics are calculated [ ]. There is little evidence to show that these efforts result in timely action. Throughout, there is a greater emphasis on programmatic needs; and little attention is paid to the needs of the key users, the patient, and the health care provider—leading to user attrition over time [ , ].
Sandboxing: The Need for Rapid Iteration
Governments currently find themselves facing tough choices between moving swiftly to reassure and protect populations by adopting promising though unvalidated technologies and proceeding cautiously, awaiting robust evidence, while the contagion spreads unabated . We believe it is possible to strike a responsible balance. In 2019, the government of India published its sweeping vision for a digital health ecosystem for India’s 1.3 billion residents, the National Digital Health Blueprint (NDHB) [ ]. The NDHB called for the provision of a regulatory sandbox—a controlled testing environment within which existing regulations may be temporarily relaxed to allow experimentation [ ]. In August 2020, the Prime Minister announced the National Digital Health Mission (NDHM), and the National Health Authority (NHA) released its “Draft Health Data Management Policy” for public comment. Simultaneously, the NHA has launched the NDHM Sandbox, allowing integration and validation of third-party software by partnering with NDHM APIs [ ]. We believe that these regulatory sandboxes can be the pill needed to combat the bane of mHealth pilotitis, in India and around the world. A sandbox will provide the enabling conditions administrators need to test new solutions in subsets of populations before mandating change at scale. They offer an opportunity to test what are often minimum viable digital health products in a responsible, controlled, and monitored real-world environment.
Let us consider, for example, follow-up home visits that are required of India’s army of accredited social health activists, or ASHAs (which means “hope” in Hindi). Several population health programs require ASHA workers to go door to door to inquire about the health status of children or expectant mothers or missed follow-up appointments. They then trudge back to the primary care centers to upload these data online, when tablet computers and broadband connectivity are not available in remote regions, and thousands of person-hours are lost weekly in this exercise. The (not entirely unfounded) fear of new technologies failing precludes administrators from experimenting or challenging age-old, resource-intensive, and sometimes ineffective practices. A sandbox would, in this case, provide state administrators with the regulatory flexibility needed to temporarily replace home visits with alternative digital solutions. In the context of the COVID-19 pandemic, there is a behavioral impetus to align the ASHA workers’ incentives to use digital solutions with patients’ needs. It is unsafe for ASHA workers to go door to door during a pandemic. The time saved by operating from home will allow them to either attend to increased household responsibilities resulting from shelter-in-place orders, spend more time with each patient, or call more patients. Patients benefit from access to care despite quarantining orders. A sandbox approach would allow the ASHA workers (and policy makers) to vet multiple solutions simultaneously, say, by comparing response rates among groups of patients self-isolating at home that received either push-notifications (inexpensive), or automated interactive text messaging (with minimal upfront development costs), or phone calls (resource-intensive). What may be effective in a certain demographic may not necessarily be generalizable, warranting constant feedback loops and the ability to rapidly iterate .
Such prototyping must begin upstream, incorporating end-user feedback early in the design process, as was successfully evinced during the development of a digital disease surveillance platform at a mass gathering in Maharashtra, India, in 2015. Preparatory consultation workshops with medical officers seconded to serve in pop-up clinics at the 2015 Kumbh Mela resulted in substantial data minimization and workflow acceleration while improving record-keeping .
Agility and Caution
We recognize that regulatory sandboxes are at risk for abuse and will lose public trust if they seek to circumvent the nation’s laws on data protection and privacy . The sandboxes could, however, themselves serve as powerful tools to test proposed technological solutions against India’s evolving data privacy jurisprudence and societal norms. Over the decades, the Indian state has amassed large volumes of health data; these data can be used to accelerate medical science and positively influence population health [ , ], but they can also result in discrimination and harm [ , ]. The ubiquity of mobile devices and cloud-based services will influence a society’s willingness to share their data—as will a better understanding of the harms and benefits of the exchange of health data. The sandboxes provide an opportunity to examine the terms on which society will engage with governments and private players to generate, exchange, and use health data and address fundamental questions about health data that are yet not fully answered across different contexts. For example, are data better utilized for research if they are secured at source and queried remotely with a role-based access, as with the Research Patient Data Registry (RPDR) by Partners Healthcare in Massachusetts [ ]; or if datasets are exported for a fee with time and purpose limitations as with research data from the Centers for Medicare and Medicaid Services (CMS) database in the United States [ ]; or without a fee, as with the MIMIC-III (Medical Information Mart for Intensive Care III) database, a comprehensive dataset of 12 years of deidentified health data from critical care units of Harvard’s Beth Israel Deaconess Medical Center [ ]? Which of these approaches are best suited to the range of infrastructure and capability differences within a country?
Policymakers have neither the requisite evidence to evaluate the veracity of the promises made by digital health technologies nor the tools to allow rapid iteration during large-scale adoption [, ]. We believe that the proposed sandboxes can provide an alternative model. These testbeds provide a means to reject what does not work in the communities served, to optimize interventions by tweaking them in a real-world context, and to generate the evidence required to scale up. Regulated with expertise, caution, and consent, they can serve as incubators for digital health innovation. Prior to making unvetted digital technologies a critical pillar of the COVID-19 response, or of the national public health system at large, a controlled launch in willing communities would provide critical insights on the technological feasibility, epidemiological utility, data protection capability, and society’s response to these innovations ( ).
Objectives of regulatory sandboxes for mHealth.
- Generate evidence for digital health interventions in the real-world context at reasonable scale
- Allow cautious relaxation of select regulations to allow for innovation
- Measure societal response to the proposed regulatory changes (and to the introduced innovations)
- Eliminate (early) interventions that fail to demonstrate impact or engender trust
Will Sandboxing Work?
The idea of testing policy changes before enforcing them is not new. The economic reforms in China from 1979 to 2012, for example, were a result of several decentralized economic experiments across cities and provinces, where hard evidence and technological expertise were relied upon to inform the rapidly changing national economic policy [, ]. The regulatory sandboxes described in the NDHB have more recent precedents in the fintech industry. First introduced in the United Kingdom in December 2015, sandboxes have rapidly spread across the Asia Pacific and used in the insurance, payments, and capital market sectors [ , ]. In November 2019, India’s 4 financial service regulators invited fintech companies to test their applications in regulatory sandboxes designed to cater to the banking, insurance, securities, and pensions sectors [ , ]. An mHealth regulatory sandbox could similarly catalyze innovation and infuse rigor in digital health implementation by providing a mechanism for evaluating the scientific validity, health impact, contextual relevance, regulatory compliance, and long-term feasibility of applications, prior to scale-up ( ).
Evaluation framework for mHealth tools in regulatory sandboxes.
- Does the innovation do what it says it does?
- Example claim: “Bluetooth technology will help identify most exposures among persons carrying enabled devices.”
- Validating this claim would require widespread testing in a target population to calculate the sensitivity and specificity of the contact-tracing app and to consequently alter detection parameters before a large-scale rollout.
- Does it result in improved clinical or population health outcomes?
- Example claim: “Human mobility dashboards predicting the impact of social distancing measures will help inform containment strategies.”
- Impact would need to be shown by a measurable change in the incidence of cases or by comparing outcomes with control group populations after implementing interventions based on information provided by the dashboard. Statistically significant testing strategies would be a prerequisite [ ].
- Are the improved outcomes a priority for the community?
- Example claim: “Digitizing vertical service delivery programs improves care quality.”
- Improved record keeping after digitization may not have an impact on population health if chronic inventory shortages and absenteeism are not addressed.
- Do regulatory relaxations result in harm, now or later?
- Example claim: “It is necessary to collect location data to monitor individual movement during the pandemic.”
- In compliance with data protection laws, data should be aggregated or anonymized to the lowest possible resolution that allows epidemiologically sound interventions without risking individual or group reidentification or harm.
- Is it easy to use? Does it integrate with existing workflows?
- Example claim: “Digitization of discharge summary notes at the bedside expedites claims of reimbursements.”
- Feasible interventions should integrate into existing systems without introducing bottlenecks or resulting in flow reorganization with stakeholder buy-in. Given the volume of patients seen in the outpatient departments at public hospitals (several hundreds), this requirement, for example, may be debilitating in certain contexts. Providers should have the option of completing discharge notes within a reasonable timeframe.
- Can it scale to meet increased demand?
- Example claim: “A successful locally developed, locally deployed electronic medical record system is ready for national expansion.”
- Sustainable scale-up proposals would require assessing the flexibility of design features to accommodate for the heterogeneity in workflow, digital literacy, and ability to report needs across jurisdictions as well as the adequacy of decision support at scale. Early findings may signal the need for subsequent RCTs.
The COVID-19 pandemic addresses 1 of the 2 key barriers to successful mHealth adoption—the discordance between the proposed solution and underlying human behavior is reduced as technological solutions are now better aligned with needs. We believe regulatory sandboxes address the other—the need for validation. As the National Health Authority and the Ministry of Health and Family Welfare launch the NDHM in August 2020, incorporating sandboxes is opportune and important . Before prescribing solutions at a national scale, the sandboxes must examine what is permissible under existing laws, where existing laws fall short, and where additional enabling regulations are needed to advance clinical or population health.
The pandemic warrants urgent intervention, not any intervention. In heterogenous populations with large socioeconomic, cultural, and behavioral diversity, this approach is far more effective than ideological adherence. Medicine is after all the original home of randomized controlled trials—now the darling of economists . The profession has, for decades, strived to make the practice evidence-based. It is time to apply the same vigilance to digital health implementation, even during the pandemic. Regulatory sandboxes may present an effective mechanism to do so.
SB received support from the Burke Fellowship at the Harvard Global Health Institute. This paper is part of a series of publications on India’s evolving digital health ecosystem by the India Digital Health Net at Harvard University.
Conflicts of Interest
- Using Data to Save Lives: The Rockefeller Foundation and Partners Launch $100 Million Precision Public Health Initiative. The Rockefeller Foundation. New York; 2019 Sep 25. URL: https://www.rockefellerfoundation.org/news/using-data-save-lives-rockefeller-foundation-partners-launch-100-million-precision-public-health-initiative/ [accessed 2020-06-09]
- Huang F, Blaschke S, Lucas H. Beyond pilotitis: taking digital health interventions to the national level in China and Uganda. Global Health 2017 Jul 31;13(1):49 [FREE Full text] [CrossRef] [Medline]
- Tomlinson M, Rotheram-Borus MJ, Swartz L, Tsai AC. Scaling up mHealth: where is the evidence? PLoS Med 2013;10(2):e1001382 [FREE Full text] [CrossRef] [Medline]
- Editorial. Show evidence that apps for COVID-19 contact-tracing are secure and effective. Nature 2020 Apr;580(7805):563. [CrossRef] [Medline]
- Sarbadhikari S, Sarbadhikari SN. The global experience of digital health interventions in COVID-19 management. Indian J Public Health 2020 Jun;64(Supplement):S117-S124 [FREE Full text] [CrossRef] [Medline]
- Baheti A. How India's lack of transparency on coronavirus data is hindering the battle against pandemic. Scroll.in. 2020 May 17. URL: https://scroll.in/article/962138/how-indias-lack-of-transparency-on-coronavirus-data-is-hindering-the-battle-against-pandemic [accessed 2020-06-09]
- Toyama K. Geek Heresy: Rescuing Social Change from the Cult of Technology. New York: PublicAffairs; May 26, 2015.
- Eysenbach G. The law of attrition. J Med Internet Res 2005 Mar 31;7(1):e11 [FREE Full text] [CrossRef] [Medline]
- mHealth Economics 2017 – Current Status and Future Trends in Mobile Health. Berlin, Germany: Research2Guidance; 2017 Nov 01. URL: https://research2guidance.com/product/mhealth-economics-2017-current-status-and-future-trends-in-mobile-health/ [accessed 2020-06-09]
- Rowland SP, Fitzgerald JE, Holme T, Powell J, McGregor A. What is the clinical value of mHealth for patients? NPJ Digit Med 2020;3:4 [FREE Full text] [CrossRef] [Medline]
- Modi D, Dholakia N, Gopalan R, Venkatraman S, Dave K, Shah S, et al. mHealth intervention "ImTeCHO" to improve delivery of maternal, neonatal, and child care services-A cluster-randomized trial in tribal areas of Gujarat, India. PLoS Med 2019 Oct;16(10):e1002939 [FREE Full text] [CrossRef] [Medline]
- Scholten H, Granic I. Use of the Principles of Design Thinking to Address Limitations of Digital Mental Health Interventions for Youth: Viewpoint. J Med Internet Res 2019 Jan 14;21(1):e11528 [FREE Full text] [CrossRef] [Medline]
- Ossiannilsson E, Altinay F, Altinay Z. MOOCs as Change Agents to Boost Innovation in Higher Education Learning Arenas. Education Sciences 2016 Aug 06;6(4):25. [CrossRef]
- Sra P, Chakraborty P. Opinion of Computer Science Instructors and Students on MOOCs in an Indian University. Journal of Educational Technology Systems 2018 Aug 28;47(2):205-212. [CrossRef]
- Dynarski S, Lalani F. Online Courses Are Harming the Students Who Need the Most Help. The New York Times. 2018 Jan 19. URL: https://www.nytimes.com/2018/01/19/business/online-courses-are-harming-the-students-who-need-the-most-help.html [accessed 2020-06-09]
- Li C. The COVID-19 pandemic has changed education forever. This is how. World Economic Forum. 2020 Apr 29. URL: https://www.weforum.org/agenda/2020/04/coronavirus-education-global-covid19-online-digital-learning/ [accessed 2020-06-09]
- Combi C, Pozzani G, Pozzi G. Telemedicine for Developing Countries. A Survey and Some Design Issues. Appl Clin Inform 2016 Nov 02;7(4):1025-1050 [FREE Full text] [CrossRef] [Medline]
- Medicare Telemedicine Health Care Provider Fact Sheet. Centers for Medicare & Medicaid Services. 2020 Mar 17. URL: https://www.cms.gov/newsroom/fact-sheets/medicare-telemedicine-health-care-provider-fact-sheet [accessed 2020-06-09]
- Cash R, Patel V. Has COVID-19 subverted global health? Lancet 2020 May 30;395(10238):1687-1688 [FREE Full text] [CrossRef] [Medline]
- Giordano V, Koch H, Godoy-Santos A, Dias Belangero W, Esteves Santos Pires R, Labronici P. WhatsApp Messenger as an Adjunctive Tool for Telemedicine: An Overview. Interact J Med Res 2017 Jul 21;6(2):e11 [FREE Full text] [CrossRef] [Medline]
- Hollander JE, Carr BG. Virtually Perfect? Telemedicine for Covid-19. N Engl J Med 2020 Apr 30;382(18):1679-1681. [CrossRef] [Medline]
- Sarbadhikari S, Nautiyal S. Disruptive Digital Health - Hastened by the Covid-19 Pandemic Internet. RGNUL Student Research Review (RSRR) Blog Series. 2020 Jun 11. URL: http://rsrr.in/2020/06/11/digital-healthcare-covid-19-technology/ [accessed 2020-08-01]
- Zastrow M. Coronavirus contact-tracing apps: can they slow the spread of COVID-19? Nature 2020 May 19. [CrossRef] [Medline]
- Holmdahl I, Buckee C. Wrong but Useful - What Covid-19 Epidemiologic Models Can and Cannot Tell Us. N Engl J Med 2020 Jul 23;383(4):303-305. [CrossRef] [Medline]
- Munich Security Conference. World Health Organization. 2020 Feb 15. URL: https://www.who.int/dg/speeches/detail/munich-security-conference [accessed 2020-06-09]
- Balsari S, Buckee C C, Khanna T. Which Covid-19 Data Can You Trust? Harvard Business Review. 2020 May 08. URL: https://hbr.org/2020/05/which-covid-19-data-can-you-trust [accessed 2020-06-09]
- McDonald S. Technology Theatre. Centre for International Governance Innovation. 2020 Jul 13. URL: https://www.cigionline.org/articles/technology-theatre [accessed 2020-08-01]
- mHealth: New Horizons for Health through Mobile Technologies. Geneva: World Health Organization; Nov 01, 2011.
- Rajan JV, Moura J, Gourley G, Kiso K, Sizilio A, Cortez AM, et al. Understanding the barriers to successful adoption and use of a mobile health information system in a community health center in São Paulo, Brazil: a cohort study. BMC Med Inform Decis Mak 2016 Nov 17;16(1):146 [FREE Full text] [CrossRef] [Medline]
- Ghazisaeidi M, Safdari R, Torabi M, Mirzaee M, Farzi J, Goodini A. Development of Performance Dashboards in Healthcare Sector: Key Practical Issues. Acta Inform Med 2015 Oct;23(5):317-321 [FREE Full text] [CrossRef] [Medline]
- Balsari S. Will AI help universalize health care? The BMJ Opinion. 2019 Sep 23. URL: https://blogs.bmj.com/bmj/2019/09/23/satchit-balsari-will-ai-help-universalize-health-care/ [accessed 2020-06-09]
- Zagury-Orly I, Schwartzstein RM. Covid-19 - A Reminder to Reason. N Engl J Med 2020 Jul 16;383(3):e12. [CrossRef] [Medline]
- Final report on National Digital Health Blueprint (NDHB). Delhi, India: Ministry of Health and Family Welfare, Govt. of India; 2019 Nov 08. URL: https://main.mohfw.gov.in/sites/default/files/Final%20NDHB%20report_0.pdf [accessed 2020-06-09]
- Matthan R. Regulatory sandbox. Livemint. 2017 Jan 25. URL: https://www.livemint.com/Opinion/jsvqNgipJgYizPaqhYtqEL/Regulatory-sandbox.html [accessed 2020-06-09]
- Ministry of Health and Family Welfare, Government of India. NDHM Sandbox Environment. URL: https://sandbox.ndhm.gov.in/ [accessed 2020-09-05]
- Moola S, John O, Bhaumik S. Encouraging health workers to use mHealth for delivering primary healthcare services: policy brief. The George Institute of Global Health. India; 2020 Jun 01. URL: https://www.georgeinstitute.org/encouraging-health-workers-to-use-mhealth-for-delivering-primary-healthcare-services-policy-brief [accessed 2020-06-09]
- Paliwal A. Tracking India's Deadly Flu Outbreak in Real Time. NOVA Next.: PBS Online; 2015 Oct 28. URL: https://www.pbs.org/wgbh/nova/article/kumbh-mela-flu/ [accessed 2020-06-09]
- Responses to the Government of India's Joint Parliamentary Committee's Consultation on The Personal Data Protection Bill. The Lakshmi Mittal and Family South Asia Institute at Harvard University. 2020 Feb 01. URL: http://mittalsouthasiainstitute.harvard.edu/wp-content/uploads/2020/03/IDHN-Comments-on-PDP-Bill.pdf [accessed 2020-06-09]
- Esteva A, Robicquet A, Ramsundar B, Kuleshov V, DePristo M, Chou K, et al. A guide to deep learning in healthcare. Nat Med 2019 Jan;25(1):24-29. [CrossRef] [Medline]
- Patel BN, Rosenberg L, Willcox G, Baltaxe D, Lyons M, Irvin J, et al. Human-machine partnership with artificial intelligence for chest radiograph diagnosis. NPJ Digit Med 2019;2:111 [FREE Full text] [CrossRef] [Medline]
- Wiens J, Saria S, Sendak M, Ghassemi M, Liu VX, Doshi-Velez F, et al. Do no harm: a roadmap for responsible machine learning for health care. Nat Med 2019 Sep;25(9):1337-1340. [CrossRef] [Medline]
- Obermeyer Z, Powers B, Vogeli C, Mullainathan S. Dissecting racial bias in an algorithm used to manage the health of populations. Science 2019 Oct 25;366(6464):447-453. [CrossRef] [Medline]
- Nalichowski R, Keogh D, Chueh HC, Murphy SN. Calculating the benefits of a Research Patient Data Repository. AMIA Annu Symp Proc 2006:1044 [FREE Full text] [Medline]
- Herland M, Khoshgoftaar TM, Bauder RA. Big Data fraud detection using multiple medicare data sources. J Big Data 2018 Sep 4;5(1):29. [CrossRef]
- Johnson AE, Stone DJ, Celi LA, Pollard TJ. The MIMIC Code Repository: enabling reproducibility in critical care research. J Am Med Inform Assoc 2018 Jan 01;25(1):32-39 [FREE Full text] [CrossRef] [Medline]
- Agarwal S, Lefevre AE, Labrique AB. A Call to Digital Health Practitioners: New Guidelines Can Help Improve the Quality of Digital Health Evidence. JMIR Mhealth Uhealth 2017 Oct 06;5(10):e136 [FREE Full text] [CrossRef] [Medline]
- Khanna T. Learning From Economic Experiments in China and India. AMP 2009 May;23(2):36-43. [CrossRef]
- Morrison W. China's Economic Rise: History, Trends, Challenges, and Implications for the United States. Congressional Research Service. 2014 Aug 21. URL: https://digital.library.unt.edu/ark:/67531/metadc770519/ [accessed 2020-06-09]
- Chen CC. Regulatory Sandboxes in the UK and Singapore: A Preliminary Survey. Rochester, NY: Social Science Research Network; 2019 Sep 14. URL: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3448901 [accessed 2020-06-09]
- PTI. RBI allows fintech companies, financial institutions to set up regulatory sandbox. The Economic Times. 2019 Aug 13. URL: https://economictimes.indiatimes.com/news/economy/finance/rbi-allows-fintech-companies-financial-institutions-to-set-up-regulatory-sandbox/articleshow/70664351.cms [accessed 2020-06-09]
- Prabhakaran D, Jha D, Prieto-Merino D, Roy A, Singh K, Ajay VS, et al. Effectiveness of an mHealth-Based Electronic Decision Support System for Integrated Management of Chronic Conditions in Primary Care: The mWellcare Cluster-Randomized Controlled Trial. Circulation 2018 Nov 10;139(3):380-391. [CrossRef] [Medline]
- Sharma A. We are 100 per cent technology-driven by nature: Dr Indu Bhushan, CEO, Ayushman Bharat. Express Healthcare. 2020 Mar 30. URL: https://www.expresshealthcare.in/public-health/we-are-100-per-cent-technology-driven-by-nature-dr-indu-bhushan-ceo-ayushman-bharat/417831/ [accessed 2020-06-09]
- McKenzie D. A Nobel Prize for Development RCTs!. World Bank Blogs: Development Impact. 2019 Oct 14. URL: https://blogs.worldbank.org/impactevaluations/nobel-prize-development-rcts [accessed 2020-06-09]
|ASHA: accredited social health activist|
|API: application programming interface|
|CMS: Centers for Medicare and Medicaid Services|
|mHealth: mobile health|
|MIMIC-III: Medical Information Mart for Intensive Care III|
|MOOC: massive open online course|
|NDHB: National Digital Health Blueprint|
|NDHM: National Digital Health Mission|
|NHA: National Health Authority|
|RPDR: Research Patient Data Registry|
Edited by G Eysenbach; submitted 09.06.20; peer-reviewed by S Sarbadhikari, F Lami; comments to author 29.06.20; revised version received 23.07.20; accepted 02.08.20; published 15.09.20Copyright
©Abhishek Bhatia, Rahul Matthan, Tarun Khanna, Satchit Balsari. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 15.09.2020.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.