This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.
Digital health has become an advancing phenomenon in the health care systems of modern societies. Over the past two decades, various digital health options, technologies, and innovations have been introduced; many of them are still being investigated and evaluated by researchers all around the globe. However, the actual trends and visibility of peer-reviewed publications using “digital health” as a keyword to reflect the topic, published by major relevant journals, still remain to be quantified.
This study aimed to conduct a bibliographic-bibliometric analysis on articles published in JMIR Publications journals that used “digital health” as a keyword. We evaluated the trends, topics, and citations of these research publications to identify the important share and contribution of JMIR Publications journals in publishing articles on digital health.
All JMIR Publications journals were searched to find articles in English, published between January 2000 and August 2019, in which the authors focused on, utilized, or discussed digital health in their study and used “digital health” as a keyword. In addition, a bibliographic-bibliometric analysis was conducted using the freely available Profiles Research Networking Software by the Harvard Clinical and Translational Science Center.
Out of 1797 articles having “digital health” as a keyword, published mostly between 2016 and 2019, 277 articles (32.3%) were published by JMIR Publications journals, mainly in the
The reflection of “digital health” as a keyword in JMIR Publications journals has increased noticeably over the past few years. To maintain this momentum, more regular bibliographic and bibliometric analyses will be needed. This would encourage authors to consider publishing their articles in relevant, high-visibility journals and help these journals expand their supportive publication policies and become more inclusive of digital health.
Digital health has become an advancing phenomenon in the health care systems of modern societies [
Globally, many academics and researchers are increasingly being involved in doing research on, utilizing, evaluating, or taking advantage of the benefits of digital health and its various related technologies for their studies on individuals, populations, or health organizations. This increasing involvement has reflected itself in the utilization of “digital health” as a keyword in published peer-reviewed literature. More specifically, in the past two decades, a growing number and diversity of research projects, study protocols, publications, and dedicated journals have played important roles in the digital health domain [
Over the past two decades, various digital health options, technologies, and innovations have been introduced; many of them are still being investigated and evaluated by researchers all around the globe [
This study aimed to take a more methodical approach to answering this question, by conducting a bibliographic-bibliometric analysis on the publications focused on using “digital health” as a keyword. We evaluated the trends, topics, and citations of research publications in different journals, with the hope to identify, and ultimately help to increase, the share and contribution of major relevant journals in publishing articles on digital health. Thereafter, for the purpose of providing an unbiased comparison among different journals on the trends and visibility of their publications, we conducted detailed subgroup analyses, individualized to specialized journals or journal publishers. This paper summarizes the specific outcomes of our analyses on articles published by JMIR Publications. The main reasons behind focusing on JMIR Publications in this study are the following: (1) JMIR Publications has been an active publisher in the digital health space since 1999, which overlaps entirely with the intended time frame of our study; (2) it has a collection of correlated journals, which covers diverse aspects of digital health research; and (3) it publishes open-access articles, which gives the authors more chances of visibility and knowledge translation and the readers more chances of verifying the results of all analyses.
On the basis of expert opinions, “digital health” is considered a relatively new term in research publications, as its appearance as a keyword seems to have increased fairly recently in peer-reviewed articles. Before this trend becomes commonplace, keywords such as “Internet research,” “cybermedicine,” “eHealth,” or “mHealth” have been (and are still being) used by authors and editorial boards of various scientific journals, including journals by JMIR Publications.
To address this recency in the adoption of “digital health” as a more common term, we followed a staged, multistep literature search strategy, implemented separately for each journal or journal group or publisher, to ensure that using “digital health” as an identifying keyword does not harm the inclusiveness of numerous options, technologies, and innovations in this space. An effort was made to find the sensitivity of using “digital health” as a keyword in identifying articles that could have otherwise been classified differently under internet search, cybermedicine, mHealth, or similar keywords had “digital health” not been assigned as a keyword by the authors or the databases.
The time frame of search was January 2000 to August 2019.
Owing to its open-access nature, we decided to use PubMed database to identify general and specialized journals and find articles published in English language, in which the focus was on using “digital health” as a keyword.
The initial, implicit assumption was that if “digital health” has been mentioned by the authors as a keyword in an article or assigned by the database organizer, for example, as Medical Subject Heading (MeSH)–assigned keyword, the topic of the article will be related to digital health. However, as mentioned above, to reduce the bias in finding relevant articles because of the recency of “digital health” being used as a term, we followed a staged search strategy, which is summarized below.
This stage involved finding all articles with “digital health” in their metadata: (1) Searching with only the keyword “digital health” in All Fields to identify all articles in PubMed, which could have the term in their metadata and (2) importing the results to a library in a bibliographic management software.
This stage involved identifying keywords/topics/subjects relevant to digital health: (1) Performing a subject bibliography analysis by extracting all author-assigned plus MeSH-assigned keywords, sorted according to their decreasing frequencies of appearance and (2) identifying and refining keywords/topics/subjects relevant to the definition of “digital health,” as provided by Murray et al [
This stage involved finding all articles that had used any of the keywords identified in the previous stage: Searching PubMed, using OR between all the keywords from
This stage involved finding all articles published by JMIR Publications: Searching with only the keyword “JMIR” in All Fields to identify all articles in PubMed, which were published by JMIR Publications.
This stage involved combining stage 3 AND stage 4: Searching PubMed, using OR between all the keywords from
This stage involved comparing the results of stage 5 and stage 4: Determining the difference between the number of articles retrieved in stage 4 and stage 5 to check the inclusiveness of our terms list.
This stage involved combining stage 1 and stage 4: (1) After ensuring the sensitivity of our search strategy, on the basis of the outcome of stage 6, we searched with the keyword “digital health” in All Fields AND the keyword “JMIR” in All Fields to identify all articles by JMIR Publications
A flowchart summarizing the outputs of this staged literature search is available in
For bibliographic management and analysis of the references, we used EndNote X8 (Thompson Reuters Inc) software, mainly its “Subject Bibliography” functionality.
For bibliometric analysis to quantify the trends and visibility of published articles using “digital health” as a keyword, we used one of the free, publicly available Web-based solutions, that is, the Profiles Research Networking (PRN) Software by the Harvard Clinical and Translational Science Center [
Bibliometric parameters as provided in the output by the Profiles Research Networking Software in its Bibliometric Summary Report.
Variable | Definition |
Num Pubs | Number of recognized PubMed IDs, overall, for each journal, or for each year, as specified in the report subsections |
First Year | Earliest article year |
Last Year | Latest article year |
Avg Authors | Average number of authors per article |
Exp Authors | Expected number of authors, matched on journal and year |
Ratio Authors | Ratio of the average number of authors to the expected number |
Avg Cites All | Average number of times an article has been cited, including self-citations |
Avg Cites | Average number of times an article has been cited, not including self-citations |
Exp Cites | Expected number of times an article has been cited, not including self-citations, matched on journal and year |
Ratio Cites | Ratio of average number of citations (no self-citations) to expected number, matched on journal and year |
Exp Cites PT | Expected number of citations (no self-citations), matched on journal, year, and publication type |
Ratio Cites PT | Ratio of average number of citations (no self-citations) to expected number, matched on journal, year, and publication type |
H-Index | Hirsch-index (using total citations, including self-citations) |
M-Index | Hirsch-index divided by the number of years since the first publication |
%Pubs | The percentage of the total publications for each journal |
Ratio Exp Pubs | The ratio of the number of publications in the field compared with the expected number, matched on year |
Num Cites All | For each year, the number of times any article was cited, including self-citations, in that year |
Num Cites | For each year, the number of times any article was cited, not including self-citations, in that year |
Cum Pubs | For each year, the cumulative number of publications |
Cum Cites All | For each year, the cumulative number of times any article was cited, including self-citations |
Cum Cites | For each year, the cumulative number of times any article was cited, not including self-citations |
Overall, with August 31, 2019 as the last publication date, we found 1797 articles indexed in PubMed, with “digital health” being assigned as one of the keywords in their metadata.
Exporting the keywords from 1797 articles provided a list of 5138 author-assigned and MeSH-assigned keywords, out of which 312 keywords were directly relevant to “digital health” options, technologies, and innovations (
In the same time frame
Out of the 1797 articles, 277 articles had both characteristics of (1) being published by JMIR Publications and (2) having “digital health” as an assigned keyword. The rest of the bibliographic-bibliometric analysis was performed on these 277 articles.
Temporal trend of the number of publications from January 2000 to August 2019 by JMIR Publications, having “digital health” as a keyword.
Using EndNote’s Subject Bibliography, a total of 1101 MeSH- and author-assigned keywords were extracted for assessing the topics of articles.
All articles were classified under “medical informatics” as the most frequent field/discipline of focus.
Cumulative number of appearances for the top 30 keywords, in descending order of appearance, in research articles having “digital health” as a keyword, published from January 2000 to August 2019 in JMIR Publications journals.
Rank | Keyword | Number of appearances |
1 | mhealth | 60 |
2 | Telemedicine | 57 |
3 | Internet | 42 |
4 | eHealth | 37 |
5 | mobile health | 36 |
6 | self-management | 18 |
7 | mobile phone | 15 |
8 | depression | 15 |
9 | physical activity | 14 |
10 | smartphone | 14 |
11 | Mobile Applications | 13 |
12 | Chronic Disease | 13 |
13 | Social Support | 12 |
14 | electronic health records | 12 |
15 | psychology | 12 |
16 | Health Behavior | 11 |
17 | medication adherence | 11 |
18 | exercise | 10 |
19 | Social Media | 10 |
20 | text messaging | 9 |
21 | obesity | 9 |
22 | education | 9 |
23 | mental health | 9 |
24 | Health Promotion | 8 |
25 | mobile apps | 8 |
26 | diabetes | 8 |
27 | Diabetes Mellitus | 8 |
28 | telehealth | 7 |
29 | Cell Phone | 7 |
30 | Health Personnel | 7 |
Bibliometric summary statistics for all articles published between January 2000 and August 2019 by JMIR Publications having “digital health” as a keyword.
Variable | Value |
Num Pubsa | 277 |
First Yearb | 2001 |
Last Yearc | 2019 |
Avg Authorsd | 6.007 |
Exp Authorse | 6.212 |
Ratio Authorsf | 0.967 |
Avg Cites Allg | 2.848 |
Avg Citesh | 2.354 |
Exp Citesi | 1.451 |
Ratio Citesj | 1.623 |
Exp Cites PTk | 1.688 |
Ratio Cites PTl | 1.394 |
H–Indexm | 12 |
M–Indexn | 1.091 |
aNum Pubs: number of recognized PubMed IDs, overall, for each journal, or for each year, as specified in the report subsections.
bFirst Year: earliest article year.
cLast Year: latest article year.
dAvg Authors: average number of authors per article.
eExp Authors: expected number of authors, matched on journal and year.
fRatio Authors: ratio of the average number of authors to the expected number.
gAvg Cites All: average number of times an article has been cited, including self-citations.
hAvg Cites: average number of times an article has been cited, not including self-citations.
iExp Cites: expected number of times an article has been cited, not including self-citations, matched on journal and year.
jRatio Cites: Ratio of average number of citations (no self-citations) to expected number, matched on journal and year.
kExp Cites PT: Expected number of citations (no self-citations), matched on journal, year, and publication type.
lRatio Cites PT: Ratio of average number of citations (no self-citations) to expected number, matched on journal, year, and publication type.
mH-Index: Hirsch-index (using total citations, including self-citations).
nM–Index: Hirsch-index divided by the number of years since the first publication.
Bibliometric statistics for all articles published between January 2000 and August 2019 by JMIR Publications having “digital health” as a keyword.
Journal | Num Pubsa (%Pubs)b, n (%) | First Yearc | Last Yeard | Avg Citese | Exp Citesf | Ratio Citesg | Exp Cites PTh | Ratio Cites PTi |
|
117 (42.2) | 2001 | 2019 | 3.79 | 2.02 | 1.88 | 2.34 | 1.62 |
|
57 (20.6) | 2014 | 2019 | 1.11 | 1.24 | 0.90 | 1.23 | 0.90 |
|
41 (14.8) | 2014 | 2019 | 1.12 | 0.65 | 1.74 | 0.65 | 1.72 |
|
12 (4.3) | 2016 | 2019 | 1.83 | 1.69 | 1.09 | 1.49 | 1.24 |
|
10 (3.6) | 2015 | 2019 | 0.50 | 0.62 | 0.81 | 0.59 | 0.86 |
|
8 (2.9) | 2017 | 2019 | 0.00 | 0.04 | 0.00 | 0.03 | 0.00 |
|
7 (2.5) | 2016 | 2019 | 0.29 | 0.77 | 0.37 | 0.79 | 0.36 |
|
7 (2.5) | 2017 | 2019 | 0.00 | 0.00 | 1.00 | 0.00 | 1.00 |
|
5 (1.8) | 2017 | 2019 | 0.40 | 0.43 | 0.94 | 0.43 | 0.93 |
|
4 (1.4) | 2013 | 2018 | 16.75 | 7.87 | 2.13 | 15.66 | 1.07 |
aNum Pubs: Number of recognized PubMed IDs, overall, for each journal, or for each year, as specified in the report subsections.
b%Pubs: The percentage of the total publications for each journal
cFirst Year: Earliest article year.
dLast Year: Latest article year.
eAvg Cites: Average number of times an article has been cited, not including self-citations.
fExp Cites: Expected number of times an article has been cited, not including self-citations, matched on journal and year.
gRatio Cites: Ratio of average number of citations (no self-citations) to expected number, matched on journal and year.
hExp Cites PT: Expected number of citations (no self-citations), matched on journal, year, and publication type.
iRatio Cites PT: Ratio of average number of citations (no self-citations) to expected number, matched on journal, year, and publication type.
Cumulative citation statistics for all articles published between January 2000 and August 2019 by JMIR Publications having “digital health” as a keyword, by year.
PubYeara | Num Pubsb | Num Cites Allc | Num Citesd | Cum Pubse | Cum Cites Allf | Cum Citesg |
2018 | 83 | 309 | 253 | 171 | 789 | 652 |
2017 | 43 | 249 | 197 | 88 | 480 | 399 |
2016 | 22 | 108 | 94 | 45 | 231 | 202 |
2015 | 14 | 55 | 42 | 23 | 123 | 108 |
2014 | 3 | 32 | 31 | 9 | 68 | 66 |
2013 | 2 | 19 | 19 | 6 | 36 | 35 |
2012 | 2 | 8 | 7 | 4 | 17 | 16 |
2011 | 0 | 6 | 6 | 2 | 9 | 9 |
2010 | 1 | 1 | 1 | 2 | 3 | 3 |
2009 | 0 | 1 | 1 | 1 | 2 | 2 |
2008 | 0 | 0 | 0 | 0 | 0 | 0 |
2007 | 0 | 0 | 0 | 0 | 0 | 0 |
2006 | 0 | 0 | 0 | 0 | 0 | 0 |
2005 | 0 | 0 | 0 | 0 | 0 | 0 |
2004 | 0 | 1 | 1 | 1 | 1 | 1 |
2003 | 0 | 0 | 0 | 0 | 0 | 0 |
2001 | 1 | 0 | 0 | 1 | 0 | 0 |
2000 | 0 | 0 | 0 | 0 | 0 | 0 |
aAuthors excluded 2019 from this table as the cumulative citations might be incomplete because of the study time frame being up to August 2019.
bNum Pubs: Number of recognized PubMed IDs, overall, for each journal, or for each year, as specified in the report subsections.
cNum Cites All: For each year, the number of times any article was cited, including self-citations, in that year.
dNum Cites: For each year, the number of times any article was cited, not including self-citations, in that year.
eCum Pubs: For each year, the cumulative number of publications.
fCum Cites All: For each year, the cumulative number of times any article was cited, including self-citations.
gCum Cites: For each year, the cumulative number of times any article was cited, not including self-citations.
Both trends and visibility of research publications containing “digital health” in their keywords and published by JMIR Publications journals increased dramatically, especially over the past 2 to 3 years, with more than two-third of the articles being published in 2018 and 2019. This important finding shows how “digital health” is becoming a mainstream theme and an established terminology in peer-reviewed publications.
The
Interestingly, “mHealth” and “mobile health” as specific keywords, appeared in 96 out of 277 articles (34.6%), followed by “Telemedicine” and “Internet,” both appearing in 57 (20.5%) and 42 (15.2%) articles, respectively. In addition, there appeared to be cumulatively repetitive or redundant keywords, either author-assigned or MeSH keywords (eg, “mobile phone,” “smartphone,” “Cell Phone,” “Mobile Applications,” and “mobile apps”), all appearing with different frequencies in collective articles. We decided to present these keywords as raw as possible in
The dramatic increase in the cumulative number of citations over the study years is a helpful indicator of the overall interest in referring to the articles pertaining to the keyword “digital health.” Moreover, an H-index of 12, plus an average number of citations of all articles being >1.6 times more than the expected number of citations, highlights the increasing interest in referring to articles on digital health.
Expectedly, “medical informatics” was found to be the most frequent field/discipline of focus in research publications having “digital health” as a keyword. This finding, in addition to considering the
The
We followed the hints provided by the PRN Software team [
The PRN Software compares the average number of authors per article and the average number of times the articles have been cited with an expected value, which is “the averages of all articles in PubMed, matched on journal and year of publication” [
In addition, if self-citations are included in the analysis subsections, they are explicitly being noted.
To determine the field/discipline of a specific journal, the NLM assigns Broad Journal Heading values to the journal, which are MeSH terms, summarizing the overall subjects of that journal. Similar to PT, a journal can be assigned to multiple Broad Journal Headings; consequently, a single publication of that journal might be listed more than once in the output tables about filed/discipline, causing the Num Pubs field to add up to more than the total number of publications. This was not the case in our analysis as all the journals by JMIR Publications were classified under Medical Informatics by the NLM.
Our study focused on English language–based journals that were indexed in PubMed as a freely available database and published by JMIR Publications. PubMed is not essentially a citation-tracking database. However, solutions such as the bibliometric solution that we used in our methodology, that is, the PRN Software by the Harvard Clinical and Translational Science Center, have been developed, which provide bibliometric outputs on PubMed-indexed articles. Other citation-based databases, specifically subscription-based bibliometric databases, such as Scopus and Web of Science, could be included in future research projects to expand the scope of this analysis.
Another main reason behind focusing only on PubMed, apart from being freely available to the public, was that PRN Software only accepts PubMed IDs for citation analysis. This held us back from using other bibliographic databases as they could not have any PubMed ID for non-PubMed-indexed journals.
In addition, the citation metrics by PRN Software were coming from one publicly available free data source and were limited to commonly used parameters. For the provision of a comprehensive bibliometric outlook on publications by JMIR Publications having the keyword “digital health,” other citation databases and metrics could also be utilized in future studies.
The reflection of “digital health” in JMIR Publications journals has been on the rise over the past few years. More comprehensive and comparative bibliographic and bibliometric analyses, with broader ranges of keywords to include eHealth, mHealth, and similar concepts, would be needed to visualize whether “digital health” continues to remain a rising keyword in the future or not.
List of refined keywords relevant to digital health, to reduce bias in the search strategy.
Search flowchart.
higher degree research
Medical Subject Headings
National Library of Medicine
Profiles Research Networking
publication type
Queensland University of Technology
This study was conducted as part of a higher degree research (HDR) project at School of Clinical Sciences, Faculty of Health, Queensland University of Technology (QUT), Brisbane, Queensland, Australia. The HDR project was supported by internal funding from QUT for research expenses, as well as 3 scholarships for AA. The authors wish to deeply thank Mr Paul Sonnier for his professional input on the evolving definition of “digital health” as a keyword and for introducing evidence and references, which helped in structuring the search strategy of the methodology of this manuscript. The authors received no funding for the development of this manuscript. The authors would like to cite the service which was made possible by the PRN Software developed under the supervision of Griffin M Weber, MD, PhD, with support from Grant Number 1 UL1 RR025758-01 to Harvard Catalyst: The Harvard Clinical and Translational Science Center from the National Center for Research Resources and support from Harvard University and its affiliated academic healthcare centers. Open Research Networking Gadgets and Search Engine Optimization additions to Profiles Research Networking Software have been made possible by The University of California, San Francisco's Clinical and Translational Science Institute, funded through the National Center for Advancing Translational Sciences (Grant Number UL1 TR000004) at the National Institutes of Health.
None declared.