Published on 22.10.18 in Vol 20, No 10 (2018): October
Preprints (earlier versions) of this paper are available at http://preprints.jmir.org/preprint/10089, first published Feb 09, 2018.
Letter to the Editor
Blood Pressure Measurement Validation Off the Cuff? Comment on “A New Cuffless Device for Measuring Blood Pressure: A Real-Life Validation Study”
J Med Internet Res 2018;20(10):e10089
We read with interest the validation study of the commercially available CheckMe blood pressure (BP) monitor by Schoot et al . We agree with the authors that the development of cuffless BP devices may improve rates of out-of-office BP measurement for adults with hypertension and that the value of such devices depends on the devices’ accuracy. We feel there are aspects of their report where clarification might be helpful.
One of the reported strengths of the study is that it met European Society of Hypertension International Protocol (ESH-IP) validation guidelines . However, some aspects of the study deviated from the ESH-IP, including obtaining of measurements in varying anatomical positions. Usually, validation is performed in the manufacturer’s recommended position of measurement, which is typically in the same position as calibration [ ]. Additional information would be helpful to better understand the performance characteristics from the device itself and to assess for bias introduced from protocol deviations.
First, presenting correlation coefficients and scatterplots can help readers assess the relationship between measurements from the devices. Next, the authors present a relative difference in mean systolic BPs, which is near zero. Presenting mean absolute difference between devices would be more informative to the accuracy of the device . A modified version of the ESH-IP’s validation table is presented not in the usual numerical categories. Calculating these categories, the CheckMe deviates from the reference device by ≤5 mmHg, ≤10 mmHg, and ≤15 mmHg for 16%, 62%, and 86% of the measures, respectively. These performance characteristics do not meet the threshold of passing ESH-IP’s first part of validation. Finally, contrary to the description in the manuscript, we note that the reference device used has not undergone independent validation and its accuracy is not known [ ]. Thus, it is difficult to interpret the accuracy of CheckMe without understanding the performance characteristics of the reference device.
We recognize that it is not possible to adhere to the ESH-IP with novel cuffless BP-measuring devices, since the ESH-IP assumes that the tested monitor does not require user-specific calibration. In response to the ongoing interest in cuffless BP measurement the Institute of Electrical and Electronics Engineers (IEEE) released a guideline specifically for the validation of these devices . Because validation of a cuffless BP monitor immediately after calibration at the same BP could artificially increase its perceived accuracy [ ], the IEEE protocol includes validation measurements after changes in BP from the calibration measurement and validation measurements after a significant period since calibration.
In conclusion, we encourage the authors to present additional analyses and results to improve the understanding of the CheckMe’s performance, and to follow up with a formal IEEE-protocol validation study. In the present study, the device is reported to be a Conformité Européenne-approved category IIa category medical device compliant with directive 93/42/European Economic Community, but it did not meet that directive’s accuracy requirements (EN 1060-3). The CheckMe has United States Food & Drug Administration approval for measurements of heart rate, oxygen saturation, temperature, and activity, but not for measurement of systolic blood pressure . As the device does not meet ESH-IP accuracy criteria for blood pressure in this study, we are concerned that it is available for sale directly to consumers and may place adults at undue harm.
Conflicts of Interest
- Schoot TS, Weenk M, van de Belt TH, Engelen LJLPG, van Goor H, Bredie SJH. A New Cuffless Device for Measuring Blood Pressure: A Real-Life Validation Study. J Med Internet Res 2016 May 05;18(5):e85 [FREE Full text] [CrossRef] [Medline]
- O'Brien E, Atkins N, Stergiou G, Karpettas N, Parati G, Asmar R, Working Group on Blood Pressure Monitoring of the European Society of Hypertension. European Society of Hypertension International Protocol revision 2010 for the validation of blood pressure measuring devices in adults. Blood Press Monit 2010 Feb;15(1):23-38. [CrossRef] [Medline]
- Boubouchairopoulou N, Kollias A, Chiu B, Chen B, Lagou S, Anestis P, et al. A novel cuffless device for self-measurement of blood pressure: concept, performance and clinical validation. J Hum Hypertens 2017 Jul;31(7):479-482. [CrossRef] [Medline]
- Medaval. 2017. Welch Allyn Vital Signs 300 Series URL: http://medaval.ie/device/welch-allyn-vital-signs-300-series/ [accessed 2018-02-03] [WebCite Cache]
- Institute of Electrical and Electronics Engineers. 2014 Aug 26. IEEE 1708-2014 Standard for Wearable Cuffless Blood Pressure Measuring Devices URL: https://standards.ieee.org/standard/1708-2014.html [accessed 2018-09-26] [WebCite Cache]
- U.S. Department of Health and Human Services Food and Drug Administration, Checkme Pro Health Monitor. 2015 Dec 10. URL: https://www.accessdata.fda.gov/cdrh_docs/pdf15/K150869.pdf [accessed 2018-02-08] [WebCite Cache]
Edited by G Eysenbach; submitted 09.02.18; peer-reviewed by J Goris; comments to author 26.09.18; revised version received 27.09.18; accepted 10.10.18; published 22.10.18
©Noud van Helmond, Timothy B. Plante, Jeffrey I. Joseph. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 22.10.2018.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.