Background

JMIR

J Med Internet Res

Journal of Medical Internet Research

1438-8871

JMIR Publications

Toronto, Canada

v26i1e62769

39388258

10.2196/62769

Review

Impact of Skin Pigmentation on Pulse Oximetry Blood Oxygenation and Wearable Pulse Rate Accuracy: Systematic Review and Meta-Analysis

Eysenbach

Gunther

Mavragani

Amaryllis

Sinha

Urjoshi

Nagamine

Takahiko

Singh

Sanidhya

BSE 1

University of Michigan Medical School

1301 Catherine Street

Ann Arbor, MI, 48109

United States 1 7349364000 sansin@med.umich.edu

https://orcid.org/0009-0004-0132-9104

Bennett

Miles Romney

MS 2

https://orcid.org/0009-0005-2087-8835

Chen

PhD 2

https://orcid.org/0000-0003-1175-3027

Shin

Sooyoon

PhD 2

https://orcid.org/0000-0002-0339-4856

Ghanbari

Hamid

MD, MPH 1

https://orcid.org/0009-0004-7826-4740

Nelson

Benjamin W

PhD 2 3

https://orcid.org/0000-0002-5474-7674

1 University of Michigan Medical School

Ann Arbor, MI

United States 2 Verily Life Sciences LLC

South San Francisco, CA

United States 3 Division of Digital Psychiatry Department of Psychiatry Beth Israel Deaconess Medical Center, Harvard Medical School

Boston, MA

United States

Corresponding Author: Sanidhya Singh sansin@med.umich.edu

2024

10 10 2024

e62769

4 6 2024 4 7 2024 25 7 2024 16 8 2024

©Sanidhya Singh, Miles Romney Bennett, Chen Chen, Sooyoon Shin, Hamid Ghanbari, Benjamin W Nelson. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 10.10.2024.

2024

This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research (ISSN 1438-8871), is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.

Background

Photoplethysmography (PPG) is a technology routinely used in clinical practice to assess blood oxygenation (SpO₂) and pulse rate (PR). Skin pigmentation may influence accuracy, leading to health outcomes disparities.

Objective

This systematic review and meta-analysis primarily aimed to evaluate the accuracy of PPG-derived SpO₂ and PR by skin pigmentation. Secondarily, we aimed to evaluate statistical biases and the clinical relevance of PPG-derived SpO₂ and PR according to skin pigmentation.

Methods

We identified 23 pulse oximetry studies (n=59,684; 197,353 paired SpO₂-arterial blood observations) and 4 wearable PR studies (n=176; 140,771 paired PPG-electrocardiography observations). We evaluated accuracy according to skin pigmentation group by comparing SpO₂ accuracy root-mean-square values to the regulatory threshold of 3% and PR 95% limits of agreement values to +5 or –5 beats per minute (bpm), according to the standards of the American National Standards Institute, Association for the Advancement of Medical Instrumentation, and the International Electrotechnical Commission. We evaluated biases and clinical relevance using mean bias and 95% CI.

Results

For SpO₂, accuracy root-mean-square values were 3.96%, 4.71%, and 4.15%, and pooled mean biases were 0.70% (95% CI 0.17%-1.22%), 0.27% (95% CI –0.64% to 1.19%), and 1.27% (95% CI 0.58%-1.95%) for light, medium, and dark pigmentation, respectively. For PR, 95% limits of agreement values were from –16.02 to 13.54, from –18.62 to 16.84, and from –33.69 to 32.54, and pooled mean biases were –1.24 (95% CI –5.31 to 2.83) bpm, –0.89 (95% CI –3.70 to 1.93) bpm, and –0.57 (95% CI –9.44 to 8.29) bpm for light, medium, and dark pigmentation, respectively.

Conclusions

SpO₂ and PR measurements may be inaccurate across all skin pigmentation groups, breaching U.S. Food and Drug Administration guidance and industry standard thresholds. Pulse oximeters significantly overestimate SpO₂ for both light and dark skin pigmentation, but this overestimation may not be clinically relevant. PRs obtained from wearables exhibit no statistically or clinically significant bias based on skin pigmentation.

photoplethysmography pulse oximetry arterial blood gas skin tone skin pigmentation bias digital technology

Introduction

Photoplethysmography (PPG) technology has been used in medicine since the 1970s to assess pulse rate (PR) and blood oxygenation (SpO₂). The accuracy of PPG-based SpO₂ and PR is critical for medical practice, clinical decision-making, and patient outcomes [1].

Technological advancements have led to rapid expansion of this technology into consumer devices [2]. This has enabled consumers to continuously and unobtrusively track health status, generating information that is becoming commonly used in health care settings and as a source of research end points [3,4].

Researchers have identified several factors that influence the accuracy of PPG-based measurements, including skin temperature, sensor contact pressure, skin thickness, and hydration level [5,6]. Notably, darker skin pigmentation may influence PPG-based SpO₂ and PR readings [7-10]. Patients with darker skin pigmentation are more likely than patients with lighter skin color to have overestimated SpO₂ readings, leading to lower hospital admission; higher occult hypoxemia; and delayed or no access to dexamethasone, therapeutic oxygen, and COVID-19 therapies, resulting in increased hospital readmission, organ dysfunction, and mortality [11-18].

Starting in 2013, a series of U.S. Food and Drug Administration (FDA) guidances, safety communications, and more recently, statements from attorneys general have called to address darker skin pigmentation bias in pulse oximeters [19]; however, these guidances lacked standards for assessing skin tone [20]. Recent pulse oximeter research has rekindled interest in skin pigmentation disparities in PPG sensor accuracy, leading to increased media and regulatory attention [21-24]. In November 2023, more than 24 attorneys general wrote a letter calling on the FDA to take urgent action to address pulse oximeter skin pigmentation disparities [25,26].

More recent research has also highlighted the potential presence of these biases in research-grade and consumer PPG-based wearable devices [4,27-29], leading to calls to address inequity, bias, and discrimination in wearable health technology and clinical practice algorithms [30,31].

Accuracy guidance for pulse oximeters and heart rate or PR devices has been delineated by regulatory bodies, industry, and medical standards. Overall accuracy of pulse oximeters can be assessed with accuracy root-mean-square (A_rms), which combines mean bias and precision (SD of bias) into a single metric [32]. FDA guidance has a threshold of A_rms≤3% [19] for transmittance devices and a threshold of A_rms≤3.5% [19] for ear clip and reflectance devices, while international thresholds are set at A_rms≤4% [32]. For heart rate or PR devices, overall accuracy can be assessed with 95% limits of agreement (LoAs). The American National Standards Institute (ANSI), Association for the Advancement of Medical Instrumentation (AAMI), and International Electrotechnical Commission (IEC) have set the recommendation that electrocardiography (ECG) devices have mean bias of +5 or –5 beats per minute (bpm) or mean absolute percentage error≤10%, whichever is greater [33,34]. Devices with accuracy measures breaching these accuracy thresholds can produce questionable results.

It is, therefore, critical to generate evidence to inform the design and calibration of these devices to reduce algorithmic bias and improve accuracy, so that PPG-based technology can generalize to all segments of the population, mitigating racial disparities in health outcomes. There have been systematic reviews on PPG skin pigmentation bias, but these have been limited to either consumer device PR [35] or pulse oximeter SpO₂ [36-38]. Only 1 meta-analysis focused on pulse oximetry for this topic is available [36], and an additional 7 studies have been published recently, providing an additional 50,980 participants and 182,369 paired observations. Therefore, performing a comprehensive meta-analysis to examine PPG accuracy, potential bias, and clinical relevance for SpO₂ and PR by skin pigmentation is timely.

Methods Search Strategy and Selection Criteria

We followed the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines [39] and developed a MEDLINE systematic search. Searches were performed between April 2022 and June 2023, and a final ad hoc search was conducted in June 2023. No additional studies were included after this date.

We included studies that (1) investigated PPG-derived SpO₂ or PR test devices per definition [40]; (2) used arterial blood gas (SaO₂) or ECG as the reference device; and (3) reported mean bias and SD, SE, 95% LoA, or 95% CI by race, ethnicity, or skin tone. Inclusion disagreements were resolved by consensus.

Exclusion criteria included (1) literature reviews, systematic reviews, commentary, and meta-analyses; (2) non-English manuscripts; (3) irretrievable full source texts; (4) studies on remote PPG; and (5) those that lack SaO₂ or ECG as the reference device. We chose to not exclude papers based on measurement hardware or underlying algorithms given that all measurement devices relied on contact-based PPG to measure the same end points of interest—SpO₂ and PR.

Data Analysis Data Extraction

Data were independently extracted from published manuscripts (BWN and SS). A third and fourth reviewer (MRB and HG) adjudicated any differences between the initial 2 reviewers and resolved any disagreements by checking manuscripts. The first author verified the quality of manuscripts. Standardized data extraction was developed to extract the study characteristics (see the Materials—Data Extraction section in Multimedia Appendix 1). For 1 paper, we used participant-level data from the supplementary materials to calculate mean bias and SE by skin pigmentation category [41]. This resulted in 27 studies in our final analysis (Figure 1 and Table 1).

Figure 1

PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) diagram. ECG: electrocardiography; PPG: photoplethysmography; SpO₂: blood oxygenation.

Table 1

Characteristics of included studies.

Device type and study (author, year)			Sample size, n		Test device evaluated		Reference device		SaO₂^a (%) range		Participant population		Research setting		Skin pigmentation method
Pulse Oximetry
	Abrams et al [42], 2003	200		Nellcor N-200		Radiometer ABL-520		Not reported		Medical—adult patients with cirrhosis		Inpatient		Race—Black and White
	Adler et al [43], 1998	298		Nellcor D-25		4-wavelength spectro-photometer, or co-oximeter (Radiometer OSM3)		50-99		Medical—adult emergency department patients		Inpatient		Skin tone—Munsell color system categorized into light, medium, or dark
	Andrist et al [44], 2022	1061		Not reported		Not reported		Not reported		Medical—pediatric		Inpatient		Race
	Barker and Wilson [45], 2023	75		Masimo SET^b pulse oximeters with RD-SET sensors		Radiometer ABL-835 Flex CO-Oximeter		70-100		Mixed—healthy and mild systemic disease		Research — laboratory		Race
	Bickler et al [10], 2005	23		Nellcor N-595 with Nellcor OxiMax A finger probe Novametrix 513s models (2 types) Nonin Onyx models (2 types)		Radiometer OSM3		60-100		Healthy—nonsmoking		Research — laboratory		Ethnicity—light (Northern European) and dark (African American)
	Bothma et al [46], 1996	100		Simed S100e Nihon Koden Ohmeda 3740		IL482 Co-oximeter System		87.8-99.2		Medical—critically ill adult patients		Inpatient		Skin tone—EEL^c reflectance spectrophotometer; all participants had dark skin tone
	Burnett et al [47], 2022	46,253		Unspecified Nellcor and Masimo devices		GEMStat Premier 3000		Not reported		Medical—patients receiving anesthesia		Inpatient		Race and ethnicity
	Crooks et al [48], 2022	2997		Not reported		Not reported		Not reported		Medical—COVID-19		Inpatient		Race
	Ebmeier et al [49], 2018	394		Marquette Rac-4A monitors with Masimo sensors and Philips IntelliVue MP70 monitors with Philips Adult Reusable SpO₂^d sensors		Radiometer ABL 800 FLEX SaO₂ analyzer		Not reported		Medical—multiple conditions		Inpatient		Ethnicity
	Fawzy et al [17], 2022	1216		Not reported		ABL825, ABL827, or ABL90 blood gas analyzers		Not reported		Medical—COVID-19		Inpatient		Race and ethnicity
	Feiner et al [8], 2007	36		Nellcor N-595 (OxiMax A adhesive probe) Nellcor N-595 (a clip-type probe) Masimo Radical (clip probe) Masimo Radical (adhesive disposable probe) Nonin 9700 (clip-type probe) Nonin 9700 (disposable adhesive probe)		Radiometer OSM3 multiwavelength oximeter		60-100		Healthy—nonsmoking		Research — laboratory		Ethnicity—light (Caucasian), intermediate (Hispanic, Indian, Filipino, or Vietnamese), and dark (African American) categories
	Foglia et al [50], 2017	36		Nellcor Oximax (Covidien) Masimo Rainbow SET Radical 7		Siemens Rapidlab 1265		60-92		Medical—infants with cyanotic congenital heart disease and oxygen saturation <90%		Inpatient		Skin tone—Munsell color system
	Jubran and Tobin [51], 1990	54		Nellcor pulse oximeter with disposable or reusable probes Ohmeda-Biox3700 pulse oximeter with reusable probe		CO-oximetry		Not reported		Medical—critically ill, ventilator-dependent patients		Inpatient		Race—Black and White
	McGovern et al [52], 1996	8		Ohmeda 3700		IL 482 Co-oximeter		Not reported		Medical—adults with stable condition with severe COPD^f		Research — laboratory		Race—all White
	Muñoz et al [53], 2008	846		Minolta Pulsox-7		IL 682 co-oximeter		Not reported		Medical—adults under assessment for long-term home oxygen therapy		Outpatient		Race—all White
	Pilcher et al [54], 2020	400		Carescape B450 monitor with Nellcor probe GE Dash 3000 Masimo Radical 7 Masimo SET Quartz (unspecified) Masimo SET Quartz Q400 Nonin 2120 Nonin 2140 Nonin Avant (unspecified) Nonin Avant 4000 Nonin Avant 9700 Nonin Lifesense Medair Novametrix Model 512 Ohmeda Biox 3700E with a GE TruSignal or Nellcor probe Philips Intellivue MP70 with a GE TruSignal Nellcor or Philips probe Welch Allyn with a Nellcor probe		Radiometer ABL800		72-100		Medical—hospitalized adult patients		Inpatient and outpatient		Skin Tone—Fitzpatrick Scale categorized into light, medium, and dark
	Ruppel et al [55], 2023	774		Not reported		Not reported		Not reported		Medical—cardiac catheterization		Inpatient		Race
	Sudat et al [56], 2023	8735		Not reported		Not reported		Not reported		Medical—multiple conditions		Inpatient		Race
	Thrush and Hodges [57], 1994	25		Critikon Dinamap Plus Model 8700 Critikon Oxyshuttle Ohmeda 3700 Catalyst Research MiniOx IV		IL482 co-oximeter		80-100		Healthy—nonsmoking adults		Research — laboratory		Race—all White
	Valbuena et al [12], 2022	372		Not reported		Not reported, blood gas analysis		Not reported		Medical—adult patients with respiratory failure or COVID-19		Inpatient		Race and ethnicity—White, Black, Hispanic, and Asian
	Vesoulis et al [58], 2021	294		Nellcor SpO₂ module with Neonatal-Adult MAX-N adhesive SpO₂ sensor (Covidien) (used with either Philips IntelliVue MP70 or MX800 monitors)		Radiometer ABL800 Flex		Not reported		Medical—preterm infants at neonatal intensive care unit		Inpatient		Race—White and Black
	Wiles et al [59], 2022	194		Nellcor reusable SpO₂ probes or Mindray disposable SpO₂ probes (GE Healthcare B1x5 M/P monitor)		RAPIDpoint 500 analyser (Siemens Healthcare GmbH)		Not reported		Medical—adult patients with COVID-19 pneumonitis		Inpatient		Race—Asian, Black, White, and other
	Zeballos and Weisman [9], 1991	33		Hewlett-Packard HP-47201A Ohmeda Biox IIA		IL282 co-oximeter		Not reported		Healthy—non-smoking volunteers		Research — laboratory		Race—all Black
Pulse Rate
	Bent et al [41], 2020	53		Empatica E4 Apple Watch Fitbit Charge Garmin Vivosmart 3 Xiaomi Miband Biovotion Everion		BittiumFaros 180, Bittium Inc.		N/A^g		Healthy		Research — laboratory		Skin tone—Fitzpatrick Scale: 1 (n=7) 2 (n=8) 3 (n=10) 4 (n=9) 5 (n=9) 6 (n=10)
	Nelson and Allen [60], 2019	1		Apple Watch 3 Fitbit Charge 2		Vrije Universiteit Ambulatory Monitoring System		N/A		Healthy		Research — real world		Skin tone—Fitzpatrick Scale: 1 (n=1)
	Sanudo et al [61], 2019	45		Apple Watch (version not reported)		Polar Chest Strap		N/A		Healthy		Research — laboratory		Skin tone—Fitzpatrick Scale: 2 (n=15) 3 (n=15) 4 (n=15)
	Chow and Yang [62], 2020	40		Garmin Vivosmart HR+ Xiaomi Mi Band 2		Polar H7 Chest Strap		N/A		Healthy		Research — laboratory		Ethnicity and skin tone—East Asian, Fitzpatrick Scale 3 and 4

^aSaO₂: arterial blood gas.

^bSET: Signal Extraction Technology.

^cEEL: electron energy loss.

^dSpO₂: blood oxygenation.

^fCOPD: chronic obstructive pulmonary disease.

^gN/A: not applicable.

Quality Assessment of the Overall Evidence

QUADAS-2 tool was used to evaluate risk of bias and applicability (Figure S1 in Multimedia Appendix 1). Funnel plots evaluated publication bias using the metafor package [63] (Figures S2 and S3 in Multimedia Appendix 1).

Statistical Analysis Skin Tone Categorization

We mapped skin tone, race, and ethnicity into 3 primary skin pigmentation groups of light, medium, and dark, following published methodology [36] (see Table S1 in Multimedia Appendix 1) and examined biases for SpO₂ and PR by each skin pigmentation group (described below).

We elected to use this same skin pigmentation categorization schema as has previously been used, with the goal of expanding upon the analysis of Shi et al [36].

Statistical Analysis

The objective of the study was to assess whether the devices were accurate in estimating SpO₂ and PR when compared with a SaO₂ and ECG reference device, respectively, for each skin pigmentation group. If these measures were found to be inaccurate, biases were quantified and their clinical relevance was assessed. The analytical approach to execute these research objectives was formulated based on methodologies used in prior meta-analyses within the discipline [36].

Evaluation End Points and Criteria

A summary of the evaluation criteria used can be found in Table 2.

Table 2

Evaluation end points and criteria for blood oxygenation (SpO₂) and pulse rate studies.

Category and objective		Evaluation end point	Evaluation criteria
SpO₂
	Evaluate accuracy	A_rms^a	Accurate if A_rms≤3%
	Statistically significant bias	Mean bias (95% CI)	Statistically significant bias if 95% CI does not contain 0
	Clinically relevant bias	Mean bias (95% CI)	Clinically relevant bias if either upper bound of 95% CI≤4% or lower bound of 95% CI>4%
Pulse rate
	Evaluate accuracy	95% LoA^b	Accurate if 95% LoA is bounded by +5 or –5 bpm^c
	Statistically significant bias	Mean bias (95% CI)	Statistically significant bias if 95% CI does not contain 0
	Clinically relevant bias	Mean bias (95% CI)	Clinically relevant bias if either upper bound of 95% CI ≤5 bpm or lower bound of 95% CI >5 bpm

^aA_rms: accuracy root-mean-square.

^bLoA: limits of agreement.

^cbpm: beats per minute.

Accuracy

To evaluate the accuracy of SpO₂ , A_rms was used to measure accuracy, because of its commonality in the regulatory space [19]. It was calculated as , where precision is the SD of bias [42,43]. A_rms is useful in clinical settings because it provides a single metric that accounts for both the systematic error (bias) and the random error (precision) in measurements and it provides a comprehensive assessment of how close the measurements are to the true values. A lower A_rms value indicates better accuracy, meaning that the measurement is closer to the true value, whereas a higher A_rms value suggests lower accuracy, indicating that the measurement is further away from the true value.

For SpO₂, our study used the stricter A_rms>3% threshold to define inaccuracy as per FDA guidance [19]. We used this threshold as 22 (96%) out of 23 of pulse oximeter studies used a transmittance device and only one used an ear clip, which would have set a more liberal threshold of 3.5%. To evaluate accuracy of PR, the 95% LoA of bias was used, and LoA was constructed as mean bias (SD 1.96). For PR, a pooled 95% LoA not bounded by +5 or –5 bpm was considered inaccurate per ANSI/AAMI/IEC standards [33,34].

Bias

To evaluate bias of both SpO₂ and PR, mean bias and its 95% CI were used. For each data point that compares the test with the reference device, bias was constructed as test device – reference device. Based on the final results, a measure was considered to have statistically significant bias if the 95% CI of its mean bias did not contain 0, with the estimated mean bias over 0 indicating overestimation and under 0 indicating underestimation.

Clinical Relevance of Estimated Bias

To evaluate the clinical relevance of the estimated bias for SpO₂, if the 95% CI of mean bias was out of the +4% to –4% range it was considered clinically relevant. This threshold is inferred from FDA guidance on Pulse Oximeters—Premarket Notification Submissions [510(k)s]: Guidance for Industry and Food and Drug Administration Staff [19]. To evaluate the clinical relevance of the estimated bias for PR, if the 95% CI of mean bias was out of the bound of +5 or –5 bpm, it was considered clinically relevant as per ANSI/AAMI/IEC standards [33,34].

Statistical Meta-Analysis

To obtain pooled results for above listed end points, we collected sample size, paired observations, mean bias, and SD from various studies. When not available, we either transformed relevant parameters, such as 95% LoA and 95% CI, into these statistics or obtained them by analyzing the raw data.

Methods to Pool Measures of Bias

Correlated and hierarchical effect (CHE) models were used to pool mean biases from various studies. Specifically, 3-level hierarchical models were constructed, with Level 1 representing individual data points collected in a study, Level 2 representing potentially multiple comparisons within a research study, and Level 3 representing various studies included in the meta-analysis. Within a study, there may be multiple devices compared using the same participants’ data, and these effect sizes are dependent. To account for this dependence, the CHE model used the robust variance estimation (RVE) method, which allowed us to combine data from single-measure design studies with multiple dependent estimates of effect size, even when the dependence is unknown [64]. To execute the CHE model with RVE, we needed to plug in assumed correlation coefficients for these dependent effect sizes. We used correlation coefficient 0.9 following published methods [36], and small (⍴=0.30) and moderate (⍴=0.60) correlation coefficients were also explored in sensitivity analyses. Additionally, one report [48] had substantially larger SE, so we conducted a separate analysis excluding its results as part of the sensitivity analysis. To evaluate the heterogeneity of the studies, we reported the overall I² (percentage of variability in the effect sizes that is not caused by sampling error) and its breakdown of within-study (I²_Level2) and between-study (I²_Level3) portions. We conducted subgroup analyses to see if there were statistically significant differences in bias between skin pigmentation categories (Table S2 from Multimedia Appendix 1). Forest plots were provided to visualize the mean bias (Figures S2 and S3 from Multimedia Appendix 1).

Methods to Pool Measures of Accuracy

When it comes to providing pooled 95% LoA and A_rms, no established methodologies exist, so we followed an analytical approach used in prior meta-analyses within the discipline [36]. Specifically, SD of bias across studies were pooled using CHE models similar to those used to pool mean biases. The pooled mean bias and pooled SD were then used to provide pooled estimates of 95% LoA and A_rms. Specifically,

Overall accuracy A_rms =

Overall 95% LoA = pooled mean bias ± 1.96 × pooled SD

Other Methods

Descriptive statistics were provided for study characteristics, the device intended use, mean bias based on different skin tones, and reference devices. All analyses were conducted using R (version 4.3.1; R Foundation for Statistical Computing) with statistical models based on R packages metafor [63] and clubSandwich [65].

Results Study Selection and Characteristics

Search strategy resulted in 8582 records. We selected 27 studies for full review (pulse oximetry: n=23, PR: n=4; Figure 1, Table 1, and Table 2). A total of 23 pulse oximetry studies involving 59,684 participants with 197,353 paired SpO₂-SaO₂ observations were included. Additionally, 4 PR studies with 176 participants and 140,771 paired PR-ECG observations were analyzed (Table 3).

Table 3

Summary characteristics of included studies^a.

Item and subitem		Pulse oximetry (n=23), n (%)	Pulse rate (n=4), n (%)
Participants (pulse oximetry: n=59,684; pulse rate: n=176)
	Light skin pigmentation	40,416 (67.72)	31 (17.61)
	Medium skin pigmentation	9967 (16.70)	129 (73.30)
	Dark skin pigmentation	9301 (15.58)	16 (9.09)
Paired observations (pulse oximetry: n=197,353; pulse rate: n=140,771)
	Light skin pigmentation	131,008 (66.38)	43,116 (30.60)
	Medium skin pigmentation	32,095 (16.26)	90,733 (64.50)
	Dark skin pigmentation	34,250 (17.35)	6922 (4·92)
Device type included in study
	Medical	23 (100)	0 (0)
	Nonregulated	0 (0)	4 (100)
Sensor type
	Transmittance	22 (96)	0 (0)
	Reflectance	1 (4)	4 (100)
Patient population
	Healthy	4 (17)	3 (75)
	Medical	18 (78)	0 (0)
	Healthy and medical	1 (4)	0 (0)
	Not reported	0 (0)	1 (25)
Research setting
	Medical inpatient	15 (65)	0 (0)
	Medical outpatient	1 (4)	0 (0)
	Medical combined	1 (5)	0 (0)
	Research laboratory	6 (26)	3 (75)
	Research in the real world	0 (0)	1 (25)
Skin pigmentation method
	Skin tone	4 (17)	4 (100)
	Race	13 (57)	0 (0)
	Ethnicity	3 (13)	0 (0)
	Race and ethnicity	3 (13)	0 (0)

^aMedical device defined as devices that received regulatory clearance for either blood oxygenation (SpO₂) or pulse rate (PR).

Descriptive Statistics on Race, Ethnicity, and Skin Tone Pulse Oximetry

Skin pigmentation was classified by race in 13 (57%) out of 23 studies, ethnicity in 3 (13%) studies, skin tone in 4 (17%) studies, and both race and ethnicity in 3 (13%) studies (Table 3). Of the 59,684 patients with 197,353 paired observations, there were a total of 40,416 (67.72%) patients with light skin pigmentation with 131,008 (66.38%) paired observations, 9967 (16.70%) patients with medium skin pigmentation with 32,095 (16.26%) paired observations, and 9301 (15.58%) patients with dark skin pigmentation with 34,250 (17.35%) paired observations.

PR Results

Skin pigmentation was classified by race or ethnicity in 0 (0%) out of 4 studies and skin tone in 4 (100%) studies (Table 3). Of the 176 patients with 140,771 paired observations, there were a total of 31 (17.61%) participants with light skin pigmentation with 43,116 (30.60%) paired observations, 129 (73.30%) participants with medium skin pigmentation with 90,733 (64.50%) paired observations, and 16 (9.09%) participants with dark skin pigmentation with 6922 (4.92%) paired observations.

PPG Accuracy and Bias by Skin Pigmentation Pulse Oximetry

The pooled A_rms across different skin pigmentation groups was 3.6%, 3.96%, 4.71%, and 4.15% for combined, light, medium, and dark skin pigmentation, respectively (Table 4 and Figures 2-4 [8-11,17,42-51,54-56,58,59]). Of note, studies implementing multiple trial conditions or using multiple study devices were shown multiple times in Figures 2-4 to delineate different devices used within the same study. We observed a pooled mean percent bias of 0.82% (95% CI 0.29%-1.35%) across all skin pigmentation groups using the CHE model. Between-study heterogeneity (I²_Level3) accounted for 14.53% of the total variation, while within-study heterogeneity (I²_Level2) explained 84.02% of total variation. Delineating by skin pigmentation, the pooled mean percent bias from the CHE model was 0.70% (95% CI 0.17%-1.22%) for light skin, 0.27% (95% CI –0.64% to 1.19%) for medium skin, and 1.27% (95% CI 0.58%-1.95%) for dark skin (Tables S3 and S4 in Multimedia Appendix 1). Subgroup analyses found no statistically significant difference between light and medium skin pigmentation (estimate=0.113, SE 0.259; 95% CI –0.459 to 0.686), but there was a statistically significant difference between light and dark skin pigmentation (estimate=0.596, SE 0.240; 95% CI 0.069-1.123), such that pooled bias for those with darker skin was higher as compared to those with lighter skin (Table S2 in Multimedia Appendix 1).

Figure 2

Forest plot showing bias in SpO₂ measurements in patients with light skin pigmentation. Multiple entries from the same study are included, each representing different devices evaluated. Squares denote study weight; center of squares denote observed study effect size; vertical lines denote study CIs; and diamond denotes pooled effect. CHE: correlated and hierarchical effect; SpO₂: blood oxygenation.

Figure 3

Forest plot showing bias in SpO₂ measurements in subjects with medium skin pigmentation. Multiple entries from the same study are included, each representing different devices evaluated. Note: squares denote study weight, center of squares denote observed study effect size, vertical lines denote study CIs, and diamond denotes pooled effect. CHE: correlated and hierarchical effect; SpO₂: blood oxygenation.

Figure 4

Forest plot showing bias in SpO₂ measurements in patients with dark skin pigmentation. Multiple entries from the same study are included, each representing different devices evaluated. Squares denote study weight; center of squares denote observed study effect size; vertical lines denote study CIs; and diamond denotes pooled effect. CHE: correlated and hierarchical effect; SpO₂: blood oxygenation.

Table 4

Pulse rate and pulse oximetry bias by skin pigmentation^a.

Device type and skin pigment category		Studies (evaluations), n	Sample size (data pairs), n	Unit	Pooled mean bias (95% CI)	Pooled SD (SE)	95% LoA^b	A_rms^c (%)	Overall I² (between- and within-study heterogeneity)
Pulse oximetry
	Light	20 (44)	40,416 (131,008)	Percent	0.70 (0.17 to 1.22) ^d	3.90 (1.36)	–6.94 to 8.34	3.96^e	97.45% (46.44% and 51.01%)
	Medium	9 (15)	9,967 (32,095)	Percent	0.27 (–0.64 to 1.19)	4.71 (1.71)	–8.95 to 9.50	4.71^e	95.31% (81.16% and 14.14%)
	Dark	19 (36)	9,301 (34,250)	Percent	1.27 (0.58 to 1.95)	3.96 (1.30)	–6.49 to 9.02	4.15^e	98.46% (0.00% and 98.46%)
	Combined	23 (95)	59,684 (197,353)	Percent	0.82 (0.29 to 1.35)	3.50 (1.28)	–6.04 to 7.68	3.60^e	98.55% (14.53% and 84.02%)
Pulse rate
	Light	3 (9)	31 (43,116)	bpm^f	–1.24 (–5.31 to 2.83)	7.54 (2.13)	–16.02^g to 13.54^g	—^h	10.99% (0.00% and 10.99%)
	Medium	3 (9)	129 (90,733)	bpm	–0.89 (–3.70 to 1.93)	9.05 (1.75)	–18.62^g to 16.84^g	—	25.01% (0.00% and 25.01%)
	Dark	1 (6)	16 (6,922)	bpm	–0.57 (–9.44 to 8.29)	16.89 (1.31)	–33.69^g to 32·54^g	—	13.70% (N/Aⁱ and 13.70%)
	Combined	4 (24)	176 (140,771)	bpm	–0.29 (–3.87 to 3.29)	8.64 (1.67)	–17.23^g to 16.65^g	—	30.66% (26.76% and 3.90%)

^aρ=0.9 was used in correlated and hierarchical effect models to pool both mean bias and SD.

^bLoA: limits of agreement.

^cA_rms: accuracy root-mean-square

^dItalicization denotes statistical significance.

^eExceeds U.S. Food and Drug Administration guidance for pulse oximetry.

^fbpm: beats per minute.

^gExceeds American National Standards Institute standards for pulse rate.

^hNot applicable.

ⁱN/A: not available.

PR Results

Analysis using the CHE model revealed LoA values of –17.23 to 16.65 bpm and a mean bias of –0.29 (95% CI –3.87 to 3.29) bpm across all studies. Heterogeneity analysis demonstrated that 26.76% of the variation in bias (I²_Level3) stemmed from between-study differences, while 73.34% (I²_Level2) originated from within-study variation. Our analysis revealed 95% LoA of –16.02 to 13.54 bpm, –18.62 to 16.84 bpm, and –33.69 to 32.54 bpm for light, medium, and dark skin pigmentation groups, respectively. Mean biases were –1.24 (95% CI –5.31 to 2.83) bpm, –0.89 (95% CI –3.70 to 1.93) bpm, and –0.57 (95% CI –9.44 to 8.29) bpm for the corresponding groups (Table 4 and Figures 5-7 [41,60-62]). Detailed results are provided in Tables S4 and S5 in Multimedia Appendix 1. Subgroup analyses found no statistically significant difference between light and medium or between light and dark skin pigmentation pooled bias (P≥.05; Table S2 in Multimedia Appendix 1).

Figure 5

Forest plot showing pulse rate measurement bias in patients with light skin pigmentation. Squares denote study weight; center of squares denotes observed study effect size; vertical lines denote study CIs; and diamond denotes pooled effect. CHE: correlated and hierarchical effect.

Figure 6

Forest plot showing pulse rate measurement bias in patients with medium skin pigmentation. Squares denote study weight; center of squares denotes observed study effect size; vertical lines denote study CIs; and diamond denotes pooled effect. CHE: correlated and hierarchical effect.

Figure 7

Forest plot showing pulse rate measurement bias in patients with dark skin pigmentation. Squares denote study weight; center of squares denotes observed study effect size; vertical lines denote study CIs; and diamond denotes pooled effect. CHE: correlated and hierarchical effect.

Sensitivity Analyses

Sensitivity analyses were performed for pulse oximetry SpO₂ by removing Crooks et al [48] as an outlier due to substantially large SE (Tables S6 and S7 in Multimedia Appendix 1). The pooled mean percent biases and 95% LoA were essentially unchanged, while the pooled A_rms across all skin pigmentation groups was 3.00%, 2.91%, and 3.40% for light, medium, and dark skin pigmentation, respectively. Sensitivity analyses using CHE models with a range of correlation coefficients (including small and moderate values) yielded similar conclusions to those obtained with a high correlation coefficient (ρ=0.9), except for the PR analysis in the light skin pigmentation group. This analysis revealed a statistically significant bias when a low correlation coefficient (ρ=0.3) was used. Details are provided in Table S6 in Multimedia Appendix 1.

Discussion Principal Findings

The study revealed a paucity of studies properly assessing skin pigmentation, a tendency for uneven distribution across different skin pigmentation groups (overrepresentation of light skin for SpO₂ and a single study with representation of dark skin for PR), and lack of consumer-device reporting despite growing use in clinical setting. Results suggest inaccurate SpO₂ and PR measurements across all skin pigmentation groups as they breach FDA and ANSI/AAMI/IEC standards, respectively, with wearable accuracy varying considerably depending on the model, which may be due to date of model production or algorithm development. Pulse oximeters may also overestimate SpO₂ significantly for light and dark skin pigmentation, but without clinically relevant bias. We did not find statistically significant or clinically relevant bias in wearable PR devices.

Despite not meeting FDA guidance across all groups, pulse oximeter SpO₂ was inaccurate only across medium and dark skin pigmentation groups when compared to the more liberal international thresholds. Additionally, in the sensitivity analyses without the outlier study of Crooks et al [48], all pooled A_rms values dropped, resulting in inaccurate pulse oximeter SpO₂ only for dark skin pigmentation and no group exceeding international thresholds.

The results showing pulse oximeters significantly overestimating SpO₂ were expected for dark pigmentation and supported by findings on patient outcomes [11-15,17], but overestimated values for light pigmentation were unexpected. Two possible reasons come to mind. First, less melanin in lighter skin could distort the PPG signal [18]. Second, devices calibrated on individuals with medium skin pigmentation (note that 48% of the US population is categorized as Fitzpatrick Skin Tone Scale III [66]) may lead to inaccurate readings for both lighter and darker skin pigmentation, since both may be suboptimally represented during algorithm training and testing.

Lastly, it should be noted that both SpO₂ and PR studies had lower between-study heterogeneity and higher within-study heterogeneity, indicating that the studies that were included in these analyses were largely consistent with one another and that most of the variation came from within studies potentially due to higher variability across devices used or participants enrolled within each study.

Overall, these findings suggest that when pulse oximetry devices are deployed in their setting of intended use (ie, uncontrolled settings, such real-world medical settings and home environments with diverse patient populations), the performance observed in analytical validation studies may not generalize.

Strengths and Limitations

There were a few limitations, mostly from limitations inherent in prior studies, that should be noted. First, our skin pigmentation categorization approach has strengths but also limitations. It was an effort to overcome the reporting heterogeneity and the tendency in the published literature to conflate data collection on race, ethnicity, and skin tone. This is particularly problematic as skin tone is a physiological concept (determined by the melanin amount in the basal layer of the epidermis), while ethnicity and race are largely social constructs, with high underlying physiological heterogeneity [66]. To reduce heterogeneity, this meta-analysis reclassified race, ethnicity, and skin tone into a universal schema for skin pigmentation based on the system used by Shi et al [36]. This method, however, classifies most White people in the United States as light rather than medium skin pigmentation [66]. Second, there were only 4 prior PR studies that collected participant skin tone that also reported on device accuracy and only 1 that had participants with dark pigmentation. Third, our study used stated FDA and ANSI/AAMI/IEC standards as set thresholds to gauge device performance. It is possible that guidelines and thresholds cited in our study may change in the future, potentially limiting the applicability of the conclusions drawn in this paper. Fourth, our study chose to group papers using a variety of patient populations and testing methodologies with the goal of aggregating the largest pool of data possible on which to draw conclusions across multiple contexts. As described above, low between-study heterogeneity indicates that the studies used in this meta-analysis were largely consistent with one another. This reduces the likelihood of a moderating variable having a significant effect between studies. Instead, it suggests that most of the variation originated within studies, possibly due to higher variability across devices used or participants enrolled within each study. Despite this, we conducted subgroup analyses to examine the potential moderating variables of medical versus healthy populations for pulse oximeters (see Table S8 in Multimedia Appendix 1). We did not conduct these analyses for pulse rate because most studies did not specify whether subjects were healthy or part of medical populations. We did not examine the potential moderating effects of sensor type, transmittance versus reflectance sensors, wavelength of light used by sensors, and location of sensor placement given the core sensor technology studied was PPG across all included studies. The level of SpO₂ was not examined given this study’s goal of gathering the largest sample upon which to study the effect of skin tone. Participant activity level was not evaluated as a moderator variable because many included studies did not delineate participant activity level explicitly. Future studies should evaluate the effect of these and other moderating variables on SpO₂ and PR.

Evidence Generation Guidelines for Future Analytical Validation Studies

The 2013 FDA guidance may be insufficient to ensure accuracy in pulse oximeters across all skin pigmentation and settings of intended use [19]. But there are now multiple FDA guidances for digital health tools requiring fit-for-purpose evidence as well as growing concern/guidance on clinical research diversity [4,30,31,67-73]. The FDA currently categorizes consumer devices as low-risk wellness products, exempting them from stringent regulatory oversight. However, as these devices are integrated into clinical decision-making and used as tools in clinical research [3,4], it becomes crucial to understand and communicate their advantages and limitations. Increased reliance on consumer devices increases the demand for accurate devices whose performance features and potential impact on health outcomes are known with transparency. To generate fit-for-purpose evidence applicable to diverse population, here, we propose 5 recommendations based on FDA guidance and literature (Textbox 1) [4,30,31,67-73]:

Recommendations for future analytical validation studies.

Recommendation 1

It is vitally important for medical pulse oximetry devices as well as nonregulated research and consumer devices to incorporate the V3 framework [74] for sensor verification and analytical validation of derived SpO₂ and PR values for regulatory submission before these metrics can be responsibly deployed in medical, consumer, and research settings.

Recommendation 2

When possible, analytically validate devices in settings of intended use [74], rather than relying on controlled laboratory settings where digits may be warmed prior to testing, and confirm device accuracy in all subgroups (sex, race, skin pigmentation, healthy vs medical populations).

Recommendation 3

Use objective measures of skin pigmentation, rather than relying on race and ethnicity, as this will reduce heterogeneity in studies and allow for a more accurate understanding of how skin pigmentation impacts device performance.

Recommendation 4

Industry should set a priori maximum allowable difference thresholds using FDA and ANSI guidelines, properly power each subgroup, and require that 95% LoA fit within these standards for each subgroup and in the setting of intended use before receiving regulatory approval, production, and deployment.

Recommendation 5

Future studies should report device and firmware versions, as firmware updates may include changes in underlying algorithms influencing accuracy of metric generation, as described previously in the literature [4].

Conclusions

PPG has been applied in clinical practice for decades, and its accuracy in patients with different skin pigmentation has long been in question. Whether this technology contributes to diagnostic biases and by how much is only more pressing for clinicians and patients with the advent of consumer wearable PPG sensors and the growing interest and incorporation of these devices into clinical practice and in clinical research. This systematic review and meta-analysis found that pulse oximeter SpO₂ and wearable PR were inaccurate across all skin pigmentation groups as the resulting accuracy values breached FDA guidance and ANSI/AAMI/IEC standard thresholds, respectively, although pulse oximeter SpO₂ was only found to be inaccurate for dark skin pigmentation in sensitivity analyses. In addition, despite not exceeding clinically relevant bias thresholds, pulse oximeters were found to significantly overestimate SpO₂ for light and dark skin pigmentation. No systematic or clinically relevant bias was found in estimation of PR. The recommendations in this paper can help advise patients, study participants, care providers, device manufacturers, application developers, researchers, and legislators on best practices going forward.

Multimedia Appendix 1

Supplementary information.

Multimedia Appendix 2

PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) 2020 checklist.

Abbreviations

AAMI

Association for the Advancement of Medical Instrumentation

ANSI

American National Standards Institute

A_rms

accuracy root-mean-square

bpm

beats per minute

CHE

correlated and hierarchical effect

ECG

electrocardiography

FDA

Food and Drug Administration

IEC

International Electrotechnical Commission

LoA

limits of agreement

PPG

photoplethysmography

pulse rate

PRIMSA

Preferred Reporting Items for Systematic Reviews and Meta-Analyses

RVE

robust variance estimation

SaO₂

arterial blood gas

SpO₂

blood oxygenation

Data Availability

Open code and data are available on Open Science Framework [75] (see the Materials—Open Code and Data section in Multimedia Appendix 1).

SS contributed to study concept and design, data collection, and writing portions of the Methods section of the manuscript. HG contributed to study concept and design, data collection, and writing portions of the Methods and Discussion sections of the manuscript. BWN contributed to study concept and design; data collection; data analysis and interpretation; and writing portions of the Abstract, Introduction, Methods, Results, and Discussion sections of the manuscript. MRB contributed to data collection and writing portions of the Methods section of the manuscript. CC contributed to statistical analysis method determination, code review, and writing portions of the Methods section of the manuscript. All authors (SS, MRB, CC, SS, HG, and BWN) reviewed the final manuscript. HG and BWN contributed to this project as shared senior authors.

BWN, MRB, SS, and CC report past or current employment and/or equity ownership in Verily Life Sciences. HG serves or has served as a consultant for Verily Life Sciences, Boston Scientific, Huxley Medical, and Happitech.

Kyriacou

Allen

Photoplethysmography: Technology, Signal Analysis and Applications 2021

Cambridge, MA

Academic Press

Knowles

Krasniansky

Nagappan

Consumer adoption of digital health in 2022: moving at the speed of trust

Rock Health 2023 2 21

2023-03-01

https://rockhealth.com/insights/consumer-adoption-of-digital-health-in-2022-moving-at-the-speed-of-trust/

Al-Alusi

Khurshid

Wang

Venn

Pipilas

Ashburner

Ellinor

Singer

Atlas

Lubitz

Trends in consumer wearable devices with cardiac sensors in a primary care cohort

Circ Cardiovasc Qual Outcomes 2022 15 7 e008833

10.1161/CIRCOUTCOMES.121.008833

35758032

PMC9308742

Nelson

Low

Jacobson

Areán

Torous

Allen

Guidelines for wrist-worn consumer wearable assessment of heart rate in biobehavioral research

NPJ Digit Med 2020 3 90

10.1038/s41746-020-0297-4

32613085

297

PMC7320189

Khan

Pretty

Amies

Elliott

Shaw

Chase

Investigating the effects of temperature on photoplethysmography

IFAC-PapersOnLine 2015 48 20 360 365

10.1016/j.ifacol.2015.10.166

Fine

Branan

Rodriguez

Boonya-Ananta

Ajmal Ramella-Roman

McShane

Coté

Sources of inaccuracy in photoplethysmography for continuous cardiovascular monitoring

Biosensors (Basel) 2021 11 4 126

10.3390/bios11040126

33923469

bios11040126

PMC8073123

Ries

Prewitt

Johnson

Skin color and ear oximetry

Chest 1989 96 2 287 290

10.1378/chest.96.2.287

2752811

S0012-3692(15)45916-1

Feiner

Severinghaus

Bickler

Dark skin decreases the accuracy of pulse oximeters at low oxygen saturation: the effects of oximeter probe type and gender

Anesth Analg 2007 105 6 Suppl S18 S23

10.1213/01.ane.0000285988.35174.d9

18048893

00000539-200712001-00004

Zeballos

Weisman

Reliability of noninvasive oximetry in black subjects during exercise and hypoxia

Am Rev Respir Dis 1991 144 6 1240 1244

10.1164/ajrccm/144.6.1240

1741533

Bickler

Feiner

Severinghaus

Effects of skin pigmentation on pulse oximeter accuracy at low saturation

Anesthesiology 2005 102 4 715 719

10.1097/00000542-200504000-00004

15791098

00000542-200504000-00004

Sjoding

Dickson

Iwashyna

Gay

Valley

Racial bias in pulse oximetry measurement

N Engl J Med 2020 383 25 2477 2478

10.1056/NEJMc2029240

33326721

PMC7808260

Valbuena

VSM

Seelye

Sjoding

Valley

Dickson

Gay

Claar

Prescott

Iwashyna

Racial bias and reproducibility in pulse oximetry among medical and surgical inpatients in general care in the veterans health administration 2013-19: multicenter, retrospective cohort study

BMJ 2022 378 e069775

10.1136/bmj-2021-069775

35793817

PMC9254870

Gottlieb

Ziegler

Morley

Rush

Celi

Assessment of racial and ethnic differences in oxygen supplementation among patients in the intensive care unit

JAMA Intern Med 2022 182 8 849 858

10.1001/jamainternmed.2022.2587

35816344

2794196

PMC9274443

Okunlola

Lipnick

Batchelder

Bernstein

Feiner

Bickler

Pulse oximeter performance, racial inequity, and the work ahead

Respir Care 2022 67 2 252 257

10.4187/respcare.09795

34772785

respcare.09795

Wong

Charpignon

Kim

Josef

de Hond

AAH

Fojas

Tabaie

Liu

Mireles-Cabodevila

Carvalho

Kamaleswaran

Madushani

RWMA

Adhikari

Holder

Steyerberg

Buchman

Lough

Celi

Analysis of discrepancies between pulse oximetry and arterial oxygen saturation measurements by race and ethnicity and association with organ dysfunction and mortality

JAMA Netw Open 2021 4 11 e2131674

10.1001/jamanetworkopen.2021.31674

34730820

2785794

PMC9178439

Fawzy

Wang

Sands

Fisher

Arnold Egloff

DellaVolpe

Iwashyna

Garibaldi

Clinical outcomes associated with overestimation of oxygen saturation by pulse oximetry in patients hospitalized with COVID-19

JAMA Netw Open 2023 6 8 e2330856

10.1001/jamanetworkopen.2023.30856

37615985

2808735

PMC10450566

Fawzy

Wang

Robinson

Farha

Bradke

Golden

Garibaldi

Racial and ethnic discrepancy in pulse oximetry and delayed identification of treatment eligibility among patients with COVID-19

JAMA Intern Med 2022 182 7 730 738

10.1001/jamainternmed.2022.1906

35639368

2792653

PMC9257583

Keller

Harrison-Smith

Patil

Arefin

Skin colour affects the accuracy of medical oxygen sensors

Nature 2022 610 7932 449 451

10.1038/d41586-022-03161-1

36261563

10.1038/d41586-022-03161-1

Pulse oximeters - premarket notification submissions [510(k)s] guidance for industry and Food and Drug Administration staff

U.S. Food and Drug Administration 2013 03 04

2024-08-24

https://www.fda.gov/media/72470/download

Review of pulse oximeters and factors that can impact their accuracy

U.S. Food and Drug Administration 2022

2024-08-24

https://www.fda.gov/media/162709/download

McFarling

FDA panel asks for improvements in pulse oximeters

STAT News 2022

2023-01-09

https://www.statnews.com/2022/11/01/fda-panel-asks-for-improvements-in-pulse-oximeters/

American Psychiatric Association

Wearable Devices as Therapy Tools

Monitor on Psychology 2021 09

2023-01-10

https://www.apa.org/monitor/2021/2021-09-monitor.pdf

Center for Devices and Radiological Health

Pulse oximeter accuracy and limitations: FDA safety communication

U.S. Food and Drug Administration 2023-01-10

https://www.fda.gov/medical-devices/safety-communications/pulse-oximeter-accuracy-and-limitations-fda-safety-communication

November 1, 2022: anesthesiology and respiratory therapy devices panel of the medical devices advisory committee meeting announcement

U.S. Food and Drug Administration 2023 4 4

2023-10-05

https://www.fda.gov/advisory-committees/advisory-committee-calendar/november-1-2022-anesthesiology-and-respiratory-therapy-devices-panel-medical-devices-advisory

McFarling

Pulse oximeters' inaccuracies in darker-skinned people require urgent action, AGs tell FDA

STAT News 2023 11 07

2023-11-07

https://www.statnews.com/2023/11/07/pulse-oximeters-attorneys-general-urge-fda-action/#:~:text=Pulse%20oximeters?%20overestimation%20of%20oxygen,for%20severe%20Covid%2D19%20infections

Attorney General letter to FDA

State of California Office of the Attorney General 2023 11 01

2023-11-01

https://oag.ca.gov/system/files/attachments/press-docs/23PR353%20Health%20Equity%20General%20Matter%20Multistate.pdf

Zinzuwadia

Singh

Wearable devices-addressing bias and inequity

Lancet Digit Health 2022 4 12 e856 e857

10.1016/S2589-7500(22)00194-7

36335031

S2589-7500(22)00194-7

Shachar

Gerke

Prevention of bias and discrimination in clinical practice algorithms

JAMA 2023 329 4 283 284

10.1001/jama.2022.23867

36602791

2800369

Goodman

Morgan

Hoffmann

Clinical algorithms, antidiscrimination laws, and medical device regulation

JAMA 2023 329 4 285 286

10.1001/jama.2022.23870

36602795

2800368

Colvonen

DeYoung

Bosompra

Owens

Limiting racial disparities and bias for wearable devices in health science research

Sleep 2020 43 10 zsaa159

10.1093/sleep/zsaa159

32893865

5902283

PMC8477341

State of California Department of Justice

Attorney General Bonta launches inquiry into racial and ethnic bias in healthcare algorithms

State of California Office of the Attorney General 2022 08 31

2023-11-08

https://oag.ca.gov/news/press-releases/attorney-general-bonta-launches-inquiry-racial-and-ethnic-bias-healthcare

Medical electrical equipment particular requirements for basic safety and essential performance of pulse oximeter equipment (British standard)

ANSI 2023-10-23

https://tinyurl.com/ayfrdpau

ANSI/AAMI/IEC

IEC 60601-2-27 particular requirements for the basic safety and essential performance of electro cardiographic monitoring

ITC India 2024-08-24

https://www.itcindia.org/iec-60601-2-27-particular-requirements-for-the-basic-safety-and-essential-performance-of-electro-cardiographic-monitoring/#:~:text=What%20is%20IEC%2060601%2D2,processing%2C%20alarms%2C%20and%20displays

ANSI/AAMI EC13-2002 cardiac monitors, heart rate meters, and alarms

ANSI 2002

2023-10-30

https://webstore.ansi.org/standards/aami/ansiaamiec132002

Koerber

Khan

Shamsheri

Kirubarajan

Mehta

Accuracy of heart rate measurement with wrist-worn wearable devices in various skin tones: a systematic review

J Racial Ethn Health Disparities 2023 10 6 2676 2684

10.1007/s40615-022-01446-9

36376641

10.1007/s40615-022-01446-9

PMC9662769

Shi

Goodall

Dumville

Hill

Norman

Hamer

Clegg

Watkins

Georgiou

Hodkinson

Lightbody

Dark

Cullum

The accuracy of pulse oximetry in measuring oxygen saturation by levels of skin pigmentation: a systematic review and meta-analysis

BMC Med 2022 20 1 267

10.1186/s12916-022-02452-8

35971142

10.1186/s12916-022-02452-8

PMC9377806

Cabanas

Fuentes-Guajardo

Latorre

León

Martín-Escudero

Skin pigmentation influence on pulse oximetry accuracy: a systematic review and bibliometric analysis

Sensors (Basel) 2022 22 9 3402

10.3390/s22093402

35591092

s22093402

PMC9102088

Al-Halawani

Charlton

Qassem

Kyriacou

A review of the effect of skin pigmentation on pulse oximeter accuracy

Physiol Meas 2023 44 5 05TR01

10.1088/1361-6579/acd51a

37172609

PMC10391744

Page

McKenzie

Bossuyt

Boutron

Hoffmann

Mulrow

Shamseer

Tetzlaff

Akl

Brennan

Chou

Glanville

Grimshaw

Hróbjartsson

Lalu

Loder

Mayo-Wilson

McDonald

McGuinness

Stewart

Thomas

Tricco

Welch

Whiting

Moher

The PRISMA 2020 statement: an updated guideline for reporting systematic reviews

BMJ 2021 372 n71

10.1136/bmj.n71

33782057

PMC8005924

Center for Devices and Radiological Health

How to determine if your product is a medical device

U.S. Food and Drug Administration 2022 09 29

2023-11-16

https://www.fda.gov/medical-devices/classify-your-medical-device/how-determine-if-your-product-medical-device

Bent

Goldstein

Kibbe

Dunn

Investigating sources of inaccuracy in wearable optical heart rate sensors

NPJ Digit Med 2020 3 18

10.1038/s41746-020-0226-6

32047863

226

PMC7010823

Abrams

Sanders

Fallon

Utility of pulse oximetry in the detection of arterial hypoxemia in liver transplant candidates

Liver Transpl 2002 8 4 391 396

10.1053/jlts.2002.32252

11965585

S1527646502803641

Adler

Hughes

Vivilecchia

Camargo

C A

Effect of skin pigmentation on pulse oximetry accuracy in the emergency department

Acad Emerg Med 1998 5 10 965 970

10.1111/j.1553-2712.1998.tb02772.x

9862586

Andrist

Nuppnau

Barbaro

Valley

Sjoding

Association of race with pulse oximetry accuracy in hospitalized children

JAMA Netw Open 2022 5 3 e224584

10.1001/jamanetworkopen.2022.4584

35357460

2790583

PMC8972025

Barker

Wilson

Racial effects on Masimo pulse oximetry: a laboratory study

J Clin Monit Comput 2023 37 2 567 574

10.1007/s10877-022-00927-w

36370242

10.1007/s10877-022-00927-w

PMC9652601

Bothma

P A

Joynt

G M

Lipman

Hon

Mathala

Scribante

Kromberg

Accuracy of pulse oximetry in pigmented patients

S Afr Med J 1996 86 5 Suppl 594 596

8914569

Burnett

Stannard

Wax

Lin

H-M

Pyram-Vincent

DeMaria

Levin

Self-reported race/ethnicity and intraoperative occult hypoxemia: a retrospective cohort study

Anesthesiology 2022 136 5 688 696

10.1097/ALN.0000000000004153

35231085

135689

Crooks

West

Morling

Simmonds

Juurlink

Briggs

Cruickshank

Hammond-Pears

Shaw

Card

Fogarty

Pulse oximeter measurements vary across ethnic groups: an observational study in patients with COVID-19

Eur Respir J 2022 59 4 2103246

10.1183/13993003.03246-2021

35086839

13993003.03246-2021

PMC8796692

Ebmeier

Barker

Bacon

Beasley

Bellomo

Knee Chong

Eastwood

Gilchrist

Kagaya

Pilcher

Reddy

Ridgeon

Sarma

Sprogis

Tanaka

Tweedie

Weatherall

Young

A two centre observational study of simultaneous pulse oximetry and arterial oxygen saturation recordings in intensive care unit patients

Anaesth Intensive Care 2018 46 3 297 303

10.1177/0310057X1804600307

29716488

20170581

Foglia

Whyte

Chaudhary

Mott

Chen

Propert

Schmidt

The effect of skin pigmentation on the accuracy of pulse oximetry in infants with hypoxemia

J Pediatr 2017 182 375 377.e2

10.1016/j.jpeds.2016.11.043

27939107

S0022-3476(16)31272-0

PMC5328979

Jubran

Tobin

Reliability of pulse oximetry in titrating supplemental oxygen therapy in ventilator-dependent patients

Chest 1990 97 6 1420 1425

10.1378/chest.97.6.1420

2347228

S0012-3692(16)32029-3

McGovern

Sasse

Stansbury

Causing

Light

Comparison of oxygen saturation by pulse oximetry and co-oximetry during exercise testing in patients with COPD

Chest 1996 109 5 1151 1155

10.1378/chest.109.5.1151

8625659

S0012-3692(15)46154-9

Muñoz

Torres

Sampol

Rios

Martí

Escrich

Accuracy and reliability of pulse oximetry at different arterial carbon dioxide pressure levels

Eur Respir J 2008 32 4 1053 1059

10.1183/09031936.00126507

18480106

09031936.00126507

Pilcher

Ploen

McKinstry

Bardsley

Chien

Howard

Lee

Beckert

Swanney

Weatherall

Beasley

A multicentre prospective observational study comparing arterial blood gas values to those obtained by pulse oximeters used in adult patients attending Australian and New Zealand hospitals

BMC Pulm Med 2020 20 1 7

10.1186/s12890-019-1007-3

31918697

10.1186/s12890-019-1007-3

PMC6953261

Ruppel

Makeneni

Faerber

Lane-Fall

Foglia

O'Byrne

Bonafide

Evaluating the accuracy of pulse oximetry in children according to race

JAMA Pediatr 2023 177 5 540 543

10.1001/jamapediatrics.2023.0071

36939727

2802301

PMC10028535

Sudat

SEK

Wesson

Rhoads

Brown

Aboelata

Pressman

Mani

Azar

KMJ

Racial disparities in pulse oximeter device inaccuracy and estimated clinical impact on COVID-19 treatment course

Am J Epidemiol 2023 192 5 703 713

10.1093/aje/kwac164

36173743

6730981

PMC9619495

Thrush

Hodges

Accuracy of pulse oximetry during hypoxemia

South Med J 1994 87 4 518 521

10.1097/00007611-199404000-00019

8153783

Vesoulis

Tims

Lodhi

Lalos

Whitehead

Racial discrepancy in pulse oximeter accuracy in preterm infants

J Perinatol 2022 42 1 79 85

10.1038/s41372-021-01230-3

34642469

10.1038/s41372-021-01230-3

PMC8508473

Wiles

El-Nayal

Elton

Malaj

Winterbottom

Gillies

Moppett

Bauchmuller

The effect of patient ethnicity on the accuracy of peripheral pulse oximetry in patients with COVID-19 pneumonitis: a single-centre, retrospective analysis

Anaesthesia 2022 77 2 143 152

10.1111/anae.15581

34542168

PMC8653100

Nelson

Allen

Accuracy of consumer wearable heart rate measurement during an ecologically valid 24-hour period: intraindividual validation study

JMIR Mhealth Uhealth 2019 7 3 e10828

10.2196/10828

30855232

v7i3e10828

PMC6431828

Sañudo

de Hoyo

Muñoz-López

Perry

Abt

Pilot study assessing the influence of skin type on the heart rate measurements obtained by photoplethysmography with the apple watch

J Med Syst 2019 43 7 195

10.1007/s10916-019-1325-2

31119387

10.1007/s10916-019-1325-2

Chow

Yang

Accuracy of optical heart rate sensing technology in wearable fitness trackers for young and older adults: validation and comparison study

JMIR Mhealth Uhealth 2020 8 4 e14707

10.2196/14707

32343255

v8i4e14707

PMC7218601

Viechtbauer

Conducting meta-analyses in R with the metafor package

J Stat Soft 2010 36 3 1 48

Pustejovsky

Tipton

Meta-analysis with robust variance estimation: expanding the range of working models

Prev Sci 2022 23 3 425 438

10.1007/s11121-021-01246-3

33961175

10.1007/s11121-021-01246-3

Pustejovsky

clubSandwich: cluster-robust (sandwich) variance estimators with small-sample corrections

R Project 2024-05-29

https://CRAN.R-project.org/package=clubSandwich

Keiser

Linos

Kanzler

Lee

Sainani

Tang

Reliability and prevalence of digital image skin types in the United States: results from national health and nutrition examination survey 2003-2004

J Am Acad Dermatol 2012 66 1 163 165

10.1016/j.jaad.2011.02.044

22177642

S0190-9622(11)00387-2

Sjoding

Iwashyna

Valley

Change the framework for pulse oximeter regulation to ensure clinicians can give patients the oxygen they need

Am J Respir Crit Care Med 2023 207 6 661 664

10.1164/rccm.202209-1773ED

36260769

PMC10037469

National Academies of Sciences, Engineering, and Medicine Policy and Global Affairs Committee on Women in Science, Engineering, and Medicine Committee on Improving the Representation of Women and Underrepresented Minorities in Clinical Trials and Research

Bibbins-Domingo

Kirsten

Helman

Alex

Improving Representation in Clinical Trials and Research: Building Research Equity for Women and Underrepresented Groups 2022 07 13

Washington, DC

National Academies Press

Lee

Resnick

Barton

Algorithmic bias detection and mitigation: Best practices and policies to reduce consumer harms

Brookings 2019 05 22

2023-11-08

https://www.brookings.edu/articles/algorithmic-bias-detection-and-mitigation-best-practices-and-policies-to-reduce-consumer-harms/

Framework for the use of digital health technologies in drug and biological product development

U.S. Food and Drug Administration 2023 03

2024-08-24

https://www.fda.gov/media/166396/download?attachment

Center for Drug Evaluation and Research

Digital health technologies for remote data acquisition in clinical investigations

U.S. Food and Drug Administration 2023 12

2023-12-05

https://www.fda.gov/regulatory-information/search-fda-guidance-documents/digital-health-technologies-remote-data-acquisition-clinical-investigations

Office of the Commissioner

Diversity plans to improve enrollment of participants from underrepresented racial and ethnic populations in clinical trials; draft guidance for industry

U.S. Food and Drug Administration 2023-12-05

https://www.fda.gov/regulatory-information/search-fda-guidance-documents/diversity-plans-improve-enrollment-participants-underrepresented-racial-and-ethnic-populations

Center for Drug Evaluation and Research

Enhancing the diversity of clinical trial populations — eligibility criteria, enrollment practices, and trial designs guidance for industry

U.S. Food and Drug Administration 2023-11-30

https://www.fda.gov/regulatory-information/search-fda-guidance-documents/enhancing-diversity-clinical-trial-populations-eligibility-criteria-enrollment-practices-and-trial

Goldsack

Coravos

Bakker

Bent

Dowling

Fitzer-Attas

Godfrey

Godino

Gujar

Izmailova

Manta

Peterson

Vandendriessche

Wood

Wang

Dunn

Verification, analytical validation, and clinical validation (V3): the foundation of determining fit-for-purpose for biometric monitoring technologies (BioMeTs)

NPJ Digit Med 2020 3 55

10.1038/s41746-020-0260-4

32337371

260

PMC7156507

Singh

Bennett

Chen

Ghanbari

Nelson

Photoplethysmography pulse oximetry and pulse rate accuracy by skin tone: a meta-analysis

Open Science Framework 2024-09-30

http://osf.io/qngmz/