JMIR J Med Internet Res Journal of Medical Internet Research 1438-8871 JMIR Publications Toronto, Canada v22i9e16179 32930671 10.2196/16179 Letter to the Editor Letter to the Editor Comment on “Designing Robust N-of-1 Studies for Precision Medicine: Simulation Study and Design Recommendations” Eysenbach Gunther Derrick Thomas Percha Bethany Landes Reid D PhD 1
Department of Biostatistics University of Arkansas for Medical Sciences 4301 W Markham St Slot 781 Little Rock, AR, 72205 United States 1 501 526 6714 rdlandes@uams.edu
https://orcid.org/0000-0002-6349-0081 Department of Biostatistics University of Arkansas for Medical Sciences Little Rock, AR United States Corresponding Author: Reid D Landes rdlandes@uams.edu 9 2020 15 9 2020 22 9 e16179 7 9 2019 10 8 2020 ©Reid D Landes. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 15.09.2020. 2020

This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.

 https://www.jmir.org/2019/4/e12641/ sample size misleading statements

In “Designing robust N-of-1 studies for precision medicine: Simulation study and design recommendations” by Percha et al [1], the authors use misleading language when speaking about the required numbers of samples regarding results in Figure 4a. For example, they write on page 8:

In Figure 4a, we see that for effect sizes of 0.1, 0.2, and 0.3, more than 100 samples are needed to obtain a power of 0.8 (at a standard 5% significance level). For an effect size of 0.4, at least 100 samples are needed. For effect sizes of 0.5, 0.6, 0.7, 0.8, 0.9, and 1.0, the numbers of samples needed to attain a power of 0.8 are approximately 65, 45, 35, 26, 21, and 18, respectively. [Figure 4]

Since Figure 4a is exactly equivalent to power curves from a two-sample, equal-variance t test (see Figure 1; generating R code provided in Textbox 1), the numbers of samples are for one of the two treatments; thus, the total numbers of samples are doubled. An easy fix in most instances of the unclear language is to add “per treatment” after “samples.” I provide a list of potential clarifying edits to the article’s text below (but may have missed some instances):

Figure 4c caption: “(ie, number of samples per treatment, with sampling rate fixed at 1 sample per time unit)”

Figure 4a and 4b: the label for the horizontal axis should be “Number of samples per treatment”

Page 8: “In Figure 4a, we see that for effect sizes of 0.1, 0.2, and 0.3, more than 100 samples per treatment are needed to obtain a power of 0.8 (at a standard 5% significance level). For an effect size of 0.4, at least 100 samples per treatment are needed. For effect sizes of 0.5, 0.6, 0.7, 0.8, 0.9, and 1.0, the numbers of samples per treatment needed to attain a power of 0.8 are approximately 65, 45, 35, 26, 21, and 18, respectively.”

Page 9: “For an effect size of 0.5 and σp=0.0, 0.4, 0.8, 1.2, 1.6, 2.0, the numbers of samples per treatment needed to obtain a power of 0.8 are 61, 76, 89, 111, 135, and 176, respectively. For an effect size of 1.0, the numbers of samples per treatment needed are 20, 24, 28, 34, 43, and 53, respectively.”

Page 9: “For σp=0.0, 0.4, 0.8, 1.2, 1.6, 2.0 and α=0.1, the numbers of samples per treatment required are 36, 64, 110, 174, 228, and 250, respectively. For α=10.0, the numbers of samples per treatment required are only 20, 23, 28, 34, 42, and 53, respectively.”

For effect sizes ranging from 0.1 to 1.0, power of a 0.05 level two-sample t test plotted by n, the number of samples in one treatment group. Total sample size is assumed to be 2n.

 R code.

#--- R code for generating Figure 1.

library(pwr)

color <- c("#F8766D", "#D89000", "#A3A500", "#39B600", "#00BF7D", "#00BFC4",

           "#00B0F6", "#9590FF", "#E76BF3", "#FF62BC")

 

#--- Producing the power plot (upper portion of figure)

plot(50, 1, type = 'n', xlim = c(0,100), ylim = c(0,1), axes = FALSE,

     ylab = substitute(paste("2-sample ", italic('t'), " test power")),

     xlab = substitute(paste(italic('n'), " = # of samples for 1 of 2 treatments") ) )

axis(1, at = seq(0, 100, 20)); axis(2, at = seq(0, 1, .2), las = 2)

abline(v = seq(10, 100, 10), col = 'gray80')

abline(h = seq(.1, 1, .1), col = 'gray85')

 

#--- Filling in the power plot

for(.d in 10:1/10){

   tmp.power <- NULL

   for( .n in 5:100){

     #--- Computes power for a 2-sample t-test, each sample with n observations.

     p <- pwr.t.test(n = .n, d = .d, sig.level = .05, power = NULL,

                     type = "two.sample", alternative = "two.sided")$power

     tmp.power <- c(tmp.power, p)

   }

   lines(5:100, tmp.power, lwd=3, col=color[.d*10])

}

 

#--- Producing the the legend (lower portion of figure)

plot(50, 1, type = 'n', xlim = c(0,100), ylim = c(.25, .75),

      axes = FALSE, xlab = 'Effect Size', ylab = '', cex.lab=1.5)

x <- 15

for(iter in seq(2,10,2)){

   mult <- (iter/2)

   .x <- x*mult

   segments( .x-3, 0.67, .x+3, lwd=5, col= color[iter-1])

   text(.x+3, .67, pos=4, (iter-1)/10, cex=1.25)

   segments( .x-3, 0.33, .x+3, lwd=5, col= color[iter] )

   text(.x+3, .33, pos=4, (iter)/10, cex=1.25)

}

Percha and colleagues have agreed to the above changes; these changes have been made to the original paper.

None declared.

 Percha Bethany Baskerville Edward B Johnson Matthew Dudley Joel T Zimmerman Noah Designing Robust N-of-1 Studies for Precision Medicine: Simulation Study and Design Recommendations J Med Internet Res 2019 04 01 21 4 e12641 10.2196/12641 30932871 v21i4e12641 PMC6462889