Letter to the Editor
Comment in: https://www.jmir.org/2020/10/e24084
We read the article “Facebook as a Novel Tool for Continuous Professional Education on Dementia: Pilot Randomized Controlled Trial” by Chan et al  with great interest. The idea that face to face education is difficult and education via the internet or social networking systems is necessary is intriguing; this is an important perspective in the midst of the COVID-19 pandemic. In the article, the editor wrote, “readers are advised to carefully assess the validity of any potential explicit or implicit claims related to primary outcomes or effectiveness”; hence, we would like to discuss some perspectives.
First, the primary outcome was measured using the differences in the scores of the pre- and postintervention knowledge assessments, which comprised the 25-item Dementia Knowledge Assessment Scale (DKAS) and a formative evaluation of 20 multiple-choice questions. However, the authors’ conclusion is focused on the outcome of improving participants’ knowledge concerning a single subscale in DKAS. This interpretation might be a spin that could warp the interpretation of results and mislead readers .
Second, the article has issues with multiple testing. When tests are divided into subscales, some of them may have significant differences. To show a significant difference in the effect, corrections to the multiple tests are required .
Finally, this study is a pre and post study; therefore, a paired t test should be used instead of a two-sample t test. The two-sample t test estimates the treatment effect using only the responses at follow-up, and it does not use any information at baseline, which may be useful for increasing efficiency if the baseline and follow-up outcomes are correlated .
Conflicts of Interest
- Chan WS, Leung AY. Facebook as a Novel Tool for Continuous Professional Education on Dementia: Pilot Randomized Controlled Trial. J Med Internet Res 2020 Jun 02;22(6):e16772 [FREE Full text] [CrossRef] [Medline]
- Boutron I, Dutton S, Ravaud P, Altman DG. Reporting and interpretation of randomized controlled trials with statistically nonsignificant results for primary outcomes. JAMA 2010 May 26;303(20):2058-2064. [CrossRef] [Medline]
- Li G, Taljaard M, Van den Heuvel ER, Levine MA, Cook DJ, Wells GA, et al. An introduction to multiplicity issues in clinical trials: the what, why, when and how. Int J Epidemiol 2017 Apr 01;46(2):746-755. [CrossRef] [Medline]
- Yang L, Tsiatis AA. Efficiency Study of Estimators for a Treatment Effect in a Pretest–Posttest Trial. The American Statistician 2001 Nov;55(4):314-321. [CrossRef]
|DKAS: Dementia Knowledge Assessment Scale|
Edited by T Derrick, G Eysenbach; submitted 30.06.20; peer-reviewed by WSY Chan; accepted 01.10.20; published 30.10.20Copyright
©Yusuke Saishoji, Akihiro Shiroshita, Yasushi Tsujimoto. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 30.10.2020.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.