This paper is in the following e-collection/theme issue:

Research Instruments, Questionnaires, and Tools (586) Dementia and Cognitive Decline (500) Methods and New Tools in Mental Health Research (461) Aging in Place (297)

Published on in Vol 25 (2023)

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/40858, first published .
An Effective Test (EOmciSS) for Screening Older Adults With Mild Cognitive Impairment in a Community Setting: Development and Validation Study

An Effective Test (EOmciSS) for Screening Older Adults With Mild Cognitive Impairment in a Community Setting: Development and Validation Study

An Effective Test (EOmciSS) for Screening Older Adults With Mild Cognitive Impairment in a Community Setting: Development and Validation Study

Authors of this article:

Jingsong Wu1, 2 Author Orcid Image ;   Jingnan Tu1, 2 Author Orcid Image ;   Zhizhen Liu2 Author Orcid Image ;   Lei Cao1, 2 Author Orcid Image ;   Youze He1, 2 Author Orcid Image ;   Jia Huang1 Author Orcid Image ;   Jing Tao3, 4 Author Orcid Image ;   Mabel N K Wong5, 6 Author Orcid Image ;   Lidian Chen3, 4 Author Orcid Image ;   Tatia M C Lee7 Author Orcid Image ;   Chetwyn C H Chan5 Author Orcid Image
Article Authors Cited by Tweetations (6) Metrics

    Original Paper

    1College of Rehabilitation Medicine, Fujian University of Traditional Chinese Medicine, Fuzhou, China

    2The Academy of Rehabilitation Industry, Fujian University of Traditional Chinese Medicine, Fuzhou, China

    3National-Local Joint Engineering Research Center of Rehabilitation Medicine Technology, Fujian University of Traditional Chinese Medicine, Fuzhou, China

    4Fujian Key Laboratory of Rehabilitation Technology, Fujian University of Traditional Chinese Medicine, Fuzhou, China

    5Department of Psychology, The Education University of Hong Kong, Hong Kong, Hong Kong

    6Department of Rehabilitation Sciences, The Hong Kong Polytechnic University, Hong Kong, Hong Kong

    7Laboratory of Neuropsychology, The University of Hong Kong, Hong Kong, Hong Kong

    *these authors contributed equally

    Corresponding Author:

    Chetwyn C H Chan, PhD

    Department of Psychology

    The Education University of Hong Kong

    Tai Po Luk Ping Road, New Territories Hong Kong

    Hong Kong, 999077

    Hong Kong

    Phone: 852 6318 8991

    Email: cchchan@eduhk.hk


    Background: Early detection of mild cognitive impairment (MCI) symptoms is an important step to its diagnosis and intervention. We developed a new screening test called “Efficient Online MCI Screening System” (EOmciSS) for use in community-dwelling older adults. It is a self-paced cognitive test to be completed within 10 minutes on tablets or smartphones in homes or care centers for older adults.

    Objective: This study aims to test the validity of EOmciSS for identifying community-dwelling older adults with MCI risks.

    Methods: Participants (N=827) completed EOmciSS and other screening tests for MCI. The psychometric properties tested were “subscale item difficulty,” “discriminative index,” “internal consistency,” and “construct validity.” We also tested between-group discrimination using the cross-validation method in an MCI group and a normal cognitive function (NCF) group.

    Results: A total of 3 accuracy factors and 1 reaction time factor explained the structure of the 20 item factors. The difficulty level of accuracy factors (ie, “trail making,” “clock drawing,” “cube copying,” “delayed recall”) was 0.63-0.99, whereas that of the reaction time factor was 0.77-0.95. The discriminative index of the medium-to-high-difficulty item factors was 0.39-0.97. The internal consistency (Cronbach α) ranged from .41 (for few item factors) to .96. The training data set contained 9 item factors (CC-Acc1, P<.001; CD-Acc1, P=.07; CD-Acc2, P=.06; CD-Acc3, P<.001; TM-Acc4, P=.07; DR-Acc1, P=.03; RS, P=.06; DR-RT1, P=.02; and DR-RT2, P=.05) that were significant predictors for an MCI classification versus NCF classification. Depressive symptoms were identified as significant factors (P<.001) influencing the performance of participants, and were an integral part of our test system. Age (P=.15), number of years of education (P=.18), and proficiency in using an electronic device (P=.39) did not significantly influence the scores nor classification of participants. Application of the MCI/NCF cutoff score (7.90 out of 9.67) to the validation data set yielded an area under the curve of 0.912 (P<.001; 95% CI 0.868-0.955). The sensitivity was 84.9%, specificity was 85.1%, and the Youden index was 0.70.

    Conclusions: EOmciSS was valid and reliable for identifying older adults with significant risks of MCI. Our results indicate that EOmciSS has higher sensitivity and specificity than those of the Computer-Administered Neuropsychological Screen for Mild Cognitive Impairment and the Computerized Cognitive Screen. The user interface, online operation, and self-paced format allowed the test system to be operated by older adults or their caregivers in different settings (eg, home or care centers for older adults). Depressive symptoms should be an integral part in future MCI screening systems because they influence the test performance and, hence, MCI risk.

    Trial Registration: Chinese Clinical Trial Registry ChiCTR2000039411; http://www.chictr.org.cn/showprojen.aspx?proj=62903

    J Med Internet Res 2023;25:e40858

    doi:10.2196/40858

    Keywords

    mild cognitive impairment; digital assessment; digital health; community dwelling; screening test; older adults; aging 


    Approximately 50 million people worldwide suffer from dementia, and this number is expected to increase to 152 million by 2050 [1]. Among older adults (≥60 years), the prevalence of dementia is about 5%-7% [2], which brings heavy burdens to families, society, and health care systems [3]. Studies have shown that early detection of cognitive impairment and intervention can delay the progression from mild cognitive impairment (MCI) to dementia [4].

    Early detection of MCI symptoms in the community-dwelling older population is the first step of this endeavor [5]. Studies have reported comparable validity between paper-and-pencil methods and computerized methods in MCI screening [6].

    Screening tests can be in various forms: self-evaluation, evaluation by an examiner, or evaluation by a caregiver. Self-rating tests rely on self-reporting by the individual [7], and an example is the Ascertain Dementia-8 (AD8) questionnaire. The AD8 questionnaire has been reported to have an internal consistency of .66 (Cronbach α), with a sensitivity of 80% and specificity of 59% [8]. The questionnaire takes about 3 minutes to complete and is used mainly to screen dementia rather than MCI risk [9,10].

    Performance tests involve the administration and scoring of the test by a medical professional. An example is the Montreal Cognitive Assessment (MoCA). The internal consistency of the MoCA is .83 (Cronbach α) [11]. A systematic review reported that the MoCA had a sensitivity of 83.9% and specificity of 74.6% for distinguishing individuals with MCI from those who were considered to have normal cognitive function (NCF). The time needed to administer the MoCA is 10–15 minutes [12]. Scholars have commented that the MoCA score is affected by the education level of the individual being tested [13] and that the score should be rated by a medical professional [14].

    The General Practitioner Assessment of Cognition (GPCOG) uses the caregiver as the informant in addition to professional scoring. The internal consistency of the GPCOG has been reported to be .84 (Cronbach α), with a sensitivity of 82% and specificity of 83% [15]. The GPCOG takes less than 10 minutes to administer [16-18].

    Apart from paper-and-pencil tests, computerized screening tests or digital screening tests take advantage of information technology and the portability of service delivery [19,20]. For instance, the Computer-Administered Neuropsychological Screen for Mild Cognitive Impairment (CANS-MCI) [21] covers comprehensive cognitive functions. It had a sensitivity of 81% and specificity of 73% for differentiating MCI from individuals with NCF. The CANS-MCI takes about 30 minutes to complete. By contrast, the Computerized Cognitive Screen (CoCoSc) [22] requires 15 minutes for completion, but has a lower ability (78% sensitivity and 69% specificity) to differentiate individuals with impaired cognition from those with normal cognition. One crucial factor that may confound individuals’ performance is their emotional state, particularly a depressive mood. The latter has been found to be a common comorbidity of MCI [23]. Depressive symptoms (eg, lack of motivation, slowness to respond) would contribute to the false-positive rate of test results [24-26]. However, the test of a depressive mood is not a feature in the aforementioned computerized screening tests.

    Given the aforesaid unmet needs and shortcomings, we developed a new screening test called “Efficient Online MCI Screening System” (EOmciSS) for use by community-dwelling older adults. First, we selected test items of relatively high sensitivity and high specificity for MCI. “cube copying” (CC) and “clock drawing” (CD) have shown a sensitivity of 65.6% and 57.1% and specificity of 53.3% and 70.0%, respectively, for individuals with MCI versus individuals with NCF [27-29]. The “trail making” (TM) item has yielded a sensitivity of 51.8% and specificity of 80.2% for differentiating individuals with MCI from those with NCF [28]. The “reaction speed” (RS) item [30] has shown an accuracy (Acc) of 80.6% for individuals with MCI versus individuals with dementia. The “delayed recall” (DR) item has been shown to have a sensitivity of 83% and specificity of 65% for MCI versus Alzheimer disease in older adults [31]. For a depressive mood, the 15-item Geriatric Depression Scale (GDS-15; cutoff score ≥8 points) has been shown to yield an internal consistency of .793 (Cronbach α) in an older population [32].

    EOmciSS has certain features that cater to the needs and self-paced operations of older users. For example, test instructions are delivered audially and visually simultaneously. Cross-audiovisual presentations can enhance the attention and, hence, the learning of users to understand the operation of test tasks [33]. The locations and times of all responses (eg, TM, CD, CC items) are recorded by pressing a touchscreen and can improve the prediction model.

    We aimed to gather evidence of the validity of EOmciSS for identifying community-dwelling older adults with MCI risks. It is a self-paced, rapid, cognitive screening test for users that can be completed within 10 minutes on tablets or smartphones in homes or care centers. We hypothesized that the test items of EOmciSS would contribute significantly to identify individuals with a high risk of MCI, and that the sensitivity and specificity of EOmciSS would be comparable with those of other paper-and-pencil tests for MCI screening.


    Inclusion Criteria

    The inclusion criteria were as follows: (1) age between 55 and 75 years; (2) community dwelling and attending an activity center regularly; (3) had received education for more than 6 years [34]; and (4) having normal vision and hearing with/without corrective devices.

    Exclusion Criteria

    The exclusion criteria were taking medications (eg, antidepressant, sedative) in the previous 2 weeks or a known history of (1) brain trauma, brain tumor, cerebral infarction, cerebral hemorrhage, intracranial infection, Parkinson, epilepsy, or other neurological diseases; (2) depression, anxiety, schizophrenia, or other mental illnesses; (3) abuse of alcohol or other substances.

    Participants

    Based on convenience sampling, participants (older adults) were recruited from 32 community centers for older adults in a city in China from June 2019 to December 2020.

    A total of 1081 participants were recruited and completed testing. Among them, 196 were verified to have received ≤6 years of education. Another 58 participants were found to have excessive amounts of missing data. The final sample size entering the analyses was 827 (Multimedia Appendix 1).

    Classification of the MCI Versus NC Groups

    Classification of participants in the MCI group was based on the diagnostic criteria published by Petersen [35] and the MCI guidelines set by the American Academy of Neurology in 2018 [36]. There were 4 criteria. The first criterion was subjective reporting of memory deficits by the participants or their informant. The second criterion was a score of 19-24 on the Chinese (Fuzhou) version of the MoCA (C-MoCA). This is an objective measure [34,37] that indicates MCI. The cutoff score of 24 (as compared with 26 in the original MoCA [11]) has been validated for use among the Chinese population. The third criterion was a score greater than 2 minus standard deviations (ie, > –2SDs) on the Lawton Instrumental Activities of Daily Living (IADL) scale [38], which indicates the intact ability of the activities of daily living. The fourth criterion was 1 point or less on the AD8 questionnaire [8], which excludes the possibility of dementia. The criteria for classifying participants in the NCF group were a score of 25 or over on the C-MoCA and 1 point or less on the AD8 questionnaire, and greater than −2SDs on the IADL scale. According to these criteria, 579 participants belonged to the NCF group and 248 to the MCI group.

    We adopted the diagnostic criteria stipulated by the National Institute on Aging-Alzheimer’s Association [39] to exclude participants who showed dementia symptoms. The criteria were (1) score of 18 or less on the C-MoCA [37]; (2) less than −2SDs on the IADL scale [38]; and (3) 2 points or more on the AD8 questionnaire [8].

    Procedures

    Research assistants helped participants to access EOmciSS using a tablet. Participants began to complete the test with assistance from the instructions and sample items provided by EOmciSS. Research assistants did not aid the participants at any point during the test. Each participant yielded a set of EOmciSS item scores and a machine-generated total score. After completing EOmciSS, participants were administered a series of criterion tests: MoCA, AD8, and IADL. These tests were administered by a different group of research assistants who received training on delivering the 3 tests using standardized procedures. Participants were also assessed by physicians within our research team who had the experience of diagnosing MCI in their clinical practices.

    EOmciSS

    There are 4 main features in the conceptualization of EOmciSS that facilitate large-scale screening in a community setting. First, the composition of the test contents takes an eclectic approach. The constructs of all the test items have been supported with sufficient evidence for differentiating individuals with MCI from individuals with NCF. These are “depressive mood,” “visual attention and flexibility,” “visuospatial and executive functions,” “memory and cognitive processing speed.” Second, the user interface is simple and allows for self-administration and the test to be completed at different times in different places. Third, the time required for most users to complete the test should be 10 minutes or less. Fourth, administration of the test and scoring are automated.

    EOmciSS has 2 sections delivered in sequence (Figure 1). The first section is the first-level screen, which requires participants to complete the GDS-15 [40] for depressive symptoms. The Chinese version of the GDS-15 has 15 items, and a positive response to 1 item scores 1 point. The total score is 15 points, with 8 points or more [41,42] being the cutoff for indicating a possible depressive mood. Individuals who score 8 points or more are advised to consult medical practitioners, and the assessment is terminated. The aim of the first-level screen is to prevent depressive symptoms (eg, slow processing time) to confound the results of the subsequent items in the cognitive test. Participants who pass the screening for depressive symptoms proceed to the second-level screen, which is the cognitive test (Figure 2).

    The cognitive test has 5 subtests that contain 20 test items (Figure 1). The 20 test items comprise 14 Acc and 6 reaction time (RT) measures. The 5 subtests are the TM (8 items; 6 Acc + 1 RT measure, and 1 order), CD (4 items; 3 Acc + 1 RT measure), CC (2 items; 1 Acc + 1 RT measure), DR (5 items; 3 Acc + 2 RT measures), and RS (1 item; 1 RT measure). In each subtest, participants view the information and instructions of the test procedures in synchronized visual and audio formats (Figure 3). There are also sample items to enable participants to familiarize themselves with the test procedure and response sets. Participants make responses by hitting the icons that appear on the touchscreen of a tablet, a smartphone, or another type of electronic device. Depending on the test items, responses are quantified according to Acc and RT (Multimedia Appendix 2).

    ‎
    Figure 1. Test contents of EOmciSS. The test contents of EOmciSS cover 5 subtypes with 20 test items. The subtypes are “depression,” “visual attention and mental flexibility,” “visuospatial and executive functions,” “memory,” and “cognitive processing speed.” EOmciSS: Efficient Online MCI Screening System; GDS-15: 15-item Geriatric Depression Scale.
    ‎
    Figure 2. EOmciSS workflow. The EOmciSS workflow starts from the “log-in system” to completion of the screening test with printing of the result. The entire workflow is to be completed within 10 minutes. EOmciSS: Efficient Online MCI Screening System; GDS-15: 15-item Geriatric Depression Scale.
    ‎
    Figure 3. Screenshot examples of EOmciSS. Screenshot examples of EOmciSS showing: (A) the login page, (B) a “depressive mood” item (GDS-15), (C) the demonstration page of a test item, (D) beginning of the “Trail Making” subscale, and (E) the printing page of test results. EOmciSS: Efficient Online MCI Screening System; GDS-15: 15-item Geriatric Depression Scale.

    External Criterion Measures

    A total of 3 measures were used as the external criteria for establishing evidence of the concurrent validity of EOmciSS. The C-MoCA [43] contains 11 subtests; 8 of them cover different cognitive domains, including “visuospatial and executive functions,” “naming,” “instantaneous memory (not scored),” “attention,” “language,” “abstraction,” “delayed recall,” and “orientation.” The AD8 questionnaire is a short test for dementia that is sensitive to early changes in cognitive function [44]. The cutoff scores of the MoCA and AD8 for MCI that we adopted are stated above. The IADL scale measures the independence of daily living [45]. It contains 8 items on the performance levels (0-4 points) in different aspects of managing daily living tasks. A higher score reflects a higher level of independence.

    Data Analyses

    Different methods were used to generate evidence on the validity of the 20 test items under EOmciSS subscales. At the test level, the structure of the 20 item factors (13 Acc + 6 RT + 1 order) was explored by confirmatory factor analysis. At item and subtest levels, “item difficulty,” “discriminative index,” and “internal consistency” were the indices. “Item difficulty” refers to the proportion of correct answers per item [46]. “Discriminative index” is the correlation between 1 item and its subtest score [47]. “Internal consistency” is Cronbach α computed for each of the 5 subtests.

    As suggested by Boateng et al [46], known-group differentiation was used to estimate the sensitivity and specificity of test items to differentiate participants with MCI from those with NCF. The known-group labels of MCI and NCF were based on participants’ scores on the MoCA, AD8, and IADL. The validity of between-group discrimination was tested further with cross validation [48] by dividing participants into 2 subsets: the MCI group and the NCF group. The 70% (579/827, 70.1%) subset was the training set. The 30% (248/827, 29.9%) subset was the validation set. Between-subset differences in demographic characteristics were tested with the Student t test, Spearman rank correlation test, or χ2 test. Multivariate binary logistic regression was applied to test the significance of the 20 item factors for predicting the membership of participants in the NCF group or the MCI group. The area under the curve (AUC) and Hosmer-Lemeshow goodness-of-fit test were used to describe the ability of the regression model to differentiate MCI from NCF. The regression model obtained from the training set was tested with the sample in the validation set. The β coefficients yielded became the item weights of each significant item factor for computing the total score. Predicted memberships were compared with observed memberships to derive the optimal cutoff score for MCI and NCF, as well as its sensitivity, specificity, Youden index, and receiver operating characteristic curve.

    Ethical Approval of the Study Protocol

    The Medical Ethics Committee of the Affiliated Rehabilitation Hospital of Fujian University of Traditional Chinese Medicine (Fuzhou, China) approved the study protocol (approval number 2019KY-002-02). The purposes of the study were explained to all participants, who provided written informed consent.


    Construct Validity

    A total of 20 item factors made up the performance Acc (n=14) and RT (n=6): TM-Acc (#1-6, #7 order), CD-Acc (#8-10), CC-Acc (#11), DR-Acc (#12-14), CC-RT (#15), CD-RT (#16), TM-RT (#17), DR-RT1 (#18), DR-RT2 (#19), and RS (#20). Confirmatory factor analysis revealed the Kaiser-Meyer-Olkin value to be 0.841, and the Bartlett test showed a significant difference (P<.001). Comparative fit index, standardized root-mean-squared residual, and root-mean-square error of approximation were 0.97, 0.007, and 0.048, respectively [49]. These results suggested a 4-factor structure for grouping the 20 item factors (Figure 4).

    The first factor appeared to cover the Acc of the responses on TM items, and was called the “TM factor.” The second factor appeared to account for the Acc of the responses on the CD and CC items, and was called the “CD+CC factor.” The third factor appeared to cover the Acc of the responses on the DR items, and was called the “DR factor.” The fourth factor covered the CC-RT, CD-RT, TM-RT, DR-RT1, DR-RT2, and RS items, and was called the “RT factor.” This factor structure appeared to be consistent with the constructs of visual attention and mental flexibility (TM), visuospatial and executive function (CD and CC), memory (DR), and cognitive processing speed (RS).

    The difficulty level of the items under the TM, CD+CC, DR, and RT subtests was 0.69-0.90, 0.63-0.99, 0.95-0.98, and 0.77-0.95, respectively. The corresponding discriminative index was 0.626-0.974, 0.224-0.886, 0.530-0.790, and 0.25-0.60 (Table 1). The internal consistency index (Cronbach α) was .959, .483, .473, and .411 for the TM factor, CD+CC factor, DR factor, and RT factor, respectively.

    ‎
    Figure 4. Path diagram showing the estimated parameter loadings from confirmatory factor analysis. A 4-factor structure is revealed on the accuracy, reaction time, and order factors for the 20 test items of EOmciSS. Acc: accuracy; CC: cube copying; CD: clock drawing; DR: delayed recall; EOmciSS: Efficient Online MCI Screening System; RT: reaction time; RS: reaction speed; TM: trail making.
    Table 1. Item difficulties (mean values) and discriminative indices (item total correlations) of the 20 item factors and internal consistencies (Cronbach α) of the 4 subscales of EOmciSSa.b
    EOmciSS item factorsItem difficultyDiscriminative index Internal consistency (Cronbach α)
    Trail making factor

    		.959

    		TMc-Acc10.900.626

    		TM-Acc20.830.797

    		TM-Acc30.760.905

    		TM-Acc40.750.917

    		TM-Acc50.700.968

    		TM-Acc60.710.957

    		TM-order-Acc0.690.974
    Clock drawing and cube copying factor

    		.483

    		CDd-Acc10.990.224

    		CD-Acc20.960.392

    		CD-Acc30.850.649

    		CC-Acc10.630.886
    Delayed recall factor

    		.473

    		DRe-Acc10.950.790

    		DR-Acc20.980.530

    		DR-Acc30.970.611

    		CCf-RT0.880.49

    		CD-RT0.880.49

    		TM-RT0.890.45

    		DR-RT10.950.25

    		DR-RT20.890.47
    Reaction time factor

    		.411

    		RSg0.770.60

    aEOmciSS: Efficient Online MCI Screening System.

    bThe 4 subscales were grouped based on the results of the confirmatory factor analysis.

    cTM: trail making.

    dCD: clock drawing.

    eDR: delayed recall.

    fCC: cube copying.

    gRS: reaction speed.

    Discriminative Validity (Training Set)

    There were 402 participants with NCF and 177 participants with MCI in the training set (Table 2). There were 175 participants with NCF and 73 participants with MCI in the validation set. The percentage of participants with MCI in the total sample for the training and validation sets was 30.6% (177/579) and 29.4% (73/248), respectively, and this difference was not significant (P=.74). In the training set, the MCI subgroup was significantly older (P=.048), had fewer years of education (P<.001), and spent less time using an electronic device (P<.001) than the NCF subgroup (Table 2). Other between-subgroup demographic variables did not show significant differences (P=.68, .89, .12, .68, and .39 for sex, BMI, concomitant chronic disease, smoking, and drinking, respectively). Participants in the NCF subgroup showed a significantly higher mean score on the MoCA (P<.001) and a lower mean score on the GDS-15 (P<.001) than those in the MCI subgroup. Significant differences were not found in the mean score for the IADL scale (P=.24).

    Participants with NCF in the training set showed significantly higher Acc than those with MCI in all EOmciSS item factors (χ21=7.52-127.75, P<.001). Participants with NCF showed faster RT on correct items than those with MCI across all item factors (Z=5.28-7.33, P<.001).

    The 20 item factors were the predictor variables. Age, education, score on the GDS-15, and proficiency using an electronic device were covariates. The regression model contained 4 item factors (CC-Acc1, P<.001; CD-Acc3, P<.001; DR-Acc1, P=.03; and DR-RT1, P=.02) as the significant predictor at P<.05; 5 item factors were marginally significant predictors at P=.05-.07 (Table 3). The score on the GDS-15 was the only significant (P<.001) covariate in the model. The AUC of the MCI prediction model versus the NCF prediction model was 0.908 (95% CI 0.881-0.934; P<.001), which was over 0.75 and suggested satisfactory discriminative power. The Hosmer-Lemeshow goodness-of-fit test also indicated good data for model fitting (χ28=11.54, P=.17). Weighted scores of the 9 item factors were derived based on the β coefficients [25] obtained from the logistic regression model. The inclusion of factors CD-Acc1 (P=.07), CD-Acc2 (P=.06), TM-Acc4 (P=.07), RS (P=.06), and DR-RT2 (P=.05), despite their marginal significance in the regression model, was to strengthen content representativeness of EOmciSS (Table 4).

    Table 2. Comparison of demographic characteristics of participants in the mild cognitive impairment and normal cognition function subsets in the training set (579/827, 70% of the data set).
    CharacteristicsNormal cognition function subset (n=402)Mild cognitive impairment subset (n=177)χ2 (df)/ZP value
    Age (year)a64.5 (60.00-70.00)66.00 (62.00-71.00)1.974 (1).048
    Sex,n (%)

    		0.170 (1).68

    		Male148 (36.8)62 (35.0)

    Female254 (63.2)115 (65.0)

    Education (year)a12.00 (9.00-14.00)9.00 (9.00-12.00)–6.818 (1)<.001
    BMI (kg/m2)a23.30 (21.48-25.59)23.74 (21.70-25.29)0.137 (1).89
    Concomitant chronic disease,n (%)

    		4.130 (2).12

    		Nil205 (51.0)78 (44.1)

    n=1140 (34.8)63 (35.6)
    n≥257 (14.2)36 (20.3)
    Smoking,n (%)

    		0.176 (1).68

    		Yes30 (7.5)15 (8. 5)

    No372 (92.5)162 (91.5)
    Drinking,n (%)

    		0.743 (1).39

    		Yes78 (19.4)29 (16.4)

    No324 (80.6)148 (83.6)
    Proficiency using electronic devices, n (%)b

    		23.156 (1)<.001

    		Yes334 (83.1)115 (65.0)

    No68 (16.9)62 (35.0)

    aA 2-sample rank test was used because the data did not conform to a normal distribution. Median values (25th percentile-75th percentile) instead of mean values were used to describe results.

    bThe cutoff for yes/no was the average time of using an electronic device (eg, smartphone) for 1 hour or more/day.

    Table 3. Logistic regression on EOmciSSa item factors predicting membership of the NCFb group or the MCIc group in the training set (579/827, 70% of the data set).
    Item factorsRegression coefficient (β)Odds ratio (95% CI)P value
    CCd-Acce11.5904.904 (2.776-8.644) <.001f
    CDg-Acc11.6635.273 (0.898-30.96).066h
    CD-Acc21.1773.244 (0.956-11.007).06h
    CD-Acc31.3313.786 (1.827-7.845)<.001f
    TMi-Acc1.4881.629 (0.486-5.461).43
    TM-Acc2.6551.926 (0.583-6.362).28
    TM-Acc3–.1970.821 (0.136-4.944).83
    TM-Acc41.8786.541 (0.856-49.988).07h
    TM-Acc520.442—j.10
    TM-Acc61.7765.903 (0.0477-43.469).47
    TM-OrderAcc–21.735—.10
    DRk-Acc11.4244.155 (1.201-14.376).03f
    DR-Acc2.5221.685 (0.242-11.731).60
    DR-Acc3.7482.114 (0.345-12.947) .42
    RSl.5691.766 (0.973-3.207).06h
    CC-RTm<.0011 (0.996-1.003).92
    CD-RT.0011.001 (0.994-1.008).84
    TM-RT.0021.002 (0.995-1.01).56
    DR-RT1.0131.014 (1.002-1.025).02f
    DR-RT2.0261.026 (1.000-1.053) .05h
    Proficiency using devicen.2931.34 (0.688-2.609).39
    Age.0361.036 (0.987-1.089).15
    Years of education–.0700.933 (0.842-1.033).18
    GDS-15o.2451.278 (1.117-1.461)<.001f

    aEOmciSS: Efficient Online MCI Screening System.

    bNCF: normal cognitive function.

    cMCI: mild cognitive impairment.

    dCC: cube copying.

    eAcc: accuracy.

    fP<.05.

    gCD: clock drawing.

    hP=.05-.07.

    iTM: trail making.

    jNot applicable.

    kDR: delayed recall.

    lRS: reaction speed.

    mRT: reaction time.

    n“Proficiency using device” is a measure of proficiency of using an electronic device.

    oGDS-15: 15-item Geriatric Depression Scale.

    Table 4. Weighted scores of significant EOmciSSa item factors based on the β coefficient values of the regression model for classification of participants with NCFb and participants with MCIc in the training set (579/827, 70% of the data set).d
    ItemsRegression coefficient (β)P valueWeighted scorese
    CCf-Accg11.590<.0011.59
    CDh-Acc11.663.071.66
    CD-Acc21.177.061.18
    CD-Acc31.331<.0011.33
    TMi-Acc41.878.071.88
    DRj-Acc11.424.031.42
    RSk.569.060.57
    DR-RTl1.013.020.01
    DR-RT2.026.050.03

    aEOmciSS: Efficient Online MCI Screening System.

    bNCF: normal cognitive function.

    cMCI: mild cognitive impairment.

    dSignificance threshold was set at P≤.07 to maximize the content of the 9 item factors for EOmciSS.

    eTotal weighted score is 9.67.

    fCC: cube copying.

    gAcc: accuracy.

    hCD: clock drawing.

    iTM: trail making.

    jDR: delayed recall.

    kRS: reaction speed.

    lRT: reaction time.

    Verification of Discriminative Validity (Validation Set)

    The validation set comprised 248 participants. Significant between-subgroup differences were in age (P=.04) and number of years of education (P<.001). Use of an electronic device showed marginal significance in subgroup differences (P=.07). The MCI subgroup (2/73, 2.7%) had significantly fewer participants reporting a history of tobacco smoking than the NCF group (20/175, 11.4%; P=.03). Participants in the NCF subgroup showed a significantly higher mean score on the MoCA (P<.001) and a lower mean score on the GDS-15 (P=.01) than those in the MCI subgroup. Significant between-subgroup differences were not found for the mean score on the IADL scale (P=.34). With the same parameters derived from the training set, the AUC for the classification of the NCF and MCI subsets was 0.912 (P<.001; 95% CI 0.868-0.955). The cutoff score of 7.90 yielded 84.9% sensitivity, 85.1% specificity, and Youden index of 0.70 for the validation set. The receiver operating characteristic curves of the validation set are presented in Figure 5.

    ‎
    Figure 5. ROC curves. ROC curves for the 9 EOmciSS item factors differentiating participants with MCI from those with NCF in the validation data set. AUC for the validation set was 0.912 (95% CI 0.868-0.955; P<.001). AUC: area under the curve; EOmciSS: Efficient Online MCI Screening System; MCI: mild cognitive impairment; NCF: normal cognitive function; ROC: receiver operating characteristic.

    Expected Findings

    We aimed to reveal the psychometric properties of EOmciSS, a newly designed, rapid, digital cognitive test for screening of individuals with MCI in community settings. The ability of EOmciSS to differentiate individuals with MCI risks from those with NCF was found to be higher than that of other online systems in this research field. Essential content domains for the MCI classification versus the NCF classification were mainly in the Acc of the performances in the order of visuospatial attention, planning, and executive function (in CC and CD); cognitive flexibility and working memory (in TM); and verbal memory (in DR). In the TM subtest, the connection from the third-to-fourth alternative symbols contributed the most to the classification model. Besides, the speed of performances in terms of RT (in RS) and recall of memory items (in DR) significantly predicted membership of the MCI group or the NCF group. Incorporation of a “depressive mood” measure in the first section of EOmciSS is a new design for a cognitive screening test. Evidence suggests that individuals with a depressive mood are likely to have low motivation and delays in executing actions, which would pose threats to the internal validity of cognitive test performances (eg, lower Acc or longer RT). For EOmciSS, the impacts would be on the measures of Acc and RT. The identification of users of EOmciSS with the GDS-15 before them entering into the cognitive test section would safeguard the validity of the cognitive screening for MCI.

    The test construct of EOmciSS was substantiated with results from the confirmatory factor analysis. The 20 item factors were clustered satisfactorily into 3 latent Acc factors with respect to the test content of visual attention and mental flexibility (TM), visuospatial and executive function (CD and CC), memory (DR), and cognitive processing speed (RS). The Acc of the 3 CD items and 1 CC item was grouped under 1 single latent factor. For CD, the 3 Acc item factors were “placing the 12 numbers (ie, “1” to “12”) on the face of a clock” (#CD-Acc1), “orienting the hour hand” (#CD-Acc2), and “orienting the minute hand” (#CD-Acc3). The CD item had 1 Acc item factor: “drawing the cube.” Within CD items, the “orienting the minute hand” Acc item factor was the most difficult (mean 0.85), but it was less difficult than the “draw the cube” item factor (mean 0.63). CD and CC are tests for visuospatial, planning, and executive functions, which have been reported to be effective for screening for individuals with MCI [50,51]. Therefore, findings by EOmciSS are consistent with those reported previously.

    We emphasized the Acc and speed of participants’ performances when designing EOmciSS. EOmciSS has made possible the capture of both performance parameters at a response level rather than at an item level. For instance, CD has 3 Acc factors and 1 RT factor. The definitions for the 4 item factors are different from those in the Digital Clock Drawing study, which quantifies the size, angle, and spacing between the clock elements, and their latencies [52]. In this study, “drawing numbers” involved dragging 12 numbered icons to the designated positions on a clock face; and the “drawing hands to 10 minutes after 11 o’clock” involved dragging the hands from 12 o’clock to the specified time. Only the Acc of the 3 parts of the drawing (and not the RT) was the significant predictor of membership of the MCI group versus the NCF group. Correctly “orienting the minute hand” was found to be the most significant predictor of classification of MCI versus NCF. Our finding is consistent with the data from 2 studies (1 conventional and 1 digital) which reported that correct placement of the “minute arm” was the best predictor of amnesia in individuals with MCI [29,53]. Such comparable results support the adoption of the 3 digital CD Acc item factors in EOmciSS.

    For the CC item, Oonuma et al [54] reported that 31.1% of individuals with normal cognition failed to complete the test correctly. The difficult level revealed for the single CC item in this study suggested that 37% (306/827) of the participants failed to complete the task. The ratio of failure to complete CC of participants with MCI to participants with NCF was around 5:1 (odds ratio 4.9). Such comparable results support the adoption of the design and format of the 1 CC Acc item factor in EOmciSS.

    The TM items, despite their marginal significance in the prediction model, appeared to have a compatibly important role as those of CD and CC. Interestingly, among the 6 factors, the “third-to-fourth symbol connection” Acc item factor (#TM-Acc4) was the best discriminative index. The “third-to-fourth symbol connection” was the only TM item to be included in the MCI versus NCF prediction model. It yielded the largest β coefficient and, therefore, was assigned with the heaviest weight contributing to the EOmciSS score. The design of the TM item is comparable with the conventional Trail Making Test [28,55]. A previous study suggested that the Trail Making Test measures task switching and working memory [56]. The correct connection of the fourth symbol from the start of the task would demand both functions. As expected, the Acc and speed of the DR items were significant predictors of membership of the MCI group versus the NCF group. The word “fruit” yielded the highest discriminative index (0.79) among the 3 words. We speculated that this superior discriminative power may be due to learning a new task rule (ie, encoding words and keeping them in mind) and learning the first word in a 3-word series. The learning involved would have placed substantial demand on the memory system of individuals with MCI. Our speculation is supported by the findings of studies on patients with MCI who showed deficits in DR of a series of items [57,58].

    About 90% (745/827) of participants completed EOmciSS in 10 minutes. This short time is a promising factor for maximizing the number of community-dwelling older adults who complete the test. Compared with other computerized tests, the CoCoSc system used by Wong et al [22] had 5 subtests, but the time taken to complete the test was 15 minutes. The CoCoSc system has a lower ability (78% sensitivity and 69% specificity) for screening MCI and NCF. Groppell et al [59] developed the BrainCheck Memory test. It has 6 subtests and takes about 21 minutes to complete. The sensitivity of the BrainCheck Memory test is 89% and specificity is 78%. By comparison, EOmciSS required a shorter time for completion but had relatively promising sensitivity (84.9%) and specificity (85.1%) for screening of MCI and NCF.

    Our study had 3 main limitations. First, participants were older adults who had been educated for more than 6 years. Thus, our results may not be generalizable to those who have only a few years of education or who are illiterate. We should explore possible adaptations of test content and a user interface that is appropriate for these groups. Second, compared with other computerized screening tests, EOmciSS could be used to differentiate individuals with MCI from individuals with NCF. However, the content representativeness of EOmciSS was limited. Therefore, EOmciSS data should be interpreted as indicating the risks, but not the diagnosis, of MCI. Additional neuropsychological and medical tests are recommended for reaching a clinical diagnosis. Third, we used a cross-sectional method to gather evidence, which hampers the repeatability and stability of test results. We must investigate the robustness of the test scores and classification results by using a longitudinal design and analytic methods, as described by Lei and colleagues [60].

    Conclusions

    EOmciSS is a new, computerized, cognitive screening test designed for use by community-dwelling older adults. The test content, user interface, and administration method enable older adults to complete the test with a smartphone or tablet within 10 minutes. The test constructs of EOmciSS can be factored into Acc and RT components, both of which contributed significantly to the ability of EOmciSS to differentiate individuals with MCI risks from those with NCF. The screening ability of EOmciSS was revealed to be the strongest among other computerized tests, with comparable test constructs and time needed for completion. EOmciSS could serve as a first-line test for identifying individuals with MCI risks in the community. Application of EOmciSS could enable rapid, inexpensive, large-scale screening among older adults. Individuals who receive positive screening results should be followed up with more comprehensive neuropsychological and medical tests before confirmation of an MCI diagnosis.

    Acknowledgments

    We thank all participants in the study as well as research staff for coordinating the research work and data collection. This study was funded by the Central Government Guides Local Science and Technology Development Fund Projects of Fujian Province (2020L3013) and the Peter T. C. Lee endowed professorship awarded to CCHC. Special thanks to Xiehua Xue and Sufang Cai for their support in assessing the participants for mild cognitive impairment diagnosis.

    Authors' Contributions

    JW, J Tu, and ZL equally contributed to this work. CCHC, TMCL, and LC are the co-corresponding authors. JW, J Tu, ZL were responsible for data collection, data analysis, and writing. LC and YH took care of data curation and writing the manuscript. JH and J Tao were responsible for the methodology, review, and writing of the manuscript. CCHC, TMCL, and LC contributed to funding acquisition, manuscript review and editing. All authors have read and approved the final version of the manuscript.

    Conflicts of Interest

    This study was sponsored by Amity Brain Health and A.I. Technology Company Limited (which develops EOmciSS) and CCHC and TMCL are paid research consultants for these organizations.

    ‎
    Multimedia Appendix 1

    Descriptions of the Participants.

    DOCX File , 42 KB
    ‎
    Multimedia Appendix 2

    Descriptions of test content of EOmciSS (Efficient Online MCI Screening System).

    DOCX File , 21 KB
    1. Livingston G, Huntley J, Sommerlad A, Ames D, Ballard C, Banerjee S, et al. Dementia prevention, intervention, and care: 2020 report of the Lancet Commission. The Lancet 2020 Aug;396(10248):413-446. [CrossRef]
    2. Prince M, Bryce R, Albanese E, Wimo A, Ribeiro W, Ferri CP. The global prevalence of dementia: a systematic review and metaanalysis. Alzheimers Dement 2013 Jan;9(1):63-75.e2. [CrossRef] [Medline]
    3. Zhou M, Wang H, Zeng X, Yin P, Zhu J, Chen W, et al. Mortality, morbidity, and risk factors in China and its provinces, 1990–2017: a systematic analysis for the Global Burden of Disease Study 2017. The Lancet 2019 Sep;394(10204):1145-1158. [CrossRef]
    4. Ganguli M, Dodge HH, Shen C, DeKosky ST. Mild cognitive impairment, amnestic type: An epidemiologic study. Neurology 2004 Jul 12;63(1):115-121. [CrossRef]
    5. Patnode CD, Perdue LA, Rossom RC, Rushkin MC, Redmond N, Thomas RG, et al. Screening for Cognitive Impairment in Older Adults: Updated Evidence Report and Systematic Review for the US Preventive Services Task Force. JAMA 2020 Feb 25;323(8):764-785. [CrossRef] [Medline]
    6. Chan JY, Kwong JS, Wong A, Kwok TC, Tsoi KK. Comparison of Computerized and Paper-and-Pencil Memory Tests in Detection of Mild Cognitive Impairment and Dementia: A Systematic Review and Meta-analysis of Diagnostic Studies. J Am Med Dir Assoc 2018 Sep;19(9):748-756.e5. [CrossRef] [Medline]
    7. Passler J, Kennedy R, Crowe M, Clay O, Howard V, Cushman M, et al. The relationship of cognitive change over time to the self-reported Ascertain Dementia 8-item Questionnaire in a general population. Arch Clin Neuropsychol 2021 Feb 12;36(2):243-252 [FREE Full text] [CrossRef] [Medline]
    8. Galvin JE, Roe CM, Coats MA, Morris JC. Patient's rating of cognitive ability: using the AD8, a brief informant interview, as a self-rating tool to detect dementia. Arch Neurol 2007 May 01;64(5):725-730. [CrossRef] [Medline]
    9. Galvin JE, Roe CM, Xiong C, Morris JC. Validity and reliability of the AD8 informant interview in dementia. Neurology 2006 Dec 11;67(11):1942-1948. [CrossRef]
    10. Galvin JE, Roe CM, Morris JC. Evaluation of cognitive impairment in older adults: combining brief informant and performance measures. Arch Neurol 2007 May 01;64(5):718-724. [CrossRef] [Medline]
    11. Nasreddine ZS, Phillips NA, Bédirian V, Charbonneau S, Whitehead V, Collin I, et al. The Montreal Cognitive Assessment, MoCA: a brief screening tool for mild cognitive impairment. J Am Geriatr Soc 2005 Apr;53(4):695-699. [CrossRef] [Medline]
    12. De Roeck EE, De Deyn PP, Dierckx E, Engelborghs S. Brief cognitive screening instruments for early detection of Alzheimer's disease: a systematic review. Alzheimers Res Ther 2019 Feb 28;11(1):21 [FREE Full text] [CrossRef] [Medline]
    13. Hu J, Zhou W, Hu S, Huang M, Wei N, Qi H, et al. Cross-cultural difference and validation of the Chinese version of Montreal Cognitive Assessment in older adults residing in Eastern China: preliminary findings. Arch Gerontol Geriatr 2013 Jan;56(1):38-43. [CrossRef] [Medline]
    14. Rossetti HC, Lacritz LH, Hynan LS, Cullum CM, Van Wright A, Weiner MF. Montreal Cognitive Assessment Performance among Community-Dwelling African Americans. Arch Clin Neuropsychol 2017 Mar 01;32(2):238-244 [FREE Full text] [CrossRef] [Medline]
    15. Brodaty H, Pond D, Kemp NM, Luscombe G, Harding L, Berman K, et al. The GPCOG: a new screening test for dementia designed for general practice. J Am Geriatr Soc 2002 Mar;50(3):530-534. [CrossRef] [Medline]
    16. Li X, Xiao S, Fang Y, Zhu M, Wang T, Seeher K, et al. Validation of the General Practitioner Assessment of Cognition – Chinese version (GPCOG-C) in China. Int. Psychogeriatr 2013 Jul 09;25(10):1649-1657. [CrossRef]
    17. Brodaty H, Kemp NM, Low L. Characteristics of the GPCOG, a screening tool for cognitive impairment. Int J Geriatr Psychiatry 2004 Sep 26;19(9):870-874. [CrossRef] [Medline]
    18. Wojtowicz A, Larner A. General Practitioner Assessment of Cognition: use in primary care prior to memory clinic referral. Neurodegener Dis Manag 2015 Dec;5(6):505-510. [CrossRef] [Medline]
    19. Chan JYC, Wong A, Yiu B, Mok H, Lam P, Kwan P, et al. Electronic Cognitive Screen Technology for Screening Older Adults With Dementia and Mild Cognitive Impairment in a Community Setting: Development and Validation Study. J Med Internet Res 2020 Dec 18;22(12):e17332 [FREE Full text] [CrossRef] [Medline]
    20. Defranceso M, Schocke M, Messner H, Deisenhammer E, Hinterhuber H, Marksteiner J, et al. Konversion von MCI (Mild Cognitive Impairment) zur Alzheimer-Demenz: Diagnostische Möglichkeiten und Prädikatoren. Neuropsychiatrie: Klinik, Diagnostik, Therapie und Rehabilitation: Organ der Gesellschaft Österreichischer Nervenärzte und Psychiater 2010;24(2):88-98. [CrossRef]
    21. Memória CM, Yassuda MS, Nakano EY, Forlenza OV. Contributions of the Computer-Administered Neuropsychological Screen for Mild Cognitive Impairment (CANS-MCI) for the diagnosis of MCI in Brazil. Int Psychogeriatr 2014 May 07:1-9. [CrossRef] [Medline]
    22. Wong A, Fong C, Mok VC, Leung K, Tong RK. Computerized Cognitive Screen (CoCoSc): A Self-Administered Computerized Test for Screening for Cognitive Impairment in Community Social Centers. JAD 2017 Aug 14;59(4):1299-1306. [CrossRef]
    23. Ismail Z, Elbayoumi H, Fischer CE, Hogan DB, Millikin CP, Schweizer T, et al. Prevalence of Depression in Patients With Mild Cognitive Impairment: A Systematic Review and Meta-analysis. JAMA Psychiatry 2017 Jan 01;74(1):58-67. [CrossRef] [Medline]
    24. Assal F, Cummings JL. Neuropsychiatric symptoms in the dementias. Curr Opin Neurol 2002 Aug;15(4):445-450. [CrossRef] [Medline]
    25. Chin J, Kim DE, Lee H, Yun J, Lee BH, Park J, et al. A Validation Study of the Inbrain CST: a Tablet Computer-based Cognitive Screening Test for Elderly People with Cognitive Impairment. J Korean Med Sci 2020 Aug 31;35(34):e292 [FREE Full text] [CrossRef] [Medline]
    26. Van Mierlo LD, Wouters H, Sikkes SAM, Van der Flier WM, Prins ND, Bremer JAE, et al. Screening for Mild Cognitive Impairment and Dementia with Automated, Anonymous Online and Telephone Cognitive Self-Tests. J Alzheimers Dis 2017;56(1):249-259 [FREE Full text] [CrossRef] [Medline]
    27. Charernboon T. Diagnostic Accuracy of the Overlapping Infinity Loops, Wire Cube, Clock Drawing Tests and their Combined Score for Cognitive Impairment in Mild Cognitive Impairment and Dementia. Eur. psychiatr 2020 Mar 23;41(S1):S647-S647. [CrossRef]
    28. Wei M, Shi J, Li T, Ni J, Zhang X, Li Y, et al. Diagnostic Accuracy of the Chinese Version of the Trail-Making Test for Screening Cognitive Impairment. J Am Geriatr Soc 2018 Jan 14;66(1):92-99. [CrossRef] [Medline]
    29. Müller S, Herde L, Preische O, Zeller A, Heymann P, Robens S, et al. Diagnostic value of digital clock drawing test in comparison with CERAD neuropsychological battery total score for discrimination of patients in the early course of Alzheimer's disease from healthy individuals. Sci Rep 2019 Mar 05;9(1):3543 [FREE Full text] [CrossRef] [Medline]
    30. Wallert J, Westman E, Ulinder J, Annerstedt M, Terzis B, Ekman U. Differentiating Patients at the Memory Clinic With Simple Reaction Time Variables: A Predictive Modeling Approach Using Support Vector Machines and Bayesian Optimization. Front Aging Neurosci 2018 May 22;10:144 [FREE Full text] [CrossRef] [Medline]
    31. Hemmy LS, Linskens EJ, Silverman PC, Miller MA, Talley KM, Taylor BC, et al. Brief Cognitive Tests for Distinguishing Clinical Alzheimer-Type Dementia From Mild Cognitive Impairment or Normal Cognition in Older Adults With Suspected Cognitive Impairment. Annals of Internal Medicine 2020 May 19;172(10):678-687. [CrossRef]
    32. SHEIKH JIYJ. Geriatric depression scale(GDS):recent evidence and development of a shorter version. Clin Gerontol 1986:5-165. [CrossRef]
    33. Zou Z, Chau BKH, Ting K, Chan CCH. Aging Effect on Audiovisual Integrative Processing in Spatial Discrimination Task. Front Aging Neurosci 2017 Nov 14;9:374 [FREE Full text] [CrossRef] [Medline]
    34. Lu J, Li D, Li F, Zhou A, Wang F, Zuo X, et al. Montreal cognitive assessment in detecting cognitive impairment in Chinese elderly individuals: a population-based study. J Geriatr Psychiatry Neurol 2011 Dec 05;24(4):184-190. [CrossRef] [Medline]
    35. Petersen RC. Mild cognitive impairment as a diagnostic entity. J Intern Med 2004 Oct;256(3):183-194 [FREE Full text] [CrossRef] [Medline]
    36. Petersen RC, Lopez O, Armstrong MJ, Getchius TS, Ganguli M, Gloss D, et al. Practice guideline update summary: Mild cognitive impairment. Neurology 2017 Dec 27;90(3):126-135. [CrossRef]
    37. Roalf DR, Moberg PJ, Xie SX, Wolk DA, Moelter ST, Arnold SE. Comparative accuracies of two common screening instruments for classification of Alzheimer's disease, mild cognitive impairment, and healthy aging. Alzheimers Dement 2013 Oct 26;9(5):529-537 [FREE Full text] [CrossRef] [Medline]
    38. Winblad B, Palmer K, Kivipelto M, Jelic V, Fratiglioni L, Wahlund L, et al. Mild cognitive impairment--beyond controversies, towards a consensus: report of the International Working Group on Mild Cognitive Impairment. J Intern Med 2004 Oct;256(3):240-246 [FREE Full text] [CrossRef] [Medline]
    39. McKhann GM, Knopman DS, Chertkow H, Hyman BT, Jack CR, Kawas CH, et al. The diagnosis of dementia due to Alzheimer's disease: recommendations from the National Institute on Aging-Alzheimer's Association workgroups on diagnostic guidelines for Alzheimer's disease. Alzheimers Dement 2011 May;7(3):263-269 [FREE Full text] [CrossRef] [Medline]
    40. Burke WJ, Roccaforte WH, Wengel SP. The short form of the Geriatric Depression Scale: a comparison with the 30-item form. J Geriatr Psychiatry Neurol 1991 Sep 02;4(3):173-178. [CrossRef] [Medline]
    41. Hu C, Zhao D, Gong F, Zhao Y, Li J, Sun Y. Risk factors for suicidal ideation among the older people living alone in rural region of China: A path analysis. Medicine (Baltimore) 2020 Jul 17;99(29):e21330 [FREE Full text] [CrossRef] [Medline]
    42. Zhang H, Peng Y, Li C, Lan H, Xing G, Chen Z, et al. Playing Mahjong for 12 Weeks Improved Executive Function in Elderly People With Mild Cognitive Impairment: A Study of Implications for TBI-Induced Cognitive Deficits. Front Neurol 2020 Mar 27;11:178 [FREE Full text] [CrossRef] [Medline]
    43. Fang Y, Tao Q, Zhou X, Chen S, Huang J, Jiang Y, et al. Patient and Family Member Factors Influencing Outcomes of Poststroke Inpatient Rehabilitation. Arch Phys Med Rehabil 2017 Mar;98(2):249-255.e2 [FREE Full text] [CrossRef] [Medline]
    44. Hendry K, Green C, McShane R, Noel-Storr A, Stott D, Anwer S, et al. AD-8 for detection of dementia across a variety of healthcare settings. Cochrane Database Syst Rev 2019 Mar 04;3(3):CD011121 [FREE Full text] [CrossRef] [Medline]
    45. Lawton MP, Brody EM. Assessment of Older People: Self-Maintaining and Instrumental Activities of Daily Living. The Gerontologist 1969 Sep 01;9(3 Part 1):179-186. [CrossRef]
    46. Boateng GO, Neilands TB, Frongillo EA, Melgar-Quiñonez HR, Young SL. Best Practices for Developing and Validating Scales for Health, Social, and Behavioral Research: A Primer. Front Public Health 2018;6:149 [FREE Full text] [CrossRef] [Medline]
    47. Hong W, Tao J, Wong AWK, Yang S, Leung M, Lee TMC, et al. Psychometric Properties of the Chinese (Putonghua) Version of the Oxford Cognitive Screen (OCS-P) in Subacute Poststroke Patients without Neglect. Biomed Res Int 2018;2018:6827854-6827812 [FREE Full text] [CrossRef] [Medline]
    48. Gaskin PS, Chami P, Nancoo T, Warner P, Barrett P, Mayers Y. Electronic based reported anthropometry-A useful tool for interim monitoring of obesity prevalence in developing states. PLoS One 2020 Dec 7;15(12):e0243202 [FREE Full text] [CrossRef] [Medline]
    49. Lin C, Luh W, Yang A, Su C, Wang J, Ma H. Psychometric properties and gender invariance of the Chinese version of the self-report Pediatric Quality of Life Inventory version 4.0: short form is acceptable. Qual Life Res 2012 Mar 15;21(1):177-182. [CrossRef] [Medline]
    50. Rakusa M, Jensterle J, Mlakar J. Clock Drawing Test: A Simple Scoring System for the Accurate Screening of Cognitive Impairment in Patients with Mild Cognitive Impairment and Dementia. Dement Geriatr Cogn Disord 2018 Jul 23;45(5-6):326-334. [CrossRef] [Medline]
    51. Chan JYC, Bat BKK, Wong A, Chan TK, Huo Z, Yip BHK, et al. Evaluation of Digital Drawing Tests and Paper-and-Pencil Drawing Tests for the Screening of Mild Cognitive Impairment and Dementia: A Systematic Review and Meta-analysis of Diagnostic Studies. Neuropsychol Rev 2022 Oct 16;32(3):566-576 [FREE Full text] [CrossRef] [Medline]
    52. Dion C, Arias F, Amini S, Davis R, Penney D, Libon DJ, et al. Cognitive Correlates of Digital Clock Drawing Metrics in Older Adults with and without Mild Cognitive Impairment. J Alzheimers Dis 2020;75(1):73-83 [FREE Full text] [CrossRef] [Medline]
    53. Leissing-Desprez C, Thomas E, Segaux L, Broussier A, Oubaya N, Marie-Nelly N, et al. Understated Cognitive Impairment Assessed with the Clock-Drawing Test in Community-Dwelling Individuals Aged ≥50 Years. J Am Med Dir Assoc 2020 Dec;21(11):1658-1664. [CrossRef] [Medline]
    54. Oonuma J, Kasai M, Meguro M, Akanuma K, Yamaguchi S, Meguro K. Necker cube copying may not be appropriate as an examination of dementia: reanalysis from the Tajiri Project. Psychogeriatrics 2016 Oct 12;16(5):298-304. [CrossRef] [Medline]
    55. Reitan RM. Validity of the Trail Making Test as an Indicator of Organic Brain Damage. Percept Mot Skills 2016 Aug 31;8(3):271-276. [CrossRef]
    56. Sánchez-Cubillo I, Periáñez J, Adrover-Roig D, Rodríguez-Sánchez J, Ríos-Lago M, Tirapu J, et al. Construct validity of the Trail Making Test: Role of task-switching, working memory, inhibition/interference control, and visuomotor abilities. J Int Neuropsychol Soc 2009 May 01;15(3):438-450. [CrossRef]
    57. Ilardi CR, Chieffi S, Iachini T, Iavarone A. Neuropsychology of posteromedial parietal cortex and conversion factors from Mild Cognitive Impairment to Alzheimer's disease: systematic search and state-of-the-art review. Aging Clin Exp Res 2022 Mar 07;34(2):289-307 [FREE Full text] [CrossRef] [Medline]
    58. Russo MJ, Cohen G, Campos J, Martin ME, Clarens MF, Sabe L, et al. Usefulness of Discriminability and Response Bias Indices for the Evaluation of Recognition Memory in Mild Cognitive Impairment and Alzheimer Disease. Dement Geriatr Cogn Disord 2017 Nov 26;43(1-2):1-14. [CrossRef] [Medline]
    59. Groppell S, Soto-Ruiz KM, Flores B, Dawkins W, Smith I, Eagleman DM, et al. A Rapid, Mobile Neurocognitive Screening Test to Aid in Identifying Cognitive Impairment and Dementia (BrainCheck): Cohort Study. JMIR Aging 2019 Mar 21;2(1):e12615 [FREE Full text] [CrossRef] [Medline]
    60. Lei LK, Lam BY, Lai DW, Bai X, Li J, Zou Z, et al. Stability of Montreal Cognitive Assessment in Individuals with Mild Cognitive Impairment: Potential Influence of Practice Effect. JAD 2022 May 31;87(3):1401-1412. [CrossRef]

    ‎
    Acc: accuracy
    AD8: Ascertain Dementia-8
    AUC: area under the curve
    CANS-MCI: Computer-Administered Neuropsychological Screen for Mild Cognitive Impairment
    CC: cube copying
    CD: clock drawing
    C-MoCA: Chinese version of the Montreal Cognitive Assessment
    CoCoSc: Computerized Cognitive Screen
    DR: delayed recall
    EOmciSS: Efficient Online MCI Screening System
    GDS-15: 15-item Geriatric Depression Scale
    IADL: Instrumental Activities of Daily Living
    MCI: mild cognitive impairment
    MoCA: Montreal Cognitive Assessment
    NCF: normal cognitive function
    ROC: receiver operating characteristic
    RS: reaction speed
    RT: reaction time
    TM: trail making

    Edited by G Eysenbach; submitted 08.07.22; peer-reviewed by A Andrade; comments to author 22.10.22; revised version received 16.12.22; accepted 19.12.22; published 30.01.23

    Copyright

    ©Jingsong Wu, Jingnan Tu, Zhizhen Liu, Lei Cao, Youze He, Jia Huang, Jing Tao, Mabel N K Wong, Lidian Chen, Tatia M C Lee, Chetwyn C H Chan. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 30.01.2023.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.