This paper is in the following e-collection/theme issue:

Electronic/Mobile Data Capture, Internet-based Survey & Research Methodology (997) Dementia and Cognitive Decline (696) Diagnostic Tools in Mental Health (326) Telepsychiatry (212) Longitudinal and Cohort Studies in Public Health (411) Cognitive and Neurorehabilitation (349) Public (e)Health, Digital Epidemiology and Public Health Informatics (1008)

Published on in Vol 27 (2025)

This is a member publication of

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/69689, first published .
Evaluating the Accuracy of Web-Based and In-Clinic Subjective Cognitive Decline Assessments in Detecting Cognitive Impairment: Multicohort Study

Evaluating the Accuracy of Web-Based and In-Clinic Subjective Cognitive Decline Assessments in Detecting Cognitive Impairment: Multicohort Study

Evaluating the Accuracy of Web-Based and In-Clinic Subjective Cognitive Decline Assessments in Detecting Cognitive Impairment: Multicohort Study

Authors of this article:

Jae Myeong Kang1, 2, 3 Author Orcid Image ;   Manchumad Manjavong1, 3, 4 Author Orcid Image ;   Adam Diaz1, 5, 6 Author Orcid Image ;   Miriam T Ashford1, 6 Author Orcid Image ;   Anna Aaronson1, 5 Author Orcid Image ;   Joseph Eichenbaum1, 5 Author Orcid Image ;   Scott Mackin1, 3 Author Orcid Image ;   Rachana Tank7 Author Orcid Image ;   Melanie J Miller1, 5, 6 Author Orcid Image ;   Bernard Landavazo1, 5 Author Orcid Image ;   Erika Cavallone1, 5 Author Orcid Image ;   Diana Truran1, 5, 6 Author Orcid Image ;   Monica R Camacho1, 5, 6 Author Orcid Image ;   Juliet Fockler1, 5 Author Orcid Image ;   Derek Flenniken1, 5, 6 Author Orcid Image ;   Sarah Tomaszewski Farias8 Author Orcid Image ;   Michael W Weiner1, 3, 5, 6, 9 Author Orcid Image ;   Rachel Nosheny1, 3, 5, 6 Author Orcid Image
Article Authors Cited by Tweetations Metrics

    1VA Advanced Imaging Research Center, San Francisco VA Medical Center, San Francisco, CA, United States

    2Department of Psychiatry, Gachon University College of Medicine, Gil Medical Center, Incheon, Republic of Korea

    3Department of Psychiatry and Behavioral Sciences, University of California San Francisco, 4150 Clement Street (114M), San Francisco, CA, United States

    4Division of Geriatric Medicine, Department of Internal Medicine, Faculty of Medicine, Khon Kaen University, Khon Kaen, Thailand

    5Department of Radiology and Biomedical Imaging, University of California San Francisco, San Francisco, CA, United States

    6Northern California Institute for Research and Education, 4150 Clement Street (151NC), San Francisco, CA, United States

    7Dementia Research Centre, UCL Institute of Neurology, University College London, London, United Kingdom

    8Department of Neurology, University of California Davis, Sacramento, CA, United States

    9Department of Medicine, University of California San Francisco, San Francisco, CA, United States

    Corresponding Author:

    Rachel Nosheny


    Background: Scalable tools to efficiently identify individuals likely to have cognitive impairment (CI) are critical in the Alzheimer disease and related dementias field. The Everyday Cognition scale (ECog) and its short form (ECog12) assess subjective cognitive and functional changes and are useful in predicting CI. Recent advances in online technology have enabled the use of web-based cognitive tests and questionnaires to identify CI with greater convenience and scalability. While the effectiveness of the ECog has been demonstrated in clinical settings, its potential to detect CI in remote, unsupervised formats remains underexplored.

    Objective: This study aimed to compare the ability of the web-based ECog and the in-clinic ECog in distinguishing between individuals with CI and those who are cognitively unimpaired (CU), and to evaluate the effectiveness of the ECog12—the short version of the ECog—compared to the full-length ECog in a web-based setting.

    Methods: Participants were recruited from the Brain Health Registry (BHR; web-based) and Alzheimer’s Disease Neuroimaging Initiative (ADNI; in-clinic) settings with available clinical diagnoses. The ability of the self-reported ECog (Self-ECog), study partner–reported ECog (SP-ECog), Self-ECog12, and SP-ECog12 to discriminate individuals with CI from CU was assessed using receiver operating characteristic (ROC) curves. Area under the ROC curves (AUCs) between BHR and ADNI were compared using the DeLong test, as were AUCs between ECog12 and ECog in BHR.

    Results: Web-based Self-ECog and SP-ECog scores effectively discriminated CI from CU with AUCs of 0.722 and 0.818, respectively. Similarly, the abbreviated web-based versions, Self-ECog12 and SP-ECog12, also demonstrated discriminative ability (AUC=0.709 and 0.777, respectively). When compared to in-clinic ECog scores, there were no significant differences in the ability to distinguish CI from CU between web-based and in-clinic versions (BHR Self-ECog AUC=0.722 vs ADNI Self-ECog AUC=0.769, DeLong P=.06; BHR SP-ECog AUC=0.818 vs ADNI SP-ECog AUC=0.840, DeLong P=.50). Additionally, the comparison between web-based ECog and ECog12 showed no significant difference in AUCs (BHR Self-ECog AUC=0.722 vs BHR Self-ECog12 AUC=0.709, DeLong P=.18).

    Conclusions: Web-based ECog scores, both the full-length and short-form, were as valid as in-clinic ECog scores for identifying clinically diagnosed CI. In addition, Self-ECog12 was as effective as full-length Self-ECog to identify CI in a web-based setting, offering a cost-effective and accessible screening tool for large-scale studies. These results highlight the value of the web-based ECog as a valid tool for identifying older adults with CI in a remote clinical study, facilitating early detection and referral for comprehensive evaluations for identifying potential candidates for disease-modifying therapy.

    J Med Internet Res 2025;27:e69689

    doi:10.2196/69689

    Keywords

    subjective cognitive decline; ECog-12; online; MCI; dementia; web-based; Everyday Cognition scale; mild cognitive impairment 


    Dementia is a syndrome characterized by cognitive and noncognitive symptoms, affecting activities of daily living. Alzheimer disease (AD) is the most common cause of dementia, marked by amyloid beta and tau neuropathology, which leads to memory decline and other cognitive symptoms that precede dementia [1]. The recent development of disease-modifying medications such as aducanumab, lecanemab, and donanemab has brought new therapeutic options into clinical practice [2]. Consequently, identifying individuals likely to have mild cognitive impairment (MCI)—potential candidates for disease-modifying therapy—has become increasingly important.

    Neuropsychological tests are widely used as diagnostic tools for cognitive impairment (CI), differentiating between individuals who are cognitively unimpaired (CU), those with MCI, and those with dementia [3]. However, these tests are time-consuming, require trained professionals, and may not effectively capture fluctuations or progressive changes in cognitive function [4]. The Everyday Cognition scale (ECog), developed by Farias et al [5], offers a practical, time-efficient alternative that can be self-administered, focusing on realistic assessments of daily functioning. A key feature of the ECog is its ability to assess subjective cognitive changes compared to 10 years ago, accounting for individual differences in baseline cognitive function. Additionally, it can address anosognosia—a common issue in individuals with MCI—through informant-administered ECog, which correlates more closely with clinical diagnoses than self-administered ECog [5-7]. Past studies have demonstrated the utility of ECog in identifying people with CI or AD pathology and predicting future conversion to MCI or dementia in healthy older adults [5-10].

    Despite its proven effectiveness in clinical settings, the potential of ECog in remote, unsupervised formats remains underexplored. The Alzheimer’s Disease Neuroimaging Initiative (ADNI) [11] recently began remote, unsupervised administration of the ECog12, a validated short form of the ECog [7,8], to participants and their study partners (SP) [12]. This shift to digital platforms marks a significant advancement in clinical research, allowing for broader reach and more continuous data collection [12]. Similarly, the Brain Health Registry (BHR), which has remotely administered ECog to over 64,680 participants since 2014 [13], exemplifies the potential of web-based platforms to facilitate large-scale cognitive assessments and long-term monitoring. These platforms [13-15] present significant advantages for identifying individuals at risk of cognitive decline, including the cost- and time-effectiveness of reducing in-person visits. Our study specifically investigates the validity and utility of the web-based ECog as compared to their traditional, in-clinic counterparts.

    The overall goal of this study was to determine the ability of the web-based ECog to distinguish individuals with CI from CU. We investigated three hypotheses: (1) the web-based ECog will be associated with clinically diagnosed CI, (2) the web-based ECog will show comparable association with CI as the in-clinic ECog, and (3) the web-based ECog12 will be as effective in identifying CI as the full-length online ECog. These hypotheses were tested by estimating associations between ECog and clinical diagnosis and comparing the accuracy of in-clinic versus web-based ECog and ECog12 versus the full ECog to distinguish diagnostic groups.


    Participants

    Participants in this study were recruited from two sources: (1) the BHR Electronic Validation of Online Methods to Predict and Monitor Cognitive Decline (eVAL) study [16,17] for the web-based component and (2) the ADNI study for the in-clinic component.

    BHR eVAL

    The BHR is a web-based neuroscience registry dedicated to the evaluation and monitoring of various cognitive disorders in a remote setting. Participants and their invited study partners visit the BHR website to complete remote cognitive assessments every 6 months [7]. The eVAL study is a multisite study that used the BHR platform to validate electronic versions of existing in-clinic cognitive monitoring instruments through the internet-based BHR platform. All eVAL participants were also evaluated in an in-clinic setting [18]. Participants were enrolled in eVAL from BHR at the University of California San Francisco and from the Alzheimer’s Disease Research Centers at the University of Alabama at Birmingham, Mayo Clinic in Rochester, MN, and Washington University [18].

    ADNI

    Data used in the preparation of this study were obtained from the ADNI database [19]. Launched in 2003 as a collaboration between public and private sectors under the leadership of the principal investigator (MWW), ADNI aims to determine if serial magnetic resonance imaging, positron emission tomography scans, and other biological markers, along with clinical and neuropsychological evaluations, can effectively track the progression of MCI and early AD. For the latest updates, visit the ADNI database website [19].

    Inclusion Criteria

    This study included participants from BHR eVAL and ADNI with the following criteria: (1) age between 55 and 90 years; (2) have completed the Self-ECog or SP-ECog questionnaire, or both; (3) have a clinical diagnosis of CU or MCI; and (4) underwent both ECog questionnaire and clinical diagnostic evaluation at the baseline evaluation period.

    Ethical Considerations

    This study was approved by the institutional review board of the University of California San Francisco. Participants provided web-based informed consent upon enrollment in the BHR and written informed consent at their first in-clinic visit for enrollment in the eVAL study. For the ADNI study, all study protocols were approved by the institutional review board of the respective participating study centers, and participants provided written informed consent. All data were anonymized. Compensation for participation varied depending on the level and duration of involvement, ranging from US $0 to US $3125 over a 5-year period [20].

    Measurement

    Everyday Cognition Scale

    This study used baseline ECog scores from all participants. ECog is a 39-item questionnaire that assesses subjective changes in cognition and instrumental activities of daily living compared to 10 years ago [5]. Each item is rated on a 1‐4 Likert scale (1, no change or better; 2, questionable or occasionally worse; 3, consistently a little worse; 4, consistently much worse), with the total score being the average of all responses and higher scores indicating greater decline. ECog ratings can be provided either by the individual themselves (self-reported ECog or Self-ECog) or by a study partner (SP-ECog). In this study, SP-ECog was used when available. ECog was assessed in a web-based unsupervised setting in the BHR eVAL study and in an in-clinic setting in the ADNI study.

    ECog12 is a short form of ECog that was developed using selected 12 items throughout all cognitive domains [8]. ECog12 has shown a reduced testing time with high consistency and clinical validity, and several previous studies have used ECog12 [6,8,21]. For this study, the full ECog was administered in both BHR and ADNI. We derived the ECog12 score from the full ECog by selecting the 12 items from the original, full-length instrument and calculating the average of the 12 items.

    Clinical Diagnosis

    All participants in this study underwent clinical diagnostic evaluations. In the BHR eVAL study, participants were clinically diagnosed as CU, having MCI, or very mild dementia according to the Uniform Data Set version 3. This standard was developed by the National Alzheimer’s Coordinating Center under the supervision of the National Institute on Aging and is implemented across all U.S. Alzheimer’s Disease Centers [22].

    In ADNI, participants were evaluated for an initial diagnosis using scores from the Clinical Dementia Rating (CDR) scale, the Mini Mental State Exam, as well as clinical judgment. Participants were categorized into baseline diagnostic categories as follows: cognitively normal across all ADNI phases (1/GO/2/3), subjective memory concern (SMC) in ADNI 2, early MCI and late MCI in ADNI GO/2, MCI in ADNI 1/3, and AD across all ADNI phases. For this study, cognitively normal, SMC, and MCI (early MCI, late MCI, and MCI) participants were included.

    Clinical diagnoses in this study were categorized into two groups: (1) CU, which includes CU from the BHR eVAL study and cognitively normal and SMC from ADNI; and (2) CI, which includes MCI and very mild dementia from the BHR eVAL study and early MCI, late MCI, and MCI from the ADNI study.

    Demographic and Clinical Information

    Participants’ demographic and clinical information were included in this study: age at baseline, gender (male and female), years of education, and race (African American, Asian, White, and other [American Indian/Alaskan Native, Hawaiian/Other Pacific Islander, more than one, and unknown/declined to state race]).

    Baseline CDR was obtained from both cohorts [23]. We used the CDR sum of boxes score, which is a reliable and valid diagnostic and dementia staging measure [24], with higher scores indicating a greater level of CI. The Geriatric Depression Scale (GDS) Short Form was also obtained for both studies [25]. One GDS item regarding memory symptoms (“Do you feel you have more problems with memory than most?”) was excluded from the GDS score, thus making the total score ranging from 0 to 14. A higher score indicates more depressive symptoms. Self-reported memory concern was also obtained from both cohorts and was a yes-or-no question: “Are you concerned that you have a memory problem?” in BHR eVAL and “Are you concerned that you have a memory or other thinking problem?” in ADNI.

    Statistical Analysis

    Descriptive Analysis

    We produced descriptive statistics according to the diagnostic groups (CU and CI) and web-based or in-clinic study setting (BHR and ADNI) and compared demographic and clinical information and ECog scores between groups using independent t test (2-tailed) and chi-square test.

    Ability of Web-Based ECog for Predicting CI

    To test the first hypothesis that the web-based ECog can distinguish diagnostic groups, we performed binomial logistic regression analyses with the diagnostic group (CU or CI) as the dependent variable. We then calculated the area under the curve (AUC) using the receiver operating characteristic curve. Subsequent metrics including optimal ECog cut point score, sensitivity, specificity, and Youden index were also calculated. Additionally, adjusted AUCs based on the predicted value of a logistic model with ECog score, age, gender, years of education, race, and GDS were calculated.

    Comparison Between Web-Based and In-Clinic ECog

    To compare the ability of web-based ECog for discriminating CI from CU to that of in-clinic ECog, we compared the AUC for discriminating CI from CU in BHR to the AUC in ADNI using the DeLong test.

    Comparison Between ECog12 and ECog in Web-Based Setting

    To compare the effectiveness of web-based ECog12 in distinguishing CI from CU to the full-length web-based ECog, we also compared the AUC of ECog12 and the AUC of full-length ECog in BHR. Statistical significance was set at P<.05, there was no correction for multiple comparisons, and R software (version 4.3.2; R Foundation for Statistical Computing) was used in all analyses.


    Participants

    Demographic and Clinical Information

    A total of 1801 participants were included in this study (n=428 in BHR; n=1373 in ADNI). Table 1 presents the demographic and clinical information of participants, including comparisons across diagnostic groups and between the BHR and ADNI cohorts. Differences between CU and CI were observed for gender (χ21=6, P=.01), CDR sum of boxes (t77.9=–7.37, P<.001), and self-reported memory concern (χ21=37.6, P<.001) in BHR. In ADNI, differences were found in all demographic and clinical variables except for age. BHR participants showed higher GDS scores in both CU (t394.5=14.51, P<.001) and CI (t70.0=5.80, P<.001) groups compared to ADNI.

    Table 1. Participant characteristics.
    BHRaADNIbBHR vs ADNI
    CUc (n=344)CId (n=84)t test (df)e or χ2 (df)fP valueCU (n=666)CI (n=707)t test (df) or χ2 (df)P valueCUCI
    t test (df) or χ2 (df)P valuet test (df) or χ2 (df)P value
    Age, mean (SD)71.47 (7.36)72.75 (7.20)–1.46 (128.7)e.1570.99 (6.70)71.65 (7.50)–1.72 (1362.7)e.091.00 (639.7)e.321.32 (105.7)e.19
    Gender (female), n (%)193 (56.1)34 (40.5)6.0 (1)f.01398 (59.8)317 (44.8)30.0 (1)f<.001–1.1 (1)f.29–0.4 (1)f.52
    Education years, mean (SD)16.94 (2.06)16.67 (2.47)0.91 (108.0)e.3616.68 (2.37)16.17 (2.58)3.80 (1369.8)e<.0011.78 (763.3)e.081.69 (101.0)e.09
    Race, n (%)18.4 (3)f.0730.1 (3)f<.001–7.97 (3)f.05–8.9 (3)f.03
    African American23 (6.7)3 (3.6)82 (12.3)42 (5.9)
    Asian15 (4.4)2 (2.4)26 (3.9)10 (1.4)
    White293 (85.2)71 (84.5)538 (80.8)632 (89.4)
    Others13 (3.7)8 (9.5)20 (3.0)23 (3.3)
    CDR-SOBg, mean (SD)0.07 (0.28)1.08 (1.24)–7.37 (77.9)e<.0010.04 (0.14)1.46 (0.95)–39.20 (739.8)e<.0011.76 (436.0)e.08–2.75 (96.1)e.007
    GDSh, mean (SD)2.33 (1.89)2.77 (2.11)–1.56 (88.8)e.120.06 (1.04)1.24 (1.34)–8.93 (1322.3)e<.00114.51 (394.5)e<.0015.80 (70.0)e<.001
    Self-reported memory concern, n (%)103 (29.9)51 (60.7)37.6 (1)f<.001269 (40.4)495 (70.0)293.6 (1)f<.001–3.3 (1)f.07–7.1 (1)f.008
    ECogi, mean (SD)
    Self-ECogj1.36 (0.39)1.67 (0.52)–4.56 (81.1)e<.0011.37 (0.33)1.82 (0.57)–18.13 (1150.3)e<.001–0.28 (518.7)e.78–2.21 (80.1)e.03
    SP-ECogk1.16 (0.25)1.72 (0.64)–7.12 (73.3)e<.0011.17 (0.25)1.74 (0.61)–22.69 (925.8)e<.001–0.83 (505.9)e.41–0.29 (80.7)e.77
    Self-ECog12l1.34 (0.39)1.64 (0.54)–4.27 (80.3)e<.0011.35 (0.34)1.77 (0.58)–16.52 (1150.9)e<.001–0.40 (517.7)e.69–1.80 (79.2)e.08
    SP-ECog121.17 (0.27)1.69 (0.66)–6.38 (73.9)e<.0011.18 (0.27)1.73 (0.62)–21.34 (957.9)e<.001–0.55 (507.2)e.58–0.47 (80.2)e.64

    aBHR: Brain Health Registry.

    bADNI: Alzheimer’s Disease Neuroimaging Initiative.

    cCU: cognitively unimpaired.

    dCI: cognitively impaired.

    et test (df)

    fχ2 (df)

    gCDR-SOB: Clinical Dementia Rating – Sum of Boxes.

    hGDS: Geriatric Depression Scale.

    iECog: Everyday Cognition scale.

    jSelf-ECog: self-reported ECog.

    kSP-ECog: study partner–reported ECog.

    lECog12: short version of ECog.

    ECog Scores in Both Groups

    Table 1 displays ECog scores. In all cases, Self-ECog scores tended to be higher than SP-ECog scores, although not statistically proven. The only significant difference between ADNI and BHR was observed in Self-ECog scores within the CI group (ADNI > BHR; t80.1=–2.21, P=.03).

    Association Between ECog Scores and CI

    Table 2 presents the AUC values for distinguishing CI from CU in BHR and ADNI. The AUCs were moderate in BHR (0.709‐0.818) and ADNI (0.747‐0.840). After adjusting for age, gender, years of education, race, and GDS, the AUCs remained fair to moderate in both BHR and ADNI (0.724‐0.851). Subsequent metrics including optimal cut point, sensitivity, specificity, and diagnostic accuracy are also presented in Table 2 for both studies. In adjusted AUC, a linear regression model was used with predictors including ECog score, age, gender, years of education, race, and GDS.

    Table 2. ECoga AUCsb for discriminating CIc from CUd.
    AUC (confidence interval)Adjusted AUCCut pointSensitivitySpecificityAccuracyYouden index
    In BHRe
     Self-ECogf0.722 (0.660‐0.783)0.736≥1.3610.7420.6630.6770.406
     SP-ECogg0.818 (0.759‐0.878)0.807≥1.2630.7250.7910.7770.515
     Self-ECog12h0.709 (0.646‐0.772)0.724≥1.3640.6520.6800.6750.331
     SP-ECog120.777 (0.759‐0.878)0.770≥1.2730.6520.7800.7540.432
    In ADNIi
     Self-ECog0.769 (0.743‐0.792)0.796≥1.4360.7310.6830.7080.414
     SP-ECog0.840 (0.819‐0.860)0.851≥1.2310.7920.7220.7580.514
     Self-ECog120.747 (0.720‐0.772)0.782≥1.3750.7240.6650.6960.389
     SP-ECog120.817 (0.794‐0.839)0.834≥1.2730.7320.7580.7440.489

    aECog: Everyday Cognition scale.

    bAUC: area under the curve.

    cCI: cognitively impaired.

    dCU: cognitively unimpaired.

    eBHR: Brain Health Registry.

    fSelf-ECog: self-reported ECog.

    gSP-ECog: study partner–reported ECog.

    hSelf-ECog12: short version of ECog.

    iADNI: Alzheimer’s Disease Neuroimaging Initiative.

    Comparing AUCs Between BHR and ADNI

    Figure 1 illustrates the AUCs of BHR and ADNI and their comparisons. Both Self-ECog (BHR AUC=0.722, confidence interval 0.660‐0.783; ADNI AUC=0.769, confidence interval 0.743‐0.792; DeLong P=.16) and SP-ECog (BHR AUC=0.818, confidence interval 0.759‐0.878; ADNI AUC=0.840, confidence interval 0.819‐0.860; DeLong P=.5) showed comparable AUC values between BHR and ADNI.

    ‎
    Figure 1. Receiver operating curves for predicting cognitive impairment. ADNI: Alzheimer’s Disease Neuroimaging Initiative; AUC: area under the curve; BHR: Brain Health Registry; ECog: Everyday Cognition scale; ECog12: short version of ECog; Self-ECog: self-reported ECog; SP-ECog: study partner–reported ECog.

    Comparing AUCs Between ECog and ECog12 in BHR

    Figure 1 depicts the comparison between ECog and ECog12 in BHR. Self-ECog and Self-ECog12 demonstrated comparable AUCs (Self-ECog AUC=0.722, Self-ECog12 AUC=0.709, DeLong P=.18), while SP-ECog exhibited a higher AUC than SP-ECog12 (SP-ECog AUC=0.818, SP-ECog12 AUC=0.777, DeLong P<.001).


    Principal Results

    This study investigated the effectiveness of a measure of subjective cognitive decline, ECog, to detect CI in two cohorts: web-based in BHR and in-clinic in ADNI. The first finding was that the ECog demonstrated moderate accuracy in distinguishing individuals with CI and CU in both BHR and ADNI. The second finding was that the web-based ECog in BHR showed a comparable ability to detect CI, similar to the in-clinic ECog scores in ADNI. The third finding was that the ECog12, the short form of the ECog, was as effective as the full-length ECog for detecting CI in the BHR. These results indicate that ECog scores, whether administered web-based or in-clinic, and in both short-form and full-length versions, are useful in identifying individuals clinically diagnosed with CI.

    The first finding of this study is that the web-based ECog demonstrated a moderate association with CI. Both online Self- and SP-ECog scores effectively distinguished individuals clinically diagnosed with MCI and CU. This result aligns with previous research that has shown a strong association between web-based SP-ECog and self-reported diagnoses of MCI or AD [7]. Unlike prior research that relied solely on either remotely collected data [7,26] or in-clinic data [6,8,9], our analysis included individuals with both web-based ECog data and clinical evaluations, allowing for a more rigorous assessment of the relationship between web-based subjective cognitive decline and objectively diagnosed CI. This finding highlights the potential clinical utility of online subjective cognitive decline measures, suggesting that they may reflect underlying cognitive impairment as determined by expert clinical evaluation. Previous in-clinic studies have reported that informant-reported ECog scores are better at predicting CI from CU than self-reported scores [6,8,9]. A prior study focusing on older adults with CI found that web-based SP-ECog could distinguish AD dementia from MCI, while Self-ECog could not [27]. The ability of both Self- and SP-ECog to identify CI in this study might be due to the fact that most participants in BHR had mild cognitive symptoms [28], which makes it less likely that they are unaware of their cognitive status [29]. It is also noteworthy that establishing web-based ECog cut point scores with good sensitivities, specificities, and accuracies is useful as a screening tool to detect CI in the older adult population. Our findings demonstrate the utility of web-based ECog for detecting the clinical diagnosis of CI, particularly in mildly impaired populations.

    The second finding is that web-based ECog and in-clinic ECog produced comparable results in discriminating CI from CU. Previous studies have found that online ECog scores closely correspond with in-clinic ECog scores [30]. Our study newly revealed that web-based ECog is as effective as in-clinic ECog for detecting CI from CU, as demonstrated by the AUC comparison, which showed no significant difference between the two measures. This finding suggests that remotely collected ECog can serve as a viable alternative to in-clinic ECog for cognitive screening, expanding accessibility to cognitive assessments beyond traditional clinical settings. The trend toward overall higher discriminative AUC values, although not significant, in ADNI compared to BHR might be attributed to the nature of web-based scales, including response quality, lower engagement due to convenience, and potential technical issues [31-33]. From a clinical perspective, however, the lack of statistical significance in the AUC comparison suggests that both web-based and in-clinic ECog can be considered equally effective for CI screening, with the web-based version offering the added benefits of convenience and accessibility, particularly in populations with limited access to in-person assessments.

    The third finding is that Self-ECog12 performed comparably to full-length Self-ECog in distinguishing CI from CU in a remote setting. ECog12, derived from the original 39-item ECog, maintains good psychometric properties for various cognitive domains, including memory, visuospatial function, language, and executive function [8]. Previous studies have shown that ECog12 is as effective as full-length ECog in identifying the risk of MCI [6,9]. The fact that web-based ECog12 shows a similar association with the diagnosis of CI versus CU as the 39-item ECog supports its use in web-based clinical studies like BHR. Recent large-scale neuroscience studies have started collecting online data from questionnaires about cognitive function, leveraging extensive and longitudinal collections [34-36]. ADNI 4, the current phase of ADNI, also recently implemented web-based ECog12 in a novel online screening approach to aid in selecting participants for plasma AD biomarker analysis and in-clinic evaluation [12,21]. However, SP-ECog12 showed a significantly lower probability of detecting CI from CU compared to SP-ECog in this study, although both SP-ECog and SP-ECog12 showed moderate AUCs. We believe this discrepancy may be due to the brevity and the reduced amount of information collected by study partners in the web-based ECog12. This might be further supported by the fact that study partners, who may not know the participants well or see them frequently, can still take part in the BHR study [7]. Nevertheless, the web-based ECog12 remains a useful and time-efficient tool for identifying individuals at risk of MCI in remote settings.

    Recently, advances in online technology have allowed the use of web-based cognitive tests to identify CI with convenience and scalability [37,38]. Self-administered web-based digital neuropsychological tests are an effective method for identifying CI, offering benefits such as accessibility, efficiency, and diagnostic accuracy for dementia [39]. Many well-known in-clinic tests, including the Montreal Cognitive Assessment, Cogstate Brief Battery, and Cambridge Neuropsychological Test Automated Battery (CANTAB, Cambridge Cognition Ltd), have been adapted for online use [40-43]. Although neuropsychological tests objectively measure cognitive abilities through rigorous standardized tasks, ECog, which is a brief measure of subjective cognitive decline, captures an individual’s or their informant’s perception of cognitive and functional changes over the past 10 years in real life. This can provide early indicators of cognitive decline that objective tests might miss [5,29]. As ECog and ECog12 can be easily administered in a remote setting with validity for identifying CI from CU, integrating both objective and subjective measures of cognitive decline in web-based studies could enhance early detection of MCI who are potential candidates for disease-modifying therapy and comprehensive understanding of cognitive health.

    Limitations

    This study has several limitations. First, the BHR eVAL study and ADNI consist predominantly of participants with over 16 years of education, 85% of whom are non-Latinx White. This lack of demographic diversity likely limits the generalizability of the findings and introduces bias related to differences in race, ethnicity, cognitive reserve, and socioeconomic status—factors that can influence the diagnosis, prognosis, and treatment response in dementia. African American and Latinx older adults are up to 1.5 to 2 times more likely to develop AD and AD-related dementias compared to White older adults [44]. These disparities may stem from differences in medical risk factors for AD (eg, depression, diabetes, obesity, and hypertension), biological factors (eg, genetics, epigenetics, or proteomic markers), factors contributing to cognitive reserve (eg, education and occupation), and nonbiological or social factors (eg, economic stability, safety, housing, and environment) [12]. Additionally, BHR has limitations, including selection bias, low data integrity, informant variability, and high dropout rates. Participants are more likely to have higher digital proficiency and increased awareness of cognitive health issues, further skewing the representativeness of the sample. On the other hand, web-based studies offer the advantage of scalability, enabling participation from individuals who might otherwise be unable to join due to geographic or time constraints. However, challenges such as data inaccuracy, missing information, and the inability to verify whether participants completed assessments independently or with assistance remain. Another important limitation is the difficulty in managing participant dropout over time. Prior studies have also shown that non-White or Latinx individuals, as well as those with lower educational attainment, are more likely to discontinue participation in follow-up assessments in web-based settings [45]. These patterns highlight the need for more inclusive recruitment strategies and efforts to retain diverse populations in future studies to ensure findings are more broadly applicable.

    The second limitation is that this study was not originally designed to compare web-based and in-clinic populations. Instead, it compared these populations retrospectively, meaning that the recruitment process was not perfectly controlled from the outset, and there may be participation self-selection biases. Third, the cross-sectional design of this study limits the interpretation of the results due to the lack of longitudinal cause-and-effect relationships between ECog scores and clinical diagnoses. Fourth, the lack of correction for multiple comparisons is an additional limitation of the study.

    Conclusions

    This study highlights the value of web-based ECog as an effective and scalable tool for identifying older adults at risk of MCI, particularly in web-based settings. Subjective cognitive changes measured in a remote, unsupervised, online setting distinguished CI from CU diagnostic groups with moderate accuracy. Compared to in-clinic scales, the web-based ECog showed a comparable ability to discriminate CI from CU. ECog12, a shorter form developed to increase brevity, also demonstrated a moderate association with clinical diagnosis in a remote setting. These results highlight the value of the web-based ECog as a valid tool for identifying older adults with MCI in an online clinical study, facilitating early detection and referral for more comprehensive evaluations in both research and clinical settings.

    Acknowledgments

    Data collection and sharing for this project were funded by the Alzheimer’s Disease Neuroimaging Initiative (ADNI) (National Institutes of Health grant U01 AG024904) and Department of Defense ADNI (award number W81XWH-12-2-0012). ADNI is funded by the National Institute on Aging, the National Institute of Biomedical Imaging and Bioengineering, and generous contributions from the following: AbbVie; Alzheimer’s Association; Alzheimer’s Drug Discovery Foundation; Araclon Biotech; BioClinica, Inc; Biogen; Bristol-Myers Squibb Company; CereSpir, Inc; Cogstate; Eisai Inc; Elan Pharmaceuticals, Inc; Eli Lilly and Company; EuroImmun; F. Hoffmann-La Roche Ltd and its affiliated company Genentech, Inc; Fujirebio; GE Healthcare; IXICO Ltd; Janssen Alzheimer Immunotherapy Research & Development, LLC; Johnson & Johnson Pharmaceutical Research & Development LLC; Lumosity; Lundbeck; Merck & Co, Inc; Meso Scale Diagnostics, LLC; NeuroRx Research; Neurotrack Technologies; Novartis Pharmaceuticals Corporation; Pfizer Inc; Piramal Imaging; Servier; Takeda Pharmaceutical Company; and Transition Therapeutics. The Canadian Institutes of Health Research is providing funds to support ADNI clinical sites in Canada. Private sector contributions are facilitated by the Foundation for the National Institutes of Health [46]. The grantee organization is the Northern California Institute for Research and Education, and the study is coordinated by the Alzheimer’s Therapeutic Research Institute at the University of Southern California. ADNI data are disseminated by the Laboratory for Neuro Imaging at the University of Southern California.

    Additional support was provided by grants from Gachon University (GCU-202309190001), the National Institute on Aging (R33AG062867), and the National Institutes of Health/National Institute of Neurological Disorders and Stroke/National Institute on Aging (U19AG024904). The sponsors had no role in the design and conduct of the study; in the collection, analysis, and interpretation of data; in the preparation of the manuscript; or in the review or approval of the manuscript.

    Data Availability

    The datasets used and analyzed during the study are available from the corresponding author on reasonable request.

    Conflicts of Interest

    MM, AD, MTA, AA, JE, SM, RT, MJM, BL, EC, DT, MRC, JF, DF, and STF report no conflicts of interest. JMK reports grant support from Gachon University during the conduct of the study. RN reports grant support from the National Institute on Aging during the conduct of the study. MWW reports grants from the National Institutes of Health/National Institute of Neurological Disorders and Stroke/National Institute on Aging, the Department of Defense, the California Department of Public Health, the University of Michigan, Siemens, Biogen, the Hillblom Foundation, the Alzheimer’s Association, Johnson & Johnson, Kevin and Connie Shanahan, GE, VUmc, the Australian Catholic University, The Stroke Foundation, and the Veterans Administration.

    Additionally, MWW reports personal fees from Boxer Capital, Cerecin, Clario, the Dementia Society of Japan, Eisai, Guidepoint, Health and Wellness Partners, Indiana University, LCN Consulting, Merck Sharp & Dohme Corp, the NC Registry for Brain Health, Prova Education, T3D Therapeutics, the University of Southern California, WebMD, the China Association for Alzheimer’s Disease, Taipei Medical University, AD/PD Congress, Cleveland Clinic, CTAD Congress, the Foundation of Learning, the Health Society (Japan), INSPIRE Project, the University of Toulouse, the Japan Society for Dementia Research, the Korean Dementia Society, and the National Center for Geriatrics and Gerontology (Japan). He also reports other support from Alzeca, Alzheon, ALZ Path, and Anven, outside the submitted work.

    1. Jack CR Jr, Bennett DA, Blennow K, et al. NIA-AA research framework: toward a biological definition of Alzheimer’s disease. Alzheimers Dement. Apr 2018;14(4):535-562. [CrossRef] [Medline]
    2. Cummings J, Zhou Y, Lee G, Zhong K, Fonseca J, Cheng F. Alzheimer’s disease drug development pipeline: 2024. Alzheimers Dement (N Y). Apr 2024;10(2):e12465. [CrossRef]
    3. Lezak MD. Neuropsychological Assessment. Oxford University Press; 2004. ISBN: 0195111214
    4. Harvey PD. Clinical applications of neuropsychological assessment. Dialogues Clin Neurosci. Mar 2012;14(1):91-99. [CrossRef] [Medline]
    5. Farias ST, Mungas D, Reed BR, et al. The measurement of everyday cognition (ECog): scale development and psychometric properties. Neuropsychology. Jul 2008;22(4):531-544. [CrossRef] [Medline]
    6. van Harten AC, Mielke MM, Swenson-Dravis DM, et al. Subjective cognitive decline and risk of MCI: the Mayo Clinic study of aging. Neurology (ECronicon). Jul 24, 2018;91(4):e300-e312. [CrossRef] [Medline]
    7. Nosheny RL, Camacho MR, Insel PS, et al. Online study partner-reported cognitive decline in the Brain Health Registry. Alzheimers Dement (N Y). 2018;4(1):565-574. [CrossRef] [Medline]
    8. Tomaszewski Farias S, Mungas D, Harvey DJ, Simmons A, Reed BR, Decarli C. The measurement of everyday cognition: development and validation of a short form of the Everyday Cognition scales. Alzheimers Dement. Nov 2011;7(6):593-601. [CrossRef] [Medline]
    9. Rueda AD, Lau KM, Saito N, et al. Self-rated and informant-rated everyday function in comparison to objective markers of Alzheimer’s disease. Alzheimers Dement. Sep 2015;11(9):1080-1089. [CrossRef] [Medline]
    10. Manjavong M, Diaz A, Ashford MT, et al. Performance of a short version of the everyday cognition scale (ECog-12) to detect cognitive impairment. J Prev Alzheimers Dis. 2024;11(6):1741-1750. [CrossRef] [Medline]
    11. Petersen RC, Aisen PS, Beckett LA, et al. Alzheimer’s Disease Neuroimaging Initiative (ADNI): clinical characterization. Neurology (ECronicon). Jan 19, 2010;74(3):201-209. [CrossRef] [Medline]
    12. Weiner MW, Veitch DP, Miller MJ, et al. Increasing participant diversity in AD research: plans for digital screening, blood testing, and a community-engaged approach in the Alzheimer’s Disease Neuroimaging Initiative 4. Alzheimers Dement. Jan 2023;19(1):307-317. [CrossRef] [Medline]
    13. Weiner MW, Nosheny R, Camacho M, et al. The Brain Health Registry: an internet-based platform for recruitment, assessment, and longitudinal monitoring of participants for neuroscience studies. Alzheimers Dement. Aug 2018;14(8):1063-1076. [CrossRef] [Medline]
    14. Reiman EM, Langbaum JBS, Fleisher AS, et al. Alzheimer’s Prevention Initiative: a plan to accelerate the evaluation of presymptomatic treatments. J Alzheimers Dis. 2011;26(Suppl 3):321-329. [CrossRef] [Medline]
    15. Walter S, Clanton TB, Langford OG, et al. Recruitment into the Alzheimer Prevention Trials (APT) Webstudy for a Trial-Ready Cohort for Preclinical and Prodromal Alzheimer’s Disease (TRC-PAD). J Prev Alzheimers Dis. 2020;7(4):219-225. [CrossRef] [Medline]
    16. Nosheny RL, Yen D, Howell T, et al. Evaluation of the electronic Clinical Dementia Rating for dementia screening. JAMA Netw Open. Sep 5, 2023;6(9):e2333786. [CrossRef] [Medline]
    17. Weiner MW, Aaronson A, Eichenbaum J, et al. Brain Health Registry updates: an online longitudinal neuroscience platform. Alzheimers Dement. Nov 2023;19(11):4935-4951. [CrossRef] [Medline]
    18. Howell T, Gummadi S, Bui C, et al. Development and implementation of an electronic Clinical Dementia Rating and Financial Capacity Instrument-Short Form. Alzheimers Dement (Amst). 2022;14(1):e12331. [CrossRef] [Medline]
    19. Sharing Alzheimer’s research data with the world. Alzheimer’s Disease Neuroimaging Initiative. URL: https://adni.loni.usc.edu/ [Accessed 2025-06-17]
    20. ADNI4. Alzheimer’s Disease Neuroimaging Initiative. URL: https://support.adni4.org/ [Accessed 2025-06-17]
    21. Miller MJ, Weiner MSW, Nosheny RL, et al. ADNI4’s novel remote screening approach – weighing priorities in participant selection. Alzheimers Dement. Dec 2023;19(S18):e075152. [CrossRef]
    22. Besser L, Kukull W, Knopman DS, et al. Version 3 of the National Alzheimer’s Coordinating Center’s uniform data set. Alzheimer Dis Assoc Disord. 2018;32(4):351-358. [CrossRef]
    23. Morris JC. Clinical Dementia Rating: a reliable and valid diagnostic and staging measure for dementia of the Alzheimer type. Int Psychogeriatr. 1997;9(Suppl 1):173-176. [CrossRef] [Medline]
    24. Williams MM, Storandt M, Roe CM, Morris JC. Progression of Alzheimer’s disease as measured by Clinical Dementia Rating Sum of Boxes scores. Alzheimers Dement. Feb 2013;9(1S):S39-S44. [CrossRef] [Medline]
    25. Lesher EL, Berryhill JS. Validation of the Geriatric Depression Scale–Short Form among inpatients. J Clin Psychol. Mar 1994;50(2):256-260. [CrossRef] [Medline]
    26. Kang JM, Manjavong M, Jin C, et al. Subjective cognitive decline predicts longitudinal neuropsychological test performance in an unsupervised online setting in the Brain Health Registry. Alzheimers Res Ther. Jan 7, 2025;17(1):10. [CrossRef] [Medline]
    27. Nosheny RL, Camacho MR, Jin C, et al. Validation of online functional measures in cognitively impaired older adults. Alzheimers Dement. Oct 2020;16(10):1426-1437. [CrossRef] [Medline]
    28. Ashford MT, Aaronson A, Kwang W, et al. Unsupervised online paired associates learning task from the Cambridge Neuropsychological Test Automated Battery (CANTAB®) in the Brain Health Registry. J Prev Alzheimers Dis. 2024;11(2):514-524. [CrossRef] [Medline]
    29. Orfei MD, Varsi AE, Blundo C, et al. Anosognosia in mild cognitive impairment and mild Alzheimer’s disease: frequency and neuropsychological correlates. Am J Geriatr Psychiatry. Dec 2010;18(12):1133-1140. [CrossRef] [Medline]
    30. Howell T, Neuhaus J, Glymour MM, Weiner MW, Nosheny RL. Validity of online versus in-clinic self-reported everyday cognition scale. J Prev Alzheimers Dis. 2022;9(2):269-276. [CrossRef] [Medline]
    31. McMaster HS, LeardMann CA, Speigle S, Dillman DA, Stander V, Pflieger J. An experimental comparison of web-push vs. paper-only survey procedures for conducting an in-depth health survey of military spouses. BMC Med Res Methodol. Apr 26, 2017;17(1):73. [CrossRef] [Medline]
    32. Couper MP. Web surveys: a review of issues and approaches. Public Opin Q. 2000;64(4):464-494. [CrossRef] [Medline]
    33. Nulty DD. The adequacy of response rates to online and paper surveys: what can be done? Assess Eval High Educ. Jun 2008;33(3):301-314. [CrossRef]
    34. Davis KAS, Coleman JRI, Adams M, et al. Mental health in UK Biobank – development, implementation and results from an online questionnaire completed by 157 366 participants: a reanalysis. BJPsych Open. Feb 6, 2020;6(2):e18. [CrossRef] [Medline]
    35. Ritchie CW, Molinuevo JL, Truyen L, et al. Development of interventions for the secondary prevention of Alzheimer’s dementia: the European Prevention of Alzheimer’s Dementia (EPAD) project. Lancet Psychiatry. Feb 2016;3(2):179-186. [CrossRef] [Medline]
    36. Ikram MA, Brusselle GGO, Murad SD, et al. The Rotterdam Study: 2018 update on objectives, design and main results. Eur J Epidemiol. Sep 2017;32(9):807-850. [CrossRef] [Medline]
    37. Chen L, Zhen W, Peng D. Research on digital tool in cognitive assessment: a bibliometric analysis. Front Psychiatry. 2023;14:1227261. [CrossRef]
    38. Watt JA, Lane NE, Veroniki AA, et al. Diagnostic accuracy of virtual cognitive assessment and testing: systematic review and meta-analysis. J Am Geriatr Soc. Jun 2021;69(6):1429-1440. [CrossRef] [Medline]
    39. Öhman F, Hassenstab J, Berron D, Schöll M, Papp KV. Current advances in digital cognitive assessment for preclinical Alzheimer’s disease. Alzheimers Dement (Amst). 2021;13(1):e12217. [CrossRef] [Medline]
    40. Cormack FK, Morris H, Barnett JH, Housden C. P1‐504: CANTAB Mobile: experience of use of a digital memory assessment tool in UK primary care. Alzheimers Dement. Jul 2018;14(7S_Part_9):P522-P523. [CrossRef]
    41. Backx R, Skirrow C, Dente P, Barnett JH, Cormack FK. Comparing web-based and lab-based cognitive assessment using the Cambridge Neuropsychological Test Automated Battery: a within-subjects counterbalanced study. J Med Internet Res. Aug 4, 2020;22(8):e16792. [CrossRef] [Medline]
    42. Cromer JA, Harel BT, Yu K, et al. Comparison of cognitive performance on the Cogstate Brief Battery when taken in-clinic, in-group, and unsupervised. Clin Neuropsychol. 2015;29(4):542-558. [CrossRef] [Medline]
    43. Snowdon A, Hussein A, Kent R, Pino L, Hachinski V. Comparison of an electronic and paper-based Montreal cognitive assessment tool. Alzheimer Dis Assoc Disord. 2015;29(4):325-329. [CrossRef]
    44. Chen C, Zissimopoulos JM. Racial and ethnic differences in trends in dementia prevalence and risk factors in the United States. Alzheimers Dement (N Y). 2018;4:510-520. [CrossRef] [Medline]
    45. Ashford MT, Eichenbaum J, Williams T, et al. Effects of sex, race, ethnicity, and education on online aging research participation. Alzheimers Dement (N Y). 2020;6(1):e12028. [CrossRef] [Medline]
    46. Our programs. Foundation for the National Institutes of Health. URL: www.fnih.org [Accessed 2025-06-17]

    ‎
    AD: Alzheimer disease
    ADNI: Alzheimer’s Disease Neuroimaging Initiative
    AUC: area under the curve
    BHR: Brain Health Registry
    CDR: Clinical Dementia Rating
    CI: cognitively impaired
    CU: cognitively unimpaired
    ECog: Everyday Cognition Scale
    ECog12: short version of ECog
    eVAL: Electronic Validation of Online Methods to Predict and Monitor Cognitive Decline
    GDS: Geriatric Depression Scale
    MCI: mild cognitive impairment
    ROC: receiver operating characteristic
    Self-ECog: self-reported ECog
    SMC: subjective memory concern
    SP-ECog: study partner–reported ECog

    Edited by Javad Sarvestan; submitted 07.12.24; peer-reviewed by Elena Popa, Yunyuan Gao; final revised version received 17.04.25; accepted 17.04.25; published 11.08.25.

    Copyright

    © Jae Myeong Kang, Manchumad Manjavong, Adam Diaz, Miriam T Ashford, Anna Aaronson, Joseph Eichenbaum, Scott Mackin, Rachana Tank, Melanie J. Miller, Bernard Landavazo, Erika Cavallone, Diana Truran, Monica R. Camacho, Juliet Fockler, Derek Flenniken, Sarah Tomaszewski Farias, Michael W. Weiner, Rachel Nosheny. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 11.8.2025.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research (ISSN 1438-8871), is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.