Large language model (LLM)-powered tools capable of generating and executing statistical code have begun to enter clinical research workflows [1-4]. Clinical data analysis requires substantial statistical and epidemiological expertise that may not be uniformly available to clinician-researchers [5-7]. LLM-powered agents offer a compelling model: a researcher states a research question, and the agent drafts a statistical plan and runs the analysis [8,9].

Despite this promise, a systematic evaluation of where LLM agents succeed and where they fail in clinical research workflows is lacking. In the field of clinical data analysis, which requires expertise to enter, it is possible for an agent to produce plausible-looking outputs while harboring methodological errors that are invisible to a nonexpert reviewer [10]. Existing benchmarks focus on code correctness for isolated tasks or on multiple-choice medical knowledge [1,11], without examining the full chain from question formulation through protocol design to statistical execution on real clinical data.

This pilot study evaluated an LLM agent across 5 stages of clinical data analysis, using a publicly released dataset and validated R (R Foundation for Statistical Computing) script from a previously published JAMA (Journal of the American Medical Association) Ophthalmology study [12] as a controlled benchmark. Three interaction modes spanning a spectrum from conversational use (Chat) to fully autonomous local execution (Code and Cowork) were compared across 3 prompt levels to identify systematic error patterns and to inform practical deployment guidance. Although the benchmark is grounded in ophthalmology, the data analysis workflow evaluated in this study (clinical question generation, statistical analysis plan [SAP] drafting, analysis execution, and result summarization and manuscript writing) is common to observational research across medical specialties. Accordingly, we identify and summarize generalizable error patterns and an evaluation framework for assessing future LLM agent releases in clinical data analysis more broadly.

The primary objective of this study was to evaluate the reliability, failure modes, and required human oversight of LLM agents across 5 stages of the clinical data analysis pipeline: research question generation, SAP formulation, data preprocessing and cohort boundary logic, statistical execution, and narrative clinical reporting. We hypothesized that while LLM agents could generate clinically relevant research questions and technically compliant statistical plans, their autonomous code execution would introduce silent, plausible-looking logical errors (such as cohort boundary mismatches, mathematical tie violations, or statistical assumption check omissions) that require mandatory expert biostatistical review and cannot be detected from runtime success or final numerical summaries alone.

This study evaluated an LLM agent across 3 clinical data analysis stages using a validated ophthalmology dataset and a fully specified reference analysis, in alignment with our primary objective of characterizing agent reliability, systematic failure modes, and required human oversight. The LLM agent was validated across different clinical data analysis stages, moving from a clinical question to an SAP, then to time-to-event code and narrative interpretation, which is common across observational research.

Relative to our stated aims, our main findings are as follows: for research question formulation, the agent successfully generated clinically relevant and important research questions, but their novelty was moderate, and 3 questions had low analytical feasibility due to data availability constraints (required clinical variables were absent from the raw CSV file).

The agent drafted complete, technically compliant SAPs, but we observed a critical dissociation where high-quality SAPs did not predict code correctness, and the agent introduced silent, plausible-looking execution errors (such as cohort boundary mismatches, time-zero tied survival time omissions, and incorrect censoring definitions) that produced no runtime error signals. Guided runs with detailed SAPs resolved cohort boundaries but introduced other repetitive error types, where the agent reproduced data structures from the SAP while ignoring explicit covariate exclusion directives. For transcription of numerical results from code logs to narrative summaries, the agent’s performance was highly accurate, but we identified in one case that the agent propagated an inverted HR into a confident but clinically incorrect narrative conclusion.

These findings suggest that while LLM agents can accelerate the initial generation of research questions, study protocols, and analysis code, they are currently better suited to supporting analytical work under strict expert supervision than to autonomously conducting it. Indeed, the substantial “verification time-tax” required to audit the code line-by-line, coupled with the absence of a pre-established ground truth in real-world practice, means that autonomous deployment carries significant risks of introducing silent, plausible-looking statistical errors that a researcher cannot easily identify without prior expertise.

For research question formulation, while all modes generated clinically grounded questions, mode selection shaped the thematic range of the output but did not map cleanly onto clinical value. Chat generated one treatment-burden question; Code generated trajectory phenotyping and withheld-injection clustering questions; and Cowork was the only mode to generate questions on ethnic disparity mediation. This pattern is consistent with lower-autonomy modes defaulting toward the most structurally prominent variables, though causal attribution to mode alone cannot be established from a nonrandomized design in which modes also differ in tool access, interface, and computational environment [16,17]. Despite this, the most divergent questions generated by higher-autonomy modes (such as ethnic-disparity mediation and withheld-injection clustering) had low feasibility because they required access to variables weakly represented in the released CSV (eg, socioeconomic status, comorbidity measures, and explicit noninjection visit coding). In addition, while the generated questions were highly important (mean score 4.2/5, SD 0.43) and reasonably feasible (mean score 3.5/5, SD 0.92), their novelty was moderate (mean score 3.0/5, SD 0.92) as many repeated familiar themes from the published literature. Consequently, clinician-researchers using the agent for question generation should review novelty, clinical priority, and dataset support separately rather than treating a well-formed question as study-ready.

The dissociation between SAP quality and code correctness is consistent with the current LLM architecture, in which language generation and code generation are functionally distinct processes [18,19], yet other explanations are also possible, such as context window limitations. Providing a complete, rationale-annotated SAP at prompt C resolved cohort boundary errors, as the agent operationalized explicit eligibility criteria correctly. However, this is still not a solution for all errors. Although every run reproduced the counting-process data structure described in the SAP, they included the injection count variable that the SAP explicitly excluded from the model formula. The SAP stated the exclusion and explained why, while the agent reproduced the observable structural pattern without operationalizing the stated omission. Thus, the practical lesson is that the verification should focus on the model formula itself instead of the stated SAP.

Beyond structural coding mistakes, several failure modes highlighted a lack of robust commonsense medical reasoning and metacognition. For example, B_Code iterations modeled VA down to the single ETDRS letter rather than defaulting to the standard 5-letter. Additionally, agents iteratively struggled to handle injection counts as a time-varying covariate, reproducing counting-process frameworks without conceptually integrating why treatment frequency necessitates specific statistical handling. Agents also narrated directionally inverted HR for VA endpoints, constructing plausible-sounding clinical summaries that completely failed to recognize the inherent biological paradox of their own conclusions. Furthermore, the agents failed to demonstrate basic statistical diagnostics spontaneously. In the original 27 runs, only one (C_Code_3) noted the PHs assumption, and none evaluated it or checked for multicollinearity. The agents executed the prespecified Cox models and KM curves blindly, ignoring significant violations of model assumptions (such as the PHs violation of baseline VA and the moderate multicollinearity between drug regimen and treatment era). However, in our post hoc update phase, when explicitly prompted to execute these diagnostics, the agent successfully wrote and executed PHs testing in 17 of the 18 runs. This indicates that while agents lack spontaneous diagnostic metacognition, they can only implement standard statistical verification checks reliably when guided by a prescriptive protocol under human oversight. Together, these errors underscore that while an LLM can mimic the syntax of clinical data analysis, it lacks the critical metacognition required to independently sanity-check its inputs, model design, and interpretations against real-world clinical logic, which is echoed by other studies [20,21].

A methodological concern when benchmarking LLM agents on retrospective data is data contamination. To assess the contamination status of our benchmark, we conducted an empirical zero-shot probing study by querying the LLM in independent sessions without any attached files [22]. These probes evaluated the model’s direct recall of the raw dataset schema, the original R analysis script, and the published results. The results revealed that while the model could retrieve the paper’s final text-based results (such as cohort sizes and specific HRs), it had no memory of the raw dataset schema (recalling incorrect variable names from a third-party R package) or the R script’s code syntax (such as the specific tmerge or counting-process implementation). One of the key findings is that we observed direct conflicts between the model’s zero-shot memory and its actual execution outputs: the model recalled the published cohort size of 5978 but dynamically output 5794 eyes due to strict-inequality code logic, and recalled the baseline VA HR of 1.43 but output a biased 1.329 HR due to formula composition errors. The agent’s language summaries anchored exclusively on these local execution logs rather than correcting them using pretraining memory, confirming that the agent was dynamically generating and executing code. Furthermore, to verify file-level noncontamination, we probed the model for unpublished dataset statistics (the mean and missing count of the mean_inj_interval column), which the model was unable to retrieve. Together, these findings indicate that the benchmark effectively measures the agent’s task execution capacity, even when the underlying study’s high-level findings are present in the model’s training data. We acknowledge that other contamination detection methods are available [23,24]; however, dense clinical and biostatistical text inherently resists word-level synonym swapping, as key clinical terms (drug names, ophthalmic measurement scales, defined statistical model names, and numerical outcome parameters) cannot be substituted without altering clinical meaning or creating nonsensical prose. More fundamentally, both methods measure the degree of text memorization, whereas the central validity question for our study is the knowledge-to-execution transfer: whether text-level recall translates into memorized code execution. Our memory-to-execution dissociation provides direct evidence that code execution was dynamic and situated rather than memorized. Even conservatively assuming that partial text-level contamination influenced the agent’s SAP method selection (eg, choosing Cox regression and survival analysis), the model still produced systematic coding errors across all 18 execution runs, which supports our implementation-level error findings.

Recent literature has increasingly explored the utility of LLMs in statistical analysis, reviewing their general capabilities in exploratory data analysis and code generation [25,26], or demonstrating proof-of-concept autonomous research pipelines [27,28]. While these studies acknowledge that LLMs exhibit “nonstandard errors” arising from variable analytical choices and emphasize the necessity of human copiloting for complex tasks [29], they frequently evaluate performance on isolated coding challenges [26] or general nonclinical domains [28,30]. Our study is distinguished by evaluating an LLM agent across the complete, end-to-end clinical data analysis workflow using a validated, real-world clinical dataset. Consistent with previous reports, we found that while LLMs efficiently generate plausible code and text, they lack robust statistical domain-aware error-checking, require human verification [29,31], and can produce silent execution errors [26]. However, our findings diverge by characterizing silent failure modes and a critical dissociation in which high-quality statistical analysis plans do not guarantee correct code implementation [26]. Whereas prior work often provides generic warnings regarding LLM hallucinations [27,32], our study provides actionable, targeted guidance for clinical researchers. Our findings indicate that analysis plans or numerical outputs alone cannot be assumed to confirm analytical correctness [26-28] and propose a concrete oversight framework that mandates expert verification of specific components of the code before AI-assisted research can be safely interpreted for clinical practice [29].

A further contribution of this study is methodological. Beyond evaluating one model on one dataset, we propose a reusable multistage evaluation framework for prospectively assessing future model releases. Rather than proposing our specific dataset-script replication task as a static benchmark, which would be highly vulnerable to benchmark leakage and memorization once published, this framework serves as a methodological blueprint. It translates our findings into a reusable infrastructure (the error taxonomy in Table 3 and the minimum evaluation framework in Table 4) that researchers can apply to their own local, unpublished clinical datasets. With specialty-appropriate datasets and reference analyses, the same framework could be applied to other clinical domains to support local validation before deployment. In real-world analytic tasks where no published reference analysis is available, the error taxonomy and evaluation framework may also help structure oversight by identifying high-risk stages of the workflow, prioritizing targeted verification checks, and documenting where errors arise, although they do not replace expert validation.

Our study has limitations. First, this is a single-dataset, single-agent pilot study. Second, the dataset used in this study has been manually cleaned so the findings may only be representative of performance on precleaned datasets, with qualitative evaluation anchored by comparison with a validated reference. Further multiagent, multidataset benchmarking is required to generalize across platforms or analytical domains. Moreover, our zero-shot probing showed the model could retrieve the published text-level results of the reference study, which may be aided by its use of live web-search tools. While we cannot definitively establish whether the original paper itself was present in the pretraining corpus, we acknowledge that this may have influenced its selection of survival analysis methods and covariates during SAP generation. Although our memory-to-execution dissociation analysis provides direct evidence that code execution was dynamic rather than memorized, we cannot fully exclude the possibility that the agent’s high-level analytical design was shaped by pretraining knowledge of the published methodology. Future studies should evaluate LLM agents on unpublished clinical datasets using our proposed evaluation framework (Table 4) and error taxonomy (Table 3) as a reusable blueprint to assess performance under conditions free of any potential text-level contamination.

In conclusion, LLM agents are best regarded as capable analytical assistants requiring a mandatory expert-verification layer rather than autonomous biostatisticians. While they can dramatically accelerate the initial draft stages of research, spanning writing protocols, generating questions, and drafting template code, their code execution remains prone to introducing silent, plausible-looking errors that can compromise clinical validity. Therefore, even when used as analytical support tools, their outputs require critical validation.

The broader implication is that safe adoption will depend not only on better models but also on rigorous, reusable evaluation frameworks. The utility of generative AI in clinical research is highly asymmetrical, benefiting experienced clinical data scientists who can quickly audit code but carrying high risks for novice researchers who cannot verify the underlying code or statistical logic. The “verification time-tax” required to check the implementation status of the SAP by the model significantly offsets any automation speedups, making autonomous AI deployment in peer-reviewed clinical research premature. Evaluating AI models based on execution success (ie, whether the code runs without throwing errors) is insufficient for safety-critical clinical informatics. Benchmarks must shift toward auditing logical correctness. This study yields a stage-based minimum evaluation framework for future LLM agents in clinical data analysis (Table 4). Although demonstrated in an ophthalmology dataset, the workflow and oversight logic are adaptable across clinical specialties.