Personality traits play a fundamental role in understanding individual differences, social behavior, and life outcomes [1-3]. However, their assessment remains challenging, as traditional self-report inventories are subject to response biases and may be limited in their ability to capture the richness of individual expression [4,5]. Prior research suggests that personality-relevant information is reflected in everyday language use, as individuals express stable patterns of thought and behavior in how they speak and write [6-8]. This perspective is grounded in the lexical hypothesis, which posits that meaningful aspects of personality are encoded in natural language and expressed through everyday communication [9-11].

Building on this foundation, recent advances in large language models (LLMs) have enabled personality prediction directly from text, offering scalable and context-sensitive alternatives to traditional assessment methods [12-18]. Existing studies suggest that LLMs can infer Big Five traits from language with modest accuracy, primarily using social media data or written corpora [19-21]. Approaches leveraging extended conversational data, such as interview transcripts, have shown improved performance, suggesting that richer, context-aware language better captures personality-relevant signals [16,22-24].

However, existing work remains limited in 3 key respects. First, many studies frame personality as categorical labels rather than continuous traits, limiting psychometric fidelity [14,25]. Second, most datasets rely on social media text [26-29], which may not reflect structured or task-oriented conversational contexts. Third, few studies jointly integrate ecologically grounded interview data [30], validated individual-level trait measures, and systematic comparisons of multiple LLM modeling paradigms under a unified evaluation framework. As a result, it remains unclear how well LLMs can predict personality from interview data in ways that align with established psychometric standards. Our prior research began to explore this challenge using prompt-based inference on the same interview-based benchmark [24], and the present study extends that work by incorporating supervised fine-tuning and representation-based regression approaches.

The Big Five, or Five-Factor Model, is the most widely accepted framework in personality psychology [31], comprising 5 broad trait dimensions: openness, conscientiousness, agreeableness, extraversion, and neuroticism. To address the gaps identified above, this study evaluates whether contemporary LLMs can predict continuous Big Five personality scores from semistructured interview transcripts paired with validated 10-item Big Five Inventory (BFI-10) self-report scores. This design better approximates naturalistic human communication and enables a more realistic assessment of model performance against psychometrically grounded benchmarks.

We compare 4 complementary modeling paradigms reflecting distinct assumptions about how personality signals are encoded and extracted from text. First, we examine prompt-based inference using GPT-5 mini (OpenAI) under both zero-shot and chain-of-thought (CoT) settings, representing inference without task-specific adaptation. Second, we evaluate parameter-efficient fine-tuning via low-rank adaptation (LoRA) applied to RoBERTa-base (Robustly Optimized BERT Approach; Meta AI) and LLaMA-2-7B (Large Language Model Meta AI; Meta Platforms, Inc). Third, we investigate supervised model adaptation through supervised fine-tuning of GPT-4.1 mini (OpenAI), which we evaluate across 2 independent training runs. Fourth, we assess embedding-based regression, in which fixed representations from 4 pretrained encoders (all-mpnet-base-v2 [Microsoft], E5-large-v2 [Microsoft], text-embedding-3-small [OpenAI], and text-embedding-3-large [OpenAI]) are used to predict personality traits via a multioutput ridge regression model. By contrasting these paradigms, we aim to disentangle the contributions of prompting strategy, representation learning, and different levels of model adaptation in personality inference and to identify which traits are most reliably captured from naturalistic conversational data.

In this study, we used real-world, semistructured interview transcripts from a pooled dataset of 555 adults across 5 prior investigations of adjustment to stress. The pooled source dataset included active-duty firefighters (n=103), patients with multiple sclerosis (n=61), adults recruited following injury or traumatic events (n=254), first-year college students (n=84), and community adults (n=53). Of the 555 participants in the pooled source dataset, 518 had complete BFI-10 responses and were included in the present analysis. The mean scores for the Big Five traits are as follows: openness (3.35, SD 0.63), extraversion (3.03, SD 1.09), neuroticism (2.93, SD 1.23), conscientiousness (2.74, SD 1.30), and agreeableness (2.63, SD 1.13), indicating that average trait levels were near the midpoint of the scale. Full descriptive statistics are reported in Table A7 of Multimedia Appendix 1.

The analytic sample was demographically diverse and balanced by sex (n=260 male and n=257 female participants), had minimal missing data, and spanned a wide age range (mean 39.32, SD 16.45). Participants represented various racial backgrounds (n=402 White and n=116 non-White/Other participants) and educational levels ranging from high school to college and above. Additional details regarding cohort characteristics and interview prompts are available via the Open Science Framework (OSF; Center for Open Science) repository [60]. The complete semistructured interview prompts are provided in Table A1 of Multimedia Appendix 1. All code used in this study is publicly available at the project repository [61].

The semistructured interview protocol varied across the 5 parent studies to reflect the specific context and circumstances of each sample. Depending on the parent study, participants completed 3 to 5 prompts, beginning with a neutral description of previous-day activities and followed by study-specific prompts concerning relevant experiences, coping, and/or positive and negative events. Participants were encouraged to speak spontaneously for up to 3 minutes in response to each prompt [62-64]. Interviews were audio-recorded and transcribed using established conventions. The resulting transcripts showed substantial variability in length. On average, participants produced 2955 (SD 1855) words per interview, with a range from 387 to 7143 words (median 2292, IQR 1569‐4388). Representative excerpts from 1 participant in the active-duty firefighter parent study are presented in Table 2.

To evaluate the capacity of LLMs for personality inference, we used the GPT-5 mini via the OpenAI API (model ID: gpt-5-mini-2025-08-07; [65]) to predict Big Five trait scores from naturalistic interview transcripts. The model was instructed to act as a psychologically insightful agent and to generate continuous scores between 1.0 and 5.0 for each of the five traits.

We implemented 2 prompting strategies. The first was a standard zero-shot prompt that directly instructed the model to return trait scores based on the interview content [66]. The second was a CoT prompt that encouraged intermediate reasoning about emotional tone and linguistic markers prior to scoring [67]. For prompt-based inference, transcripts were used in their conversational form, with interviewer prompts preserved to retain contextual structure. Both prompting strategies were executed using the same API configuration without specifying a temperature parameter. Model outputs followed a predefined score format, and trait scores were automatically extracted using rule-based parsing implemented with regular expressions. Compared with our prior work [24], the prompts were slightly refined with additional instructions while maintaining the same overall prompting structure. The complete zero-shot and CoT prompt templates, along with an example model output, are provided in the Textboxes 1 to 3 of Multimedia Appendix 1.

To improve prediction performance with respect to self-reported Big Five personality scores, we applied LoRA [50] for efficient fine-tuning of transformer-based language models. Here, r denotes the rank of the low-rank decomposition, representing the dimensionality of the trainable adaptation matrices. A larger rank increases model capacity but requires more parameters, whereas a smaller rank improves parameter efficiency. Specifically, we fine-tuned the open-source LLaMA-2-7B model [68] to jointly predict all 5 personality traits from the input text. LoRA modules are injected into the attention and feedforward layers, allowing for parameter-efficient adaptation while preserving the model’s general language capabilities.

Our implementation used LoRA fine-tuning with 4-bit quantization to reduce memory requirements. The training configuration used a learning rate of 1e-4, a per-device batch size of 2, gradient accumulation steps of 4, weight decay of 0.01, and 3 training epochs. The LoRA configuration used a scaling factor α of 64, a dropout rate of 0.05, and no bias terms. In the rank-sensitivity experiments, adapter ranks of 8, 16, and 32 were evaluated using the same configuration unless otherwise specified.

In addition to decoder-based models, we evaluated encoder-only architectures to assess the role of representation learning in personality prediction. We used RoBERTa-base [39] as a representative encoder model. Because of its limited input length, interview transcripts were tokenized and segmented into nonoverlapping 512-token chunks, with an overlap ratio of 0.0 and a stride of 512 tokens. Each chunk was encoded independently, and chunk-level embeddings were aggregated using mean pooling to form a global transcript representation. A lightweight nonlinear regression head was then trained on these pooled embeddings to predict continuous trait scores.

To examine whether supervised model adaptation improves personality trait prediction, we conducted supervised fine-tuning using a pretrained, closed-source LLM. Specifically, we used GPT-4.1 mini (April 14, 2025; [69]) as the base model and trained it on the personality prediction dataset.

Each training instance consisted of a participant’s textual responses as the input and the corresponding continuous scores for the 5 Big Five personality traits as the target outputs. The dataset was split into an 80% fine-tuning set and a 20% test set. Fine-tuning was performed using a supervised learning objective for 3 epochs, with a batch size of 1 and a learning rate multiplier of 2. No separate validation set was used.

To assess the consistency of the fine-tuning process, we conducted 2 independent training runs using the same hyperparameter configuration. These runs resulted in 2 models (FT1 and FT2), which were evaluated separately.

To complement prompt- and fine-tuning–based approaches, we evaluated static embedding-based personality inference across 4 pretrained encoders of varying scales and training paradigms: all-mpnet-base-v2 (768-dimensional, built on masked and permuted pretraining for language understanding) [70] from the Sentence-Transformers library; E5-large-v2 (1,024-dimensional) [71], trained with contrastive learning on large-scale text pairs and requiring a “passage” (prefix at inference time); and OpenAI text-embedding-3-small (1536-dimensional) and text-embedding-3-large (3072-dimensional), accessed via the OpenAI API [72]. This selection spans a broad range of model capacities and training paradigms, enabling systematic comparison among open-source encoders, contrastive-learning–based embeddings, and large-scale commercial models.

For all models, each participant’s interview transcript was encoded into a fixed-size dense vector at the transcript level. These embeddings, standardized to have 0 mean and unit variance, were passed to a multioutput ridge regression model (α=1.0) [73] that jointly predicted all 5 Big Five trait scores. The dataset was split into 80% training and 20% testing partitions. We further conducted a sample-size sensitivity analysis by training the text-embedding-3-small ridge regression model on increasing proportions of the training pool (20%, 40%, 60%, 80%, and 100%) and evaluating performance on the same held-out test set. The 20% to 80% conditions were repeated 5 times with different random subsamples; the 100% condition used the full training pool.

We assessed model performance using 2 complementary metrics: Pearson correlation (r) and mean absolute error (MAE). Pearson correlation quantifies the consistency between model-predicted and self-reported trait scores, serving as an indicator of criterion-related validity. Correlations above r=0.5 are typically interpreted as stronger evidence of convergence with self-report scores, indicating that the model captures meaningful individual differences in personality. Values between 0.3 and 0.5 suggest moderate agreement with self-report scores, while correlations below 0.3 suggest weak predictive consistency. MAE provides a direct measure of prediction error on the original trait scale (1.0‐5.0). Lower MAE values (eg, ≤0.5) reflect high precision, while values between 0.5 and 0.8 are common and may be acceptable depending on context. MAE values exceeding 1.0 typically signal poor calibration or unreliable estimates. Unlike correlation, MAE is sensitive to systematic bias and offers a more interpretable estimate of absolute prediction fidelity.

We also calculated sample-specific internal consistency for each-item trait scale and reported both Cronbach α and interitem correlations in Table A9 of Multimedia Appendix 1.

This study used deidentified semistructured interview transcripts and BFI-10 personality assessment data collected from 5 prior Kent State University Institutional Review Board–approved studies under protocol numbers #17‐056, #14‐233, #13‐134, #15‐446, and #12‐257 [60]. Participants provided informed consent in the original studies for the use of their data for research purposes. The present study involved secondary analysis of deidentified data and did not involve recontacting participants or collecting new identifiable information; therefore, additional informed consent was not required for this analysis.

To protect participant privacy and confidentiality, all textual and psychometric data were anonymized or deidentified before analysis and direct identifiers were not used in model training, evaluation, or reporting. Results are reported only in aggregate. Any interview excerpts included in the manuscript were shortened or edited to avoid revealing identifying details. No identifiable participant images, screenshots, or user-identifying information are included in the manuscript or supplementary materials. Participants in the original studies were financially compensated for their time at a rate of approximately US $10 per hour. The total compensation varied across the parent studies because study activities and participation requirements differed; study-specific procedures are described in the corresponding published reports [74-78].

Given the sensitive nature of personality prediction, the models evaluated in this study are intended for research purposes only. They should not be used to identify individuals, make diagnostic judgments, or support high-stakes decisions without appropriate validation, ethical oversight, and human review.

Table 3 summarizes the performance of GPT-5 mini using zero-shot and CoT prompting. Overall, prompt-based methods exhibited weak and inconsistent performance across traits. The highest correlation was observed for conscientiousness in the zero-shot setting (r=0.158). Most other traits, including neuroticism, openness, and extraversion, showed correlations near 0 or were negative, indicating limited predictive validity. Prediction errors remained high across all traits. Only openness achieved a MAE below 0.7, whereas MAE values for conscientiousness and agreeableness exceeded 1.4, indicating poor calibration of predicted trait scores.

LoRA-based fine-tuning substantially improved personality prediction performance compared to prompt-based methods, with effects varying across traits and adaptation ranks.

As shown in Table 4, performance gains were most pronounced for agreeableness, which showed statistically significant correlations across all configurations (r=0.299 at rank 8, r=0.329 at rank 16, and r=0.229 at rank 32). Openness showed a different pattern, with performance improving as rank increased and reaching statistical significance only at rank 32 (r=0.216, P<.05).

As shown in Table 5, embedding-based LoRA models achieved stronger performance than other LoRA configurations. Conscientiousness achieved the highest correlation (r=0.352 at rank 8), which represents the best result within this model family. Openness and neuroticism also showed relatively stable and statistically significant performance across ranks (r≈0.23‐0.26).

In contrast, extraversion and agreeableness were generally weaker, with only extraversion, at rank 32, reaching statistical significance. Increasing the LoRA rank did not consistently improve performance, as some traits achieved their best results at lower ranks. Overall, these findings suggest that embedding-based models captured some personality-relevant signals more consistently, although performance varied across traits.

As shown in Table 6, fine-tuning results were largely consistent for conscientiousness, agreeableness, and neuroticism across the 2 runs. Conscientiousness showed the strongest performance (r=0.686 and 0.718), followed by agreeableness and neuroticism. Extraversion was less stable across runs (r=0.389 vs 0.172), while openness remained weak and nonsignificant (r=−0.085 and −0.138). These results suggest that fine-tuning more reliably captured certain personality traits, such as conscientiousness and agreeableness, while others remain more difficult to infer from text.

As shown in Table 7, embedding-based models achieved moderate, trait-dependent performance across pretrained encoders. Overall, text-embedding-3-large performed best, yielding the highest correlations for conscientiousness (r=0.568), agreeableness (r=0.531), and neuroticism (r=0.249), all of which are statistically significant.

Across traits, conscientiousness and agreeableness were the most consistently well-predicted, with significant correlations across multiple encoders. Extraversion showed modest-to-moderate performance, with significant results for all-mpnet-base-v2 (r=0.302) and text-embedding-3-large (r=0.250), but weaker performance for other models. In contrast, neuroticism showed limited performance, reaching statistical significance only with the largest embedding model. Openness remained difficult to predict across all encoders, with weak and nonsignificant correlations, consistent with patterns observed in other methods.

To examine whether predictive performance improved with larger training samples, we conducted a sample-size sensitivity analysis using the text-embedding-3-small, embedding-based regression model as a representative embedding baseline. Increasing proportions of the training pool were used for model training (20%, 40%, 60%, 80%, and 100%; corresponding to n=82, 165, 248, 331, and 414), while evaluation was performed on the same held-out test set. As shown in Table A8 of Multimedia Appendix 1, increasing the training sample did not consistently improve predictive performance. Averaged across the 5 traits, Pearson correlations were 0.312, 0.326, 0.305, 0.300, and 0.267, respectively. At the trait level, conscientiousness and neuroticism generally decreased as the training sample increased, with their highest correlations observed at the 20% condition (r=0.581 and r=0.227, respectively), both higher than the corresponding full-training values. Agreeableness peaked at the 40% condition (r=0.546) and then declined. In contrast, openness increased modestly across the training fractions, while extraversion increased through the 80% condition before declining at 100%; however, extraversion correlations remained below 0.20 across all training conditions. This pattern may partly reflect the use of a fixed ridge regularization parameter without systematic hyperparameter tuning or a separate validation set.

Across all modeling paradigms, personality prediction performance exhibits 2 consistent patterns: strong dependence on model adaptation strategy and substantial variation across personality traits. Consistent with our prior work on these data [24], prompt-based approaches (zero-shot and CoT prompting) yield weak and unstable performance. In contrast, supervised learning or embedding-based regression demonstrate clear improvements by enabling models to learn from labeled data. In addition, prediction performance varies across the Big Five traits, suggesting that different personality traits may benefit from different modeling approaches.

First, prompt-based inference methods without task-specific adaptation (zero-shot and CoT prompting) showed weak and unstable performance in directly predicting continuous Big Five personality scores from semistructured interview transcripts. In contrast, Wright et al [30] reported moderate convergence between LLM-generated and self-reported personality trait scores based on spontaneous thought narratives and daily video diaries collected from 2 distinct samples. Compared with their unconstrained narrative data, our study evaluated personality prediction from semistructured interview transcripts, which may partly explain the lower zero-shot performance. In our benchmark, supervised approaches consistently outperformed prompt-based inference.

Second, performance varies systematically across traits. Conscientiousness and agreeableness are consistently the most predictable traits, achieving moderate-to-strong correlations across multiple methods, particularly under supervised fine-tuning (r=0.718 for conscientiousness and r=0.631 for agreeableness). Neuroticism shows moderate and relatively consistent performance, whereas extraversion is more variable across methods and runs. In contrast, openness shows the strongest and most consistent performance under embedding-based LoRA fine-tuning with RoBERTa-base (r=0.251‐0.258), while LLaMA-2-7B LoRA achieves modest but statistically significant performance (best r=0.216). Prompt-based methods, supervised fine-tuning, and embedding-based regression remain comparatively weak, suggesting that prediction performance for openness is highly method-dependent.

Finally, increasing model complexity does not uniformly improve performance. While supervised fine-tuning provides the strongest overall results, parameter-efficient methods such as LoRA yield only modest gains, and embedding-based approaches offer a competitive trade-off between performance and efficiency. The sample-size sensitivity analysis did not show a clear performance gain within the current training range; however, larger cohorts and systematic variation in transcript length are still needed to determine the data conditions required for stable personality estimates.

These results are consistent with prior work showing that narrative interview language can be used to predict Big Five personality scores when language models are trained on validated psychometric measures [79]. They also support recent findings that free-form language may require psychologically grounded supervision to connect linguistic cues with questionnaire-based personality constructs [80].

The stronger performance of supervised fine-tuning, compared with prompting and most parameter-efficient configurations, indicates that mapping language representations to validated personality constructs requires direct optimization. At the same time, the competitive performance of embedding-based regression highlights that meaningful personality signals are already present in pretrained representations but require appropriate extraction mechanisms. In our implementation, transcripts are segmented into fixed-length chunks (512 tokens) and aggregated using mean pooling, which may smooth localized linguistic signals in longer conversational inputs.

The observed trait-specific differences further suggest that not all personality dimensions are equally expressed in language. Traits such as conscientiousness and agreeableness appear to have stronger and more stable linguistic markers, whereas openness may depend on richer contextual or cognitive signals that are not fully captured by the available interview responses [81,82].

Although this study provides a systematic comparison of 4 modeling paradigms for personality inference from interview transcripts, several limitations constrain the generalizability of our findings.

First, although the dataset is based on real-world interview data, the sample size (n=518) remains modest relative to the parameter scale of large models, particularly decoder-based architectures such as LLaMA-2-7B. The sample-size sensitivity analysis did not show a clear performance gain within the current training range; however, future work should examine larger cohorts and systematically vary both participant sample size and transcript length to determine the data conditions needed for stable personality estimates. In addition, because the analytic sample pooled participants across occupational, clinical, student, and community cohorts, the findings may not generalize uniformly to any single population.

Second, several limitations also apply specifically to the embedding-based regression models. Variability in transcript length and the use of truncation, chunking, or mean pooling may lead to semantic information loss when long interview responses are compressed into fixed-size representations [83]. The ridge regression models were trained with a fixed regularization setting, without systematic hyperparameter tuning or a separate validation set for model selection; therefore, these results should be interpreted as baseline comparisons rather than fully optimized estimates of embedding-based model performance. Model performance was also evaluated using a single 80 to 20 train-test split rather than repeated k-fold cross-validation, which may make the reported estimates sensitive to the specific hold-out partition. Future work should use repeated cross-validation or external validation to better assess the robustness of these findings.

Third, the use of the short-form BFI-10 introduces inherent psychometric limitations. In the present sample, Cronbach α ranged from 0.429 for openness to 0.693 for conscientiousness, indicating nontrivial measurement noise in the self-reported criterion scores, which may attenuate observed model-criterion correlations and contribute to the apparent performance ceiling. It is also important to clarify that the criterion variables in this study are self-reported Big Five trait scores rather than personality in its full conceptual scope. Self-reports capture one form of personality-related information, partly reflecting individuals’ explicit self-concepts and self-images [5]. Accordingly, LLM-derived estimates from interview language should be interpreted as predictions of self-reported trait scores, not as direct measurements of personality itself.

Fourth, our evaluation of prompting methods was limited to zero-shot and CoT strategies, which do not fully capture the broader space of prompt design strategies. Future studies should investigate few-shot prompting, trait-specific instructions, and prompting strategies that explicitly incorporate emotional or contextual signals, particularly for traits such as neuroticism.

Fifth, while we explored parameter-efficient adaptation using LoRA, our analysis was limited to variations in rank under a fixed configuration. Other factors, such as layer-wise adaptation, learning rate scheduling, and integration with retrieval-based methods (eg, RAG), may further improve performance, particularly for effectively salient traits such as neuroticism.

Finally, all models evaluated in this study were general-purpose pretrained models. Domain-adapted models trained on psychologically relevant or interview-based corpora may encode more informative representations for personality inference. Future work should compare language-based estimates with longer personality inventories, informant reports, behavioral indicators, and other criterion measures to evaluate whether they provide complementary information beyond self-report.

This study shows that personality inference from natural language is both feasible and fundamentally constrained, revealing a pattern that is expressive but uneven and highly dependent on correspondence between language representations and psychometric constructs.

Contemporary LLMs encode meaningful personality-related signals but do not reliably translate these signals into calibrated trait estimates without explicit supervision. This pattern helps explain why models that appear linguistically fluent may nevertheless fail to produce stable or consistent personality predictions. Reliable personality inference therefore depends not only on model capability but also on how representations are aligned, adapted, and evaluated against validated psychological frameworks.

From a modeling perspective, personality prediction should be understood as a measurement problem rather than a purely generative task, in which performance is shaped by the interaction between data structure, model adaptation, and trait-specific linguistic expressivity. This reframing highlights that improving personality inference requires not only larger models but also a more principled integration of psychometric theory and representation learning.

Future work should extend this framework to more interactive and longitudinal settings, examining how personality signals evolve across extended conversations and diverse contexts. Additionally, incorporating richer personality measures and domain-adapted models may improve the reliability of trait-level inference.