Empathy, a fundamental aspect of human interaction, can be characterized as the ability to experience the emotions of another being within oneself. The origin of the word “empathy” dates back to the 1880s, when Theodore Lipps determined the word “einfuhlung” (“in-feeling”) to describe the emotional appreciation of another’s feelings [1]. Empathy involves recognition of others’ feelings, the causes of these feelings, and the ability to participate in an emotional experience of an individual without becoming part of it [1].

Empathy is described as “the ability to see the world through someone else’s eyes,” having the ability to imagine what someone else is thinking and feeling in a given situation [2]. It is commonly understood to encompass cognitive and affective components: the ability to understand another’s feelings (cognitive empathy) and to experience emotions in response to others (affective empathy) [1,3].

In health care, empathy has an important role in patient care, improving patient satisfaction and treatment adherence. Empathy allows health care professionals to understand the emotional and psychological states of patients, fostering better communication and trust [4].

Large language models (LLMs) have demonstrated remarkable capabilities across various tasks, including text summarization, question-answering, and text generation [5]. There are numerous studies on potential applications in health care, as an educational tool and as a support tool in clinical work [6,7]. These models are already being integrated into practice. For instance, Epic has integrated GPT4 in its electronic health record software [8,9].

While LLMs have the potential to improve and automate some medical tasks, there are significant limitations to these models and their integration [10,11]. Despite the promising natural language processing capabilities, these models make errors and their performance in clinical tasks is challenging to evaluate on a large scale [12]. Many studies thus rely on multiple-choice questions assessment, which do not reflect real-world clinical applications [13]. These models can introduce bias [14], and can be susceptible to cyberattacks [15]. Some studies that evaluated these models for medical tasks suggested that despite impressive capabilities, LLMs lack empathy, a quality that is unique to humans and is imperative in health care [16-19].

Recent studies discuss and evaluate LLMs performance in tasks related to emotional intelligence, theory of mind, and empathy [20-25]. Some evidence suggests that these models may show aspects of cognitive empathy, including emotions recognition and providing supportive responses [17,26-28]. Furthermore, commercial LLM-based applications are being developed to offer emotional support to patients [29]. Given these developments, the aim of our study was to systematically review the literature on the capacity of LLMs in demonstrating empathy.

We searched the literature on LLMs and empathy using MEDLINE, Google Scholar, PsyArXiv, medRxiv, and arXiv. Studies published between December 2022 and February 2024 were included. The search query was “((“large language models”) OR (llms) OR (gpt) OR (chatgpt)) AND ((empathy) OR (“emotional awareness”) OR (“emotional intelligence”) OR (emotion)) OR ((“social robots”) OR (“artificial emotional intelligence”) OR (“emotional artificial intelligence”) OR (“emotional chatbots”) OR (“affective computing”) OR (HRI) OR (“Human robot interaction”)).” We also searched the references lists of relevant studies, including some key studies from major medical journals, for any additional studies that may have been missed during the initial search.

The inclusion and exclusion criteria are detailed in Table 1. Two reviewers (VS and EK) independently performed the search and screened the titles and abstract of the articles resulting from the search. Differences in search results were resolved through discussion to reach a consensus. The reviewers then screened selected articles’ full text for final inclusion. Ultimately, 12 publications were included in this review. The results of the included studies including the LLMs used, performance in empathy tasks, and limitations of the models, along with studies’ publication details, authors, and other relevant information were systematically summarized in a table.

Figure 1 presents a flow diagram of the screening and inclusion process. All 12 studies included in this review were published in 2023. Six studies compared ChatGPT-3.5 with other LLMs including GPT-4, versions of LLaMA, and fine-tuned chatbots. Six studies evaluated only ChatGPT-3.5. Seven studies evaluated empathy in ChatGPT in medical context. The results of the studies included are summarized in Table 2. This table provides a detailed summary of studies included in this review that evaluate aspects of empathy exhibited by large language models. The table outlines each study’s objectives, the specific large language model used key findings from the evaluations, sample sizes, and the methods used to assess empathy. It also highlights whether the reviewers were blinded to whether responses came from large language models or humans. The limitations of the LLMs as detailed in the different studies are detailed in Table S1 in Multimedia Appendix 1.

Empathy is essential in medicine, particularly when breaking bad news to patients. It allows physicians to deliver difficult information in a manner that respects the patient’s emotions and perspective. Webb [23] used ChatGPT to simulate a role play for breaking bad news in the emergency department. The chatbot successfully set up a training scenario, role played as a patient and provided clear feedback through the application of the SPIKES (Setting up, Perception, Invitation, Knowledge, Emotions with Empathy, and Strategy or Summary) framework for breaking bad news [23]. In another study, Yeo et al [31] tested ChatGPT’s ability to provide emotional support to patients diagnosed with hepatocellular carcinoma, and their caregivers. ChatGPT was able to acknowledge the likely emotional response of the patient to their diagnosis. Furthermore, the chatbot provided clear and actionable starting points for a newly diagnosed patient and offered motivational responses encouraging proactive steps. For caregivers, ChatGPT provided psychological and practical recommendations [31].

Ayers et al [26] compared the quality and empathy of responses given by ChatGPT and physicians with 195 randomly drawn patient questions from a social media forum. The study found that patients preferred the chatbot’s responses over physician responses in 78.6% of cases. ChatGPT’s responses were rated significantly higher for both quality and empathy, while physician responses were 41% less empathetic than the chatbot responses. The authors noted that ChatGPT tended to provide more lengthy responses, which could potentially be erroneously associated with greater empathy. They concluded that the chatbot may have potential in aiding drafting responses to patient questions [26].

Another study also assessed empathy in chatbot’s responses to patient’s questions. Liu et al [32] developed a model based on a pretrained LLaMA-65B and finetuned to generate physician-like responses that are professional and empathetic. They evaluated the model on 10 actual patient questions in primary care and compared the responses with those generated by ChatGPT-3.5 and GPT-4, rating them based on empathy, responsiveness, accuracy, and usefulness. When evaluating empathy, GPT-4 and ChatGPT-3.5 outperformed their model. Interestingly, all language models outperformed physician-generated responses significantly [32].

Understanding and addressing patients’ emotions is fundamental in mental health. Chen et al [27] used ChatGPT-powered chatbots to simulate psychiatrists and patients in clinical psychiatric scenarios. The chatbots showed potential in simulating some aspects of empathy. However, they sometimes forgot initial instructions and repeated general empathy phrases too often. They also asked fewer in-depth questions about symptoms compared with physicians, potentially affecting their ability to fully understand the patient’s condition. When simulating patients, the chatbots reported symptoms inaccurately [27].

The Levels of Emotional Awareness Scale (LEAS) is a psychological tool that assesses an individual’s capacity to identify and describe emotions in themselves and others, a fundamental aspect of empathy [37]. Elyoseph et al [28] compared the LEAS score of ChatGPT to the general population norms. They found that ChatGPT demonstrated significantly higher emotional awareness performance. When repeating the test following 1 month interval, the chatbot’s performance further improved, almost reaching the maximum possible LEAS score. The authors propose that ChatGPT could be helpful for cognitive training of people with emotional awareness impairment, as well as for psychiatric assessment support [28].

Zhao et al [30] compared ChatGPT with supervised models in terms of emotional dialogue understanding and generation. The tasks they assessed included emotion recognition, emotion cause recognition, dialog act classification, empathetic response generation, and emotional support conversation. The authors found that while supervised models surpassed ChatGPT in emotion recognition, ChatGPT produced longer, more diverse, and context-specific responses, especially when interacting with users in negative emotional states. Interestingly, Zhao et al [30] also observed a repetitive pattern in ChatGPT’s empathy expressions, similar to the results described by Chen et al [27].

Brin et al [33] evaluated ChatGPT and GPT-4 on USMLE (United States Medical Licensing Examination) questions involving communication skills, ethics, empathy, and professionalism. They have used questions from the USMLE website and the AMBOSS question bank and compared the performance of the LLMs with the reported performance at the AMBOSS website. GPT-4 correctly answered 90% of questions, outperforming ChatGPT and humans [33]. Huang et al [34] evaluated emotional responses of 5 different LLMs to various situations designed to evoke emotions, The LLMs’ responses were compared with human responses collected from 1266 participants worldwide. The authors reported for each model the changes in emotion scores relative to human benchmarks. They conclude that the different LLMs generally demonstrate appropriate emotional responses. However, none of the models exhibited strong alignment with human references. GPT-3.5-turbo demonstrated the highest alignment in the scores after imagining being in the situations. The 13B version of LLaMA-2 exhibited the strongest comprehension of human emotions.

Chen et al [25] constructed an empathetic conversation dataset of over 2 million samples and used it to fine-tune an LLM to provide empathetic responses. Their finetuned LLM outperformed other LLMs including ChatGPT in responses’ coherence and relevancy, as well as empathy, helpfulness and safety.

Blekhir and Sadat [35] evaluated whether prompt engineering and external emotion classifier can enhance empathy in ChatGPT’s responses. The study evaluated 2 versions of ChatGPT: 1 incorporating user emotions with an emotion classifier and another adapting to emotions without external tools. They evaluated these versions against the standard ChatGPT, demonstrating that tailored emotional responses significantly improve ChatGPT’s empathetic capabilities [35].

Qian et al [36] evaluated ChatGPT compared with other deep-learning models trained for empathetic interactions. They also propose 3 improvement methods including semantically similar in-context learning, 2-stage interactive generation, and combination with knowledge base. These methods improved the quality of responses generated by ChatGPT, which outperformed other models evaluated [36].

Lee et al [38] used Chain-of-Empathy prompting to reason emotion and situational factors that may assist the model to infer the emotional experience. They evaluated GPT-3.5 and compared 4 unique prompts that used Chain-of-Empathy in generating empathetic responses to Reddit posts. The Chain-of-Empathy strategy resulted in improved the model’s empathy expression [38].