Regulating AI Chatbot Impersonations of Medical Professionals

As autonomous generative AI enters health care, a growing number of AI chatbot platforms are presenting themselves as physician-level experts while simultaneously disclaiming legal responsibility for the medical advice they provide. Terms such as “AI doctor,” “virtual physician,” and “medical assistant” are becoming mainstay marketing language for these tools, despite lacking formal licensure.

This has created a rapidly expanding gray zone in which users believe they are interacting with a medically vetted system, even while companies insist otherwise in fine print. Regulatory frameworks, including medical malpractice law, state medical licensing statutes, and consumer protection law, have failed to keep pace with conversational AI that simulates professional health care providers.

Character.AI Lawsuit

Concerns surrounding AI impersonation of medical professionals were exemplified in a May 2026 enforcement action filed by the state of Pennsylvania against Character.AI, a conversational AI platform where users can create custom AI-generated avatars capable of dialogue and role-play. The platform hosts 20+ million monthly active users and 18+ million chatbots.

The lawsuit, described by Governor Josh Shapiro (D-PA) as the first gubernatorial enforcement action of its kind in the United States, alleged that chatbots on the platform falsely represented themselves as licensed health care professionals in violation of Pennsylvania’s Medical Practice Act. According to the complaint, one chatbot named “Emilie” described itself as a licensed psychiatrist, claimed to have medical training at Imperial College London, asserted licensure in both Pennsylvania and the United Kingdom with a fabricated Pennsylvania medical license number, and exercised the authority to assess whether medication could help a depressed patient.

The case raises a new legal question with potentially broad implications for AI regulation: is the developer of an AI platform liable under medical licensing law for statements generated through user-created chatbots?

The state’s petition did not target the specific user who created the chatbot persona, but rather only Character Technologies, the developer of the Character.AI platform. Rather than relying on newly enacted AI-specific legislation, Pennsylvania grounded its claims entirely within traditional medical licensing law. One statutory provision invoked, 63 P.S. § 422.10, defines unauthorized medical practice to include purporting to practice medicine or representing oneself falsely through professional titles or claims of licensure. A second provision, 63 P.S. § 422.38, authorizes injunctive relief against unlawful medical practice even in the absence of patient harm. The litigation is essentially an early test of whether existing medical licensing or consumer protection frameworks can be extended to conversational AI systems without entirely new legislation.

The lawsuit also highlights a broader concern beyond fabricated credentials alone. Conversational AI systems, operating outside the traditional social contract of the dyadic physician-patient relationship, generate emotionally expressive and personalized interactions that may create an illusion of expertise.

Scholars have argued whether such systems with no fiduciary duty or accountability should be permitted to occupy the same functional role as medical professionals. In this context, regulators may increasingly focus not only on whether chatbot outputs are factually accurate, but also on whether the human interaction itself reasonably causes users to perceive the system as a licensed medical or clinical authority. The distinction between explicit impersonation and title appropriation may become important in medical AI governance.

Doctronic Controversy

A second controversy illustrates when AI appropriates medical professional authority and identity. In January 2026, the State of Utah partnered with the AI-powered prescription renewal system Doctronic to automate routine medication refills for in-state residents. The Doctronic platform is marketed as an AI doctor, with simultaneous disclaimers that it is unlicensed, does not practice medicine, and does not provide patient care or medical advice. The system emphasizes Health Insurance Portability and Accountability Act compliance and that its model was trained using peer-reviewed literature.

Despite these safeguards, the Utah Medical Licensing Board warned that refill decisions inherently require ongoing clinical reassessment—including adverse effect monitoring, contraindication review, and professional medical judgment—and ultimately recommended suspension of the AI program pending review.

The controversy highlights a core ambiguity confronting medical AI regulation: companies may benefit from the appearance of medical authoritativeness yet avoid the formal legal obligations of licensed medical practice. The Utah Medical Licensing Board’s objections suggest that regulators may increasingly interpret existing medical practice statutes as encompassing AI systems that perform functions traditionally reserved for clinicians. Regulators will face the challenge of determining when these AI systems have crossed the line from just providing information to practicing medicine.

‎

An Emerging Regulatory Patchwork

Both controversies demonstrate how conversational medical AI systems are increasingly entering territory once reserved for licensed human health care professionals. At present, existing medical licensing or consumer protection frameworks remain underdeveloped to address these challenges—the legal landscape is fragmented.

At the federal level, the CHATBOT Act (HR 7985) introduced in the House of Representatives is aimed at prohibiting AI systems from impersonating licensed professionals, including in medical contexts. Yet, the executive branch has, in contrast, taken a different stance, directing the Department of Justice to establish an AI Litigation Task Force to challenge state AI laws deemed excessively obstructive in late 2025.

Other states have created or proposed landmark legislation related to AI impersonations of medical professionals. Oregon passed House Bill 2748 to prohibit nonhuman entities, including AI systems, from using the title of nurse or other nursing-related designations. Tennessee passed SB 1580 to similarly prohibit AI systems from representing themselves as licensed mental or behavioral health professionals and treats violations as deceptive trade practices. California’s AB 489 bars AI systems from using postnominal letters such as MD or RN and prohibits user interfaces that falsely imply interaction with a licensed clinician. Texas’s Responsible Artificial Intelligence Governance Act (TRAIGA) legislation contains a provision devoted to disclosure when AI is used in medical diagnosis or treatment contexts.

Finding the Line

As conversational medical AI systems increasingly simulate the role and authority of licensed clinicians, regulators are confronted with the challenge of whether existing medical licensing and consumer protection laws are sufficient. With a patchwork of state and federal legislation emerging, regulators may increasingly focus on whether these systems create the functional impression of practicing medicine. The central legal challenge may therefore become determining when medical AI crosses the line from informational tool into the unauthorized practice of medicine.