Charting a Course for AI in Medical Education

From applications that help physicians make clinical decisions to ambient scribes that interpret patient conversations, AI’s presence is already being felt in the clinic. The technology is starting to streamline processes and help physicians deal with heavy caseloads. To prepare students for this new clinical world, medical schools are beginning to incorporate AI into their classes. The technology could help educate students on clinical concepts and ease the teaching burden of faculty.

“When used well and deliberately, AI can give students access to far more cases and knowledge than any single hospital or medical school can provide, offer instant feedback, and adapt to each student’s level in a personalized way,” says Nan Liu, PhD, associate professor at the Centre for Biomedical Data Science at the Duke-NUS Medical School in Singapore.

Despite these benefits, many experts worry that students who rely on AI instruction could treat it as a crutch and never learn important critical thinking skills, a concept called “never-skilling.” A related concern is that students trained with erroneous information produced by AI models could become “mis-skilled,” and once they enter the clinic, they could lose what skills they’ve learned by relying too heavily on the technology, a concept called “de-skilling.”

Several papers warn of these potential risks, but few data exist, and reports often seem to contradict each other. One oft-cited study compared the proficiency of endoscopists before and after a period when they had conducted AI-assisted procedures. They had a slightly lower adenoma detection rate after the AI period, suggesting deskilling occurred. However, another report showed that the error rates of radiology residents who had examined X-rays with AI improved.

“To be fair, these two studies are not really comparable because they use different populations, different designs, and different specialties, but that’s kind of the point,” says Laurah Turner, PhD, associate dean for Artificial Intelligence and Educational Informatics at the University of Cincinnati College of Medicine. “The reason we don’t have a clean, matched comparison of these comparative frameworks and the workflow is because the research hasn’t actually been done yet; we just have sparks of signal.”

It’s Not What You Do, It’s How You Do It

Cognitive learning theory proposes that learning is an active process in which people make new connections by paying attention, remembering concepts, and solving problems. In other words, the brain needs to wrestle with information to make it stick. When people offload difficult problems to AI models for quick answers, they are not going through this process, and the information may not become second-nature.

“When AI is deployed as a substitute for cognitive effort, we actually see de-skilling,” says Turner. “But when we see it deployed as a scaffold that actually demands cognitive effort, we are seeing sparks of potential upskilling. I think that’s really telling, and anecdotally in our research and in our use cases, we see the exact same thing.”

In a recent perspective article, Liu and his team propose that never-skilling most likely occurs through three processes. Competency acquisition failure can happen when AI supplies the answers so the student never has to develop appropriate clinical reasoning skills. A calibration deficit occurs when learners do not know when to trust outputs and when to verify them through other means. With metacognitive erosion, students aren’t able to take a step back and monitor their own reasoning and recognize uncertainty.

‎

To maintain critical thinking skills, some common educational exercises may require a slight tweak. For example, Turner explains that faculty at the University of Cincinnati were upset that students were using ChatGPT-generated text in reflective assignments. Instead of banning the students from interacting with AI for the assignment, Turner modified an AI coach she had previously developed so that it walked students through the reflective practice. She is currently reviewing the results of this deployment.

Honing interviewing skills could be another good application of the technology, according to Andy Qiao, a fourth-year medical student at the New York University Grossman School of Medicine. In his psychiatry clerkship, he appreciated the opportunity to use AI tools to practice communicating with patients. “Interviewing patients who are presenting for psychiatric concerns is quite different from medical interviews in other contexts,” he says. “I think having a tool to practice those skills in a safe, simulated setting before I was on the wards was really helpful and reduced the anxieties I had going into these clinical spaces.”

Get the Timing Right

Although researchers tend to agree that developing and nurturing critical thinking skills is crucial to the successful implementation of AI in medical education, not everyone is on the same page with regard to the timing. Some educators are diving in headfirst, having students test-drive apps in real classes, while others propose a more conservative approach.

Turner’s team laid the foundation for their first AI application for a clinical skills course over a single weekend and had it ready to go in the fall of 2022—the same year that ChatGPT brought large language models to the public. Now, her college includes courses with exercises based on AI technology that students take throughout their entire educational journey, from first-year basic science classes to later-year clerkships. AI also helps faculty with curriculum mapping to align courses with specific goals for reaccreditation. Some of Turner’s team’s technologies are currently being developed for use by practicing physicians.

AI technologies that Turner’s lab develops are cocreated and tested by learners before being deployed into courses. When they are about 80% certain that a technology is where they want it to be, they implement it with members of her team, observing and analyzing how the students use it.

Liu, who teaches and conducts research on the ethical and responsible use of AI, offers an alternative perspective. He and his team propose a phased training framework. “Students must demonstrate that they can reason independently before using AI as a clinical tool,” he says. “Then, [we should] introduce AI alongside structured exercises where students identify and correct its errors, building the habit of critical evaluation.” In later stages, students should learn to collaborate wisely with AI, determining when to trust it and when to use their own knowledge and skills to solve a problem, he says.

Looking Ahead

As a tool, AI is becoming more pervasive in everyone’s lives and physicians are no different. In the end, researchers studying AI’s use in medical education will need to keep the goal of educating the next generation of competent physicians in mind.

“I’m excited to see how the field continues to grow,” says Qiao. “Hopefully, it will be done in a way that’s thoughtful and sustainable and ultimately is at the benefit of the student, empowering them to develop skills in a way that wouldn’t be possible without AI tools, while avoiding issues of never-skilling and de-skilling.”