Health systems need 'a voice' in shaping the future of AI
As health systems expand their use of artificial intelligence (AI), physicians and organizational leaders must consider how to adopt the technology without compromising clinical judgment, professional expertise, learning, or care quality, panelists said at Cleveland Clinic’s A.I. Summit for Healthcare Professionals.
Nick Ruthmann, MD, of Cleveland Clinic’s Heart, Vascular and Thoracic Institute, and Lara Jehi, MD, Cleveland Clinic’s chief research information officer, moderated a six-member panel on the realities of AI adoption in health care. They asked attendees to consider where their organizations fell along the adoption continuum—from early exploration and pilot projects to targeted deployment and routine use.
The discussion reflected the fact that AI integration is advancing at different speeds across health systems. Gilles Clermont, MD, professor of critical care medicine at the University of Pittsburgh School of Medicine, said uneven adoption is expected because AI tools differ widely in their intended uses and risk profiles. “People want to think in terms of AI ecosystem, [but we are] not there yet,” he said.
A separate challenge is the implementation process itself, according to Giovanni Cacciamani, MD, FEBU, an associate professor of research, urology, and radiology at the University of Southern California (USC) and director of the AI Center at USC's Institute of Urology. He noted that integrating an AI solution into an existing electronic medical record can take an average of 18 months without institutional support. “We can make all the efforts to find the best AI solution for decreasing our workload and improving performance at the point of care, but the biggest bottleneck is integration into our daily practice and workflows,” he said.
Building on that point, Robert Abouassaly, MD, MS, a urologist and institute chief of the Integrated Surgical Institute at Cleveland Clinic Abu Dhabi, suggested successful implementation may require health systems to rethink existing processes rather than simply layer AI onto workflows that were not designed for it. “We need to rethink everything and build solutions that are AI-first,” he said.
Deployment should also not be the endpoint, according to Abhinav Khanna, MD, a urologic oncologist and robotic surgeon at Mayo Clinic in Rochester, Minnesota. He said “blind spots” can arise in post-deployment monitoring.
“We have layers of committees to develop, validate, and internally deploy, but we don’t necessarily have a formal mechanism to go back and revisit that cycle after deployment,” he said. “That’s an area where we need to do a little bit better.”
Does AI improve thinking—or just productivity?
The discussion also raised a different question: What happens to human performance as organizations increasingly rely on AI?
Brian Uzzi, PhD, the Richard L. Thomas Professor of Leadership and Organizational Change at the Kellogg School of Management at Northwestern University, who studies the role of AI in mind and machine partnerships, said AI can raise the floor of creative performance but also suppresses the ceiling.
He cited research from Nature that examined scientific publications and found that authors using AI produced more work, but that their research became narrower and more incremental. “Each paper was less important, and I'm not sure science needs that,” he said.
Dr. Uzzi added that the same concern applies within health systems. Organizations using AI should distinguish between tasks where greater productivity is desirable and those where better performance should be the priority. “There are people in the organization [from whom] you don't want more productivity, you want higher quality,” he said. “For instance, you want surgeons to be better at making clinical diagnoses.”
That focus on quality also applies to how AI is used, Dr. Uzzi said. When the goal is to improve reasoning, he advised treating AI as a “sparring partner” that challenges rather than validates the user's thinking. “Don't say to the bot, ‘Hey, what do you think about this idea?’ Instead, say, ‘Give me three cases where my predictions are wrong. Give me disconfirming evidence,’” he said. “It's that friction and conflict of ideas that leads to better solutions.”
Can reliance on AI erode skills?
Asked whether reliance on AI could erode skills, Jonathan Chen, MD, PhD, director for medical education and artificial intelligence at Stanford University, responded with an analogy, comparing AI to a chainsaw. “It's a powerful tool, but if you don't know what you're doing with it, you will hurt yourself and other people,” he said. “It will undermine your learning, your professionalism, and all your practices.”
He illustrated that concern with an experience from his medical reasoning course. Students performed well on homework, but twice as many as usual failed a closed-book examination. Dr. Chen attributed the discrepancy to students relying on AI to complete their assignments. The students had missed an important part of the learning process, he said: struggling with the material. Rather than prohibiting AI, Dr. Chen continued to allow students to use it for homework but emphasized that they remained responsible for their own performance, whether on closed-book exams or during live patient encounters. The following year, students performed better on their exams.
Dr. Cacciamani said practicing physicians also need to consider “de-skilling,” or losing proficiency in skills they already possess because AI performs those tasks instead. Rather than using AI as a shortcut, Dr. Cacciamani argued that physicians should use it to “upskill”—reducing burdens while improving performance at the point of care. At the same time, he emphasized that empathy is an aspect of patient care that AI cannot replace.
What makes a good physician in the AI era?
The moderators closed with a broader question: As AI becomes increasingly integrated into medical practice, what will define a good physician 5 or 10 years from now?
Dr. Chen framed what physicians will continue to need around three attributes: competence, communication, and character. Competence still requires medical knowledge, he said, but also “judgment through practice and experience.”
Communication also extends beyond simply sharing information. “Who cares what you know if the patient won't take the medicine or the insurance won't pay for the procedure,” he said. “You have to be able to influence behavior.” Dr. Chen tied character to accepting responsibility for patient care, something he said cannot be transferred to technology. “A computer cannot take responsibility,” he said.
Dr. Abouassaly said future physicians should be judged by their ability to use available tools, whether AI or something else, to produce the best possible outcomes for patients.
Dr. Khanna focused on the growing number of AI tools physicians will need to evaluate. He said an increasingly important skill would be the ability to discern “signal from noise” and determine which tools meaningfully improve care, which provide incremental gains, and which add little value.
Dr. Uzzi concluded with a concern about who is shaping the technology physicians are being asked to use. He cautioned that the priorities driving AI development may not always align with the qualities physicians consider essential to good medicine. “We want a great doctor, but the people who build the machines don't have any of that in mind,” he said.
For that reason, Dr. Uzzi said health care needs to take an active role in shaping how AI is developed and used. Otherwise, priorities, such as quality, could become harder to preserve as AI development pushes toward greater productivity and automation. “You have to get to the table, and you have to have a voice,” he said.
AACE Endocrine AI is published by Conexiant under a license arrangement with the American Association of Clinical Endocrinology, Inc. (AACE®). The ideas and opinions expressed in AACE Endocrine AI do not necessarily reflect those of Conexiant or AACE. For more information, see Policies.