Illustrated by Maura Roche/Axios
ChatGPT, the generative AI juggernaut, is getting smarter when it comes to healthcare.
Important reasons: Many clinical diagnoses and decisions will one day be made by machines rather than human doctors.
News promotion: As part of a recent research experiment, ChatGPT recently passed all three parts of the US medical licensing exam, but just barely.
- As the researchers point out, second-year medical students often spend hundreds of hours preparing for part one, while part three is usually taken by medical graduates.
Zoom in: Ansible Health, a Silicon Valley startup focused on treating COPD, was researching various AI and machine learning tools to improve treatment.
- Ansible CEO and former Google Product Manager Jack Po explains:
- “When we started testing, we were pretty amazed by the results. Not only did it get it right, but how it explained itself.”
- Po and a few others decided to let ChatGPT get USMLE, first making sure that “no answers, explanations, or related content were indexed by Google.” They then published their results, which are now under peer review.
big surprise ChatGPT was able to perform very well without being trained on the medical dataset.
- One caveat is that the researchers excluded a set of ‘indeterminate’ responses. ChatGPT appears to be programmed not to provide anything that could be construed as medical advice.
- “These answers were so general that it was hard to tell if they were right or wrong,” said co-author of the paper, an investor at Bessemer Venture Partners and now a medical student at Brown University. One Morgan Cheatham explains.
Line spacing: Generative AI is still in its early stages, so for now it will augment rather than replace healthcare operations.
- For example, Ansible uses ChatGPT to explain specific concepts to patients after being reviewed by trained experts.
What’s next: Over time, it may be applied to medical examinations and other general practitioner tasks.
- As technology moves beyond just text, it can incorporate data inputs such as tone of voice, body language, and facial expressions.
- One of the benefits is the ability to instantly capture patient medical records. Doctors sometimes only have a moment to scan a lifetime of charts.
Reality check: Don’t expect machines to be able to diagnose patients autonomously anytime soon. AI models like ChatGPT can make confident claims that can turn out to be false. This could prove dangerous in medical applications.
What they say: Vijay Pande, healthcare investor at Andreessen Horowitz and adjunct professor of bioengineering at Stanford University, said:
- “In 2000, it was insane to think a computer could beat a master trader on Wall Street. Today, it’s insane to think a master trader can beat a computer.”
To the point: Many people turn to “Dr. Google” for their medical information needs. I may rely on “Dr.ChatGPT” in the future.
Editor’s Note: This article has been revised to reflect that Part 1 of the USMLE is typically taken two years after medical school.
