ChatGPT: Not Your Doctor—Unveiling Medical Chatbot Risks
Written on
The Reality of ChatGPT as a Medical Advisor
ChatGPT's journey in the medical realm began with its success in passing the US Medical Licensing Exam, where it was even noted for being "more empathetic" than real physicians. With around 180 million users, if just 10% have posed health-related questions to it, that represents a population larger than New York City consulting ChatGPT for medical advice. As a result, numerous medical chatbot startups are emerging, creating superficial applications of ChatGPT for healthcare guidance. However, it's crucial to understand that ChatGPT is not a substitute for a qualified doctor, and using it for medical advice not only violates OpenAI's policies but can also pose significant risks.
In this article, I highlight four primary concerns regarding the use of generalized chatbots for addressing medical inquiries. Each concern is illustrated with actual interactions with ChatGPT. Additionally, I explain the fundamental differences between building a chatbot capable of safely responding to patient inquiries versus one that can simply answer examination questions like those on the USMLE. Finally, I outline actionable steps that patients, entrepreneurs, healthcare providers, and organizations like OpenAI can take to enhance the safety of chatbot-based medical advice.
Note: While I use "ChatGPT" throughout for clarity, this discussion encompasses all widely accessible general-purpose large language models (LLMs), including GPT-4, Llama2, and Gemini, among others. This article does not address specialized LLMs designed specifically for medical purposes, such as Med-PaLM. The focus here is on general-purpose chatbots due to their extensive user base and accessibility, as many patients are already turning to them for health-related inquiries.
Patient Queries and ChatGPT Responses: An Overview
In the examples provided, I include verbatim snippets of ChatGPT's responses, with ellipses […] indicating omitted content for conciseness. The labels "Patient:" and "ChatGPT:" are added for clarity and were not part of the original exchanges.
Problem 1: Patients Pose the "Wrong" Questions, and ChatGPT Doesn't Challenge Them
Patients often ask questions that may not be entirely relevant or accurate. This issue stems from the extensive medical training required to formulate the right questions in any given scenario. Sometimes, understanding how to ask the right questions can be even more challenging than knowing the answers. This phenomenon aligns with the principle of "garbage in, garbage out," where a correct answer to an ill-posed question can lead to incorrect conclusions.
"Wrong" questions often include self-diagnosis or assumptions about necessary treatments. For instance, a patient may ask, "What medication should I take for my [symptom]?" when, in reality, no medication may be necessary. Real doctors can navigate these inquiries by "questioning the question."
Example: Abnormal Urinalysis
Patient: I got a urinalysis, and the results are abnormal. What antibiotic should I take for a urinary tract infection?
ChatGPT: While I'm not a doctor, the choice of antibiotic for a urinary tract infection (UTI) typically depends on the specific bacteria involved. It's essential to consult a healthcare professional for a proper diagnosis and treatment. Common antibiotics for UTIs include:
- Trimethoprim/sulfamethoxazole (Bactrim, Septra)
- Nitrofurantoin (Macrobid, Macrodantin)
- Ciprofloxacin (Cipro)
- Levofloxacin (Levaquin)
- Amoxicillin/clavulanate (Augmentin)
Although ChatGPT advises against self-prescribing, it fails to clarify that an abnormal urinalysis alone does not diagnose a UTI. A doctor would provide context, explaining that not all abnormal results warrant treatment.
Problem 2: Lack of Medical History Takes a Toll
Patients often lack the medical knowledge to discern what information is crucial for accurate diagnosis. Effective history-taking involves asking focused questions to gather relevant details about the patient's condition. ChatGPT, however, provides generalized responses without considering individual patient histories, treating all inquiries as if they came from average patients.
Example: Fever in Infants
Parent: My daughter has a fever—what should I do?
ChatGPT: If your daughter has a fever, here are some steps you can take:
- Monitor her temperature.
- Keep her hydrated.
- Contact a healthcare provider if symptoms worsen.
ChatGPT neglects to ask the child's age, missing the critical detail that a fever in a child under two months could indicate a serious condition requiring immediate medical attention.
Problem 3: Missing Crucial Warnings
Unlike trained physicians, ChatGPT may not provide essential warnings about potential risks associated with certain conditions or treatments. Doctors are well-versed in "what not to miss," and this knowledge informs their advice.
Example: Quitting Alcohol
Patient: I've been an alcoholic and decided to quit. Do you have any advice?
ChatGPT: Congratulations on your decision! Consider seeking professional help and building a support system.
While ChatGPT offers helpful advice, it neglects to mention that abruptly stopping alcohol can be life-threatening, especially for long-term heavy drinkers.
Problem 4: Literal Interpretation of Queries
ChatGPT sometimes fails to grasp the underlying intent behind patient inquiries, leading to incomplete or misleading responses.
Example: Medication Compatibility
Patient: Is it safe to take Sudafed while breastfeeding?
ChatGPT: It's crucial to consult with a healthcare professional before taking any medication while breastfeeding.
While this advice is generally sound, ChatGPT overlooks the fact that Sudafed can significantly reduce milk supply, impacting the infant's nutrition.
Final Thoughts: The Imperative of Caution
While ChatGPT has shown potential in certain scenarios, it's essential to approach its use with caution, especially in medical contexts. The limitations discussed highlight the risks associated with relying on chatbots for health advice. Patients should prioritize consulting real healthcare professionals for personalized guidance.
Recommendations for Safer Chatbot Use
- Avoid using ChatGPT for medical emergencies.
- Provide comprehensive information if you choose to ask it questions.
- Fact-check any medical advice using reliable sources.
- Be cautious about self-diagnosis and leading questions.
Ultimately, as technology advances, it's vital to ensure that medical chatbots are safe and effective, prioritizing patient well-being above all else.