The AI Health Assistant Rush: Why ChatGPT Health and Claude for Healthcare Mark a Pivotal—and Precarious—Moment for Medicine

The first week of January 2026 may well be remembered as the moment when AI health assistants moved from experimental curiosity to mainstream reality—for better or worse. Within days, both OpenAI and Anthropic unveiled dedicated healthcare products that promise to fundamentally reshape how millions of people engage with medical information and healthcare systems. But as someone who has spent fifteen years at the intersection of artificial intelligence and healthcare, I find myself experiencing equal parts excitement and deep concern about what I’m witnessing.

On January 7, OpenAI announced ChatGPT Health, revealing the staggering statistic that 230 million people already ask health and wellness questions on their platform each week. Just five days later, Anthropic countered with Claude for Healthcare, positioning their offering as more sophisticated and enterprise-focused. Meanwhile, Google has been quietly removing AI Overviews from certain medical queries following investigations that revealed they were providing potentially dangerous misinformation.

This convergence of events crystallizes the central tension in AI healthcare: tremendous potential colliding with serious risks, all happening at Silicon Valley speed rather than at the measured pace medicine traditionally requires.

The Scale of What’s Already Happening

#

Let’s start with the numbers that should give us all pause. OpenAI’s disclosure that 230 million people per week are already asking ChatGPT about their health isn’t just impressive—it’s alarming. That’s more patient interactions than most healthcare systems see in a year, happening largely outside any clinical oversight or regulatory framework.

These aren’t trivial queries about general wellness tips. People are describing symptoms, asking about medication interactions, seeking guidance on whether they should go to the emergency room, and making decisions that could have life-or-death consequences. They’re doing this because our healthcare system has failed them in fundamental ways—appointments are hard to get, costs are prohibitive, and even when you can see a doctor, visits feel rushed and impersonal.

OpenAI’s CEO of Applications, Fidji Simo, frames ChatGPT Health as addressing “existing issues in the healthcare space, like cost and access barriers, overbooked doctors, and a lack of continuity in care.” She’s not wrong about the problems. The question is whether large language models are the right solution—or whether we’re applying a powerful but fundamentally flawed technology to challenges that require something more robust.

What These Systems Actually Offer

#

Both ChatGPT Health and Claude for Healthcare share some common features. They create dedicated spaces for health conversations, separate from other AI interactions. They can integrate with personal health data from devices and apps like Apple Health and MyFitnessPal. Both companies promise they won’t use health conversations to train their models—a crucial privacy commitment, though one that requires trust in their data handling practices.

But there are important differences in their approaches. ChatGPT Health appears more consumer-focused, essentially creating a silo for health discussions within the broader ChatGPT experience. If you start a health conversation outside the Health section, the AI nudges you to switch over, maintaining context separation.

Anthropic’s Claude for Healthcare takes a more enterprise-oriented approach with what they call “agent skills” and “connectors.” These integrations give Claude access to authoritative databases including the Centers for Medicare and Medicaid Services Coverage Database, ICD-10 diagnostic codes, the National Provider Identifier Standard, and PubMed. Anthropic positions Claude as a tool for providers and payers, not just patients.

The most compelling use case Anthropic presents is automating prior authorization review—the bureaucratic nightmare where doctors must submit documentation to insurance companies for medication or treatment approval. Mike Krieger, Anthropic’s Chief Product Officer, notes that “clinicians often report spending more time on documentation and paperwork than actually seeing patients.” He’s absolutely right, and if Claude can genuinely reduce this administrative burden, that would be tremendously valuable.

The Hallucination Problem We Can’t Ignore

#

Here’s where we need to talk about the elephant in the room: large language models fundamentally work by predicting the most likely next word in a sequence, not by determining what is actually true. They don’t have a concept of factual correctness. They generate plausible-sounding text based on patterns in their training data, and sometimes—too often for medical applications—they confidently state things that are completely wrong.

This isn’t a minor technical glitch we can fix with better prompting or more training data. It’s intrinsic to how these models work. Even OpenAI’s own terms of service state that ChatGPT is “not intended for use in the diagnosis or treatment of any health condition.”

The recent incident with Google’s AI Overviews illustrates exactly why this matters. The Guardian’s investigation found that when users asked “what is the normal range for liver blood tests,” Google’s AI provided numbers without accounting for factors like age, sex, ethnicity, or nationality—potentially leading people to believe abnormal results were healthy, or vice versa. This isn’t theoretical harm; this is the kind of misinformation that could delay crucial treatment or cause unnecessary panic.

Google’s response—removing AI Overviews from some medical queries but not systematically addressing the underlying problem—shows how difficult it is to make these systems safe. Even with the removals, variations on problematic queries could still trigger misleading AI summaries.

The Dangerous Illusion of Medical Competence

#

What makes AI health assistants particularly treacherous is that they’re exceptionally good at sounding knowledgeable. The same language modeling capabilities that allow them to write compelling prose also enable them to construct medically-sounding explanations that feel authoritative, even when they’re dangerously wrong.

In my research, I’ve seen how even doctors can be fooled by confidently stated AI-generated medical information. The technology creates what I call “competence theater”—it performs expertise without actually possessing it. And for patients desperate for answers, struggling with symptoms doctors have dismissed, or unable to afford proper care, that performance can be dangerously convincing.

The power dynamics are troubling. These systems are being deployed to populations who are already vulnerable—people who can’t access traditional healthcare, who don’t have the medical literacy to spot errors, or who are in crisis and grasping for any guidance they can find. When OpenAI and Anthropic warn users to seek professional medical advice, they’re essentially placing responsibility on patients to know when AI guidance is insufficient—a burden that’s unrealistic and unfair.

Where AI Can Actually Help (And Where It Can’t)

#

I don’t want to be entirely doom-and-gloom here. There are legitimate applications where AI can improve healthcare without the catastrophic risks of direct patient advice.

The prior authorization use case Anthropic highlights is genuinely promising. This is administrative work that doesn’t require diagnostic judgment—it’s pattern matching and paperwork, exactly what AI is good at. If Claude can reduce the hours doctors spend fighting with insurance companies, that creates more time for actual patient care. Similarly, helping researchers quickly search PubMed, organizing medical records, or flagging potential drug interactions in existing treatment plans—these are valuable applications with appropriate human oversight.

But there’s a crucial distinction between AI as a tool for healthcare professionals and AI as a replacement for professional judgment. The former can be powerful and beneficial. The latter is where we’re headed into genuinely dangerous territory.

The fundamental problem is that medicine isn’t just about information retrieval—it’s about judgment, context, and the kind of nuanced pattern recognition that comes from years of clinical experience. A good doctor doesn’t just know what diseases exist and their symptoms; they understand how different conditions present in different populations, how social determinants of health influence outcomes, when seemingly unrelated symptoms might be connected, and when a patient’s “vague feeling something is wrong” deserves investigation even if all the tests look normal.

These are the skills that save lives, and they’re not reducible to statistical patterns in training data.

The Regulatory Vacuum

#

Perhaps most concerning is that these products are launching into a regulatory vacuum. The FDA has frameworks for evaluating medical devices and diagnostic tools, but AI assistants that don’t claim to diagnose or treat (even while effectively doing exactly that) occupy a gray area. They’re marketed as information tools, even as millions of people use them to make medical decisions.

We need regulatory frameworks that can keep pace with AI deployment. But that’s enormously challenging when the technology evolves faster than rule-making processes can adapt, and when companies can reach global audiences overnight. By the time we develop appropriate oversight, these tools will already be deeply embedded in how people manage their health.

There’s also a business model problem lurking here. Both OpenAI and Anthropic are venture-funded companies that need to demonstrate growth and eventually profitability. Healthcare represents an enormous market—the U.S. alone spends nearly $5 trillion annually on health. The financial incentives to expand these products are massive, potentially overriding cautious, safety-first deployment.

What Happens When Things Go Wrong?

#

Here’s a scenario that keeps me up at night: A person experiencing chest pain asks ChatGPT Health whether they should go to the emergency room. The AI, basing its response on statistical patterns rather than the ability to actually assess the patient, suggests it’s probably anxiety and recommends relaxation techniques. The person, relieved to avoid an expensive ER visit, stays home. They’re having a heart attack.

Who’s responsible? The patient, for not seeking professional care? OpenAI, for deploying a system that gave dangerous advice? The healthcare system, for being so inaccessible that people turn to chatbots in emergencies?

Or consider someone who gets misleading information about medication dosing, or a parent whose child’s symptoms are misinterpreted, or a patient whose serious condition is normalized by an AI that doesn’t recognize an uncommon presentation of a common disease.

These aren’t hypothetical scenarios—they’re inevitable outcomes of deploying hallucination-prone systems for medical guidance at scale. The question isn’t whether harm will occur, but how much, and what we’ll do about it when it does.

The Path Forward: Honest Reckoning Required

#

If I could mandate one thing, it would be brutal honesty about what these systems can and cannot do. Not disclaimer text buried in terms of service, but prominent, unavoidable warnings whenever health advice is being given. Something like: “This AI cannot actually determine what is wrong with you. It might be right. It might be dangerously wrong, and it has no way to know which. Use it only for general information, never for decisions about your health.”

Would such a warning reduce the utility and adoption of these tools? Absolutely. That’s precisely the point. We should be reducing inappropriate use of technology that poses serious risks.

Beyond warnings, we need several things to happen quickly:

First, rigorous, independent evaluation of these systems’ accuracy across diverse populations and health conditions. Not company-conducted studies, but research by medical institutions with no financial stake in the outcomes.

Second, clear regulatory frameworks that classify what these tools are actually doing (providing medical advice) rather than what companies claim they’re doing (offering information). If they function as diagnostic or treatment tools, they should be regulated as such.

Third, genuine accountability mechanisms. When these systems give harmful advice, there needs to be recourse and responsibility, not just shrugged shoulders and reminders that users were warned.

Fourth, a hard look at why we’re turning to AI for healthcare in the first place. The underlying problems—access, cost, provider burnout—are policy failures, not technology problems. We’re using AI as a band-aid for systemic wounds that need actual surgery.

The Bigger Picture: AI in Healthcare at a Crossroads

#

The January 2026 launches of ChatGPT Health and Claude for Healthcare represent a fork in the road for AI in medicine. We can continue down the path of rapid deployment, treating healthcare as just another market to disrupt, learning from failures after people are harmed. Or we can insist on a more measured approach that prioritizes patient safety over market share.

The optimist in me wants to believe these tools can be part of solutions to healthcare’s genuine challenges. The AI researcher in me sees the technical limitations that make truly safe deployment extraordinarily difficult. And the doctor in me understands that medicine’s core duty—first, do no harm—cannot be compromised in the rush to innovate.

We’re at a pivotal moment. The decisions we make now about how to develop, deploy, and regulate AI health assistants will shape medicine for generations. The stakes couldn’t be higher, because we’re not just building technology—we’re determining how millions of people will receive guidance about their health, their lives, their loved ones.

My hope is that we can harness AI’s genuine capabilities—pattern recognition, information synthesis, administrative automation—while maintaining appropriate boundaries around the irreplaceable value of human clinical judgment. But that requires something Silicon Valley traditionally struggles with: accepting that some problems shouldn’t be solved by moving fast and breaking things. Because in healthcare, the things that break are people.

References

#

• TechCrunch (January 7, 2026). “OpenAI unveils ChatGPT Health, says 230 million users ask about health each week.” https://techcrunch.com/2026/01/07/openai-unveils-chatgpt-health-says-230-million-users-ask-about-health-each-week/ (Accessed January 12, 2026)
• TechCrunch (January 12, 2026). “Anthropic announces Claude for Healthcare following OpenAI’s ChatGPT Health reveal.” https://techcrunch.com/2026/01/12/anthropic-announces-claude-for-healthcare-following-openais-chatgpt-health-reveal/ (Accessed January 12, 2026)
• TechCrunch (January 11, 2026). “Google removes AI Overviews for certain medical queries.” https://techcrunch.com/2026/01/11/google-removes-ai-overviews-for-certain-medical-queries/ (Accessed January 12, 2026)
• The Guardian (January 2, 2026). “Google AI Overviews risk harm with misleading health information.” https://www.theguardian.com/technology/2026/jan/02/google-ai-overviews-risk-harm-misleading-health-information (Accessed January 12, 2026)
• Anthropic (January 12, 2026). “Introducing Claude for Healthcare and Life Sciences.” https://www.anthropic.com/news/healthcare-life-sciences (Accessed January 12, 2026)