When AI symptom-checkers disagree with patients, patients tend to walk away
In 6,772 users of a US health system's AI triage tool, people engaged about twice as often when the AI matched what they already planned — raising a hard question about what the tools are steering.
AI "symptom checkers" now sit at the digital front door of many US health systems, greeting patients on public websites and inside patient portals and recommending where to seek care. A study published on 2 September in npj Health Systems looked at what actually happens when one of these tools tells a patient something different from what the patient already had in mind — and the answer is a pattern that should give designers pause [s1].
What the study measured
Researchers ran a prospective cohort analysis of 6,772 randomly selected users who completed AI self-triage on either the public website (unauthenticated) or the patient portal (authenticated) of a large integrated US health system between September 2023 and May 2024 [s1]. Before seeing a recommendation, users stated their planned site of care — their "pre-intent." Each encounter was then classified as validated, when the AI's recommendation matched that intent, or re-directed, when it differed; re-directed encounters were further split into escalated (pushed toward more intensive care) or de-escalated [s1]. The team tracked whether users then engaged with any on-screen call to action [s1].
Most users arrived already logged in: 6,264 (92.5%) were authenticated and 508 (7.5%) were not [s1].
What it found
Two numbers frame the tool's behaviour. Among unauthenticated users who had planned to self-care, 89% were escalated by the AI toward more formal care [s1]. Among authenticated users who had planned an office visit, 42% were re-directed to something other than what they intended [s1]. In other words, the tool frequently disagreed with the people using it.
The consequential finding is what happened next. Call-to-action interaction was roughly twice as high when the AI's recommendation matched the user's pre-intent than when it did not [s1]. And of 636 non-engagers who answered a follow-up survey — a low 9.4% response rate — all reported plans to seek care off-platform [s1]. The authors conclude that authentication status and the alignment between a user's intent and the AI's recommendation are strong correlates of whether people engage with digital care-seeking at all [s1].
Why that pattern is tricky
Read one way, this is a story about engagement: people click when the tool agrees with them. Read another way, it is a warning. The entire clinical value of a triage tool lies in the cases where it disagrees with the patient — telling someone who planned to tough it out that they should be seen, or someone heading to the emergency department that urgent care will do. Those are exactly the encounters where the study finds engagement drops off, and where non-engagers say they simply sought care elsewhere [s1]. A tool that is best obeyed when it merely confirms what a patient already believed is not adding much triage value.
That tension is not new to the AI era. A decade ago, a BMJ audit of 23 online symptom checkers found they gave the correct diagnosis first in only 34% of standardized cases and appropriate triage advice in 57%, and that their advice skewed risk-averse — nudging people toward care even when self-care was reasonable [s2]. The new study's finding that the AI escalated 89% of self-care-minded unauthenticated users echoes that older risk-averse tendency [s1][s2], now wrapped in a more capable interface.
What it means
The study is a single-health-system analysis, and it measures digital engagement, not health outcomes — it cannot say whether the re-directed patients who left the platform ended up better or worse off [s1]. Its value is in reframing the design goal. Measuring a symptom checker by how many people it "engages" rewards agreement, but the clinically useful moments are the disagreements. If patients disengage precisely when a tool challenges them, the metric and the mission are pulling in opposite directions.
For readers, the practical caution is modest and not medical advice: an AI triage prompt is a starting suggestion from a system with known, generally cautious biases, not a diagnosis. The more interesting implication is for the health systems deploying these tools — that "engagement" is the wrong yardstick for a technology whose job is sometimes to tell people something they did not want to hear.
Sources
- [s1] "Authentication status and AI triage concordance among care seekers in a US health system." npj Health Systems, 2 September 2026.
- [s2] Semigran HL, Linder JA, Gidengil C, Mehrotra A. "Evaluation of symptom checkers for self diagnosis and triage: audit study." BMJ. 2015;351:h3480.
Sources
- Authentication status and AI triage concordance among care seekers in a US health system — npj Health Systems , September 2, 2026
- Evaluation of symptom checkers for self diagnosis and triage: audit study — BMJ , July 8, 2015
Most AI models that predict who will skip their medicines aren't ready for the clinic
A review of 41 studies found that the great majority of AI medication-adherence prediction models carried high risk of bias, and that fancier algorithms did not reliably predict better.
A consumer AI health tool undertriaged half the emergencies in a stress test
Researchers ran 960 responses through ChatGPT Health using clinician-written vignettes. Failures clustered at both extremes, and crisis safeguards activated unpredictably.
A computer that grades each colonoscopy raised how often endoscopists found adenomas
A Danish stepped-wedge trial gave endoscopists automated feedback on their technique after every procedure. Adenoma detection rose from 43.4% to 48.6% — a different tool from real-time polyp AI.
Four Latin American countries digitised their health systems. Tech was the easy part.
Thirty-two interviews across Uruguay, Argentina, Chile and Mexico find implementation turned on governance, not software. A separate patent analysis shows where digital therapeutics are not being built.