AI therapy chatbots reduce symptoms in trials. The safety scaffolding is thin

AI chatbots designed to deliver mental health support do reduce symptoms in randomised trials, by small to moderate amounts, but the safety machinery meant to catch the moments that matter most is underbuilt [s1][s3]. That is the two-part answer, and the two parts have to be held together: the efficacy is real enough to take seriously, and the risks are real enough that "download a therapist" is the wrong way to think about it.

What the trials show

The largest synthesis to date pooled 48 randomised controlled trials of conversational agents — both AI-driven and older rule-based systems — covering 28,071 participants [s1]. It found small-to-moderate but statistically significant reductions in depression (standardised mean difference -0.27), anxiety (SMD -0.20) and stress (SMD -0.26) [s1]. Effects were larger in clinical populations and in shorter-duration interventions, risk of bias was largely low, and publication bias was judged negligible [s1]. A negative SMD here means the chatbot group improved more than the control group.

An earlier meta-analysis focused specifically on AI-based agents drew a similar map with larger numbers. Across 15 randomised trials, it reported significant reductions in the symptoms of depression (Hedges' g = 0.64, 95% CI 0.17 to 1.12) and distress (g = 0.70, 95% CI 0.18 to 1.22) [s2]. But overall psychological well-being did not significantly improve (g = 0.32, 95% CI -0.13 to 0.78) — the confidence interval crosses zero [s2]. The effects were more pronounced for agents that were multimodal, generative, integrated into messaging apps, and aimed at clinical or elderly populations [s2]. Both syntheses land in the same place: chatbots move symptom scores, they do not yet demonstrably improve broader well-being, and long-term durability is untested [s1][s2].

The safety problem is structural, not incidental

The concern with generative chatbots is not that they are useless but that, unlike a scripted rule-based bot, they generate open-ended text that can be inaccurate or unsafe [s3]. A 2026 scoping review examined how safety is actually implemented in purpose-built mental health chatbots, across 21 studies from 11 countries [s3]. Its findings describe a field that has bolted on technical fixes faster than it has built clinical safeguards.

Most interventions included at least one technical safety mechanism, most commonly fine-tuning and prompt engineering, and a smaller subset layered retrieval systems, content filters or risk classifiers on top [s3]. But the human-facing protections were weak. Human oversight during delivery was limited; crisis referral protocols "varied in rigour but were mostly underdeveloped"; and systematic monitoring for adverse events was sparse [s3]. The documented safety failures the review catalogued are the ones that matter most: missed suicidal ideation, and the provision of inaccurate clinical information [s3]. The authors' conclusion is that these tools need a genuine sociotechnical approach — technical safeguards plus clinician co-design, procedural controls and human oversight — rather than a model and a disclaimer [s3].

That distinction between a purpose-built therapy chatbot and a general-purpose assistant is not academic. When a consumer AI tool was stress-tested on triage rather than therapy, it undertriaged half the emergencies in one structured test, with crisis safeguards activating unpredictably — the same failure mode the safety review describes, in a different clinical task.

What it means for a reader

The evidence supports a narrow, hedged claim: for depression, anxiety and stress, a chatbot intervention can produce a small-to-moderate short-term improvement over no support [s1][s2]. It does not support treating a chatbot as a substitute for a clinician, particularly in crisis, where the safeguards are precisely the part reviewers found least developed [s3]. The strongest evidence comes from purpose-built, studied programmes — not from repurposing a general chatbot as a confidant, a use for which no comparable trial evidence exists and where the documented harms cluster.

There is a narrower role where the signal is cleaner: a separate line of work has tested AI chatbots as a support tool for alcohol use, the kind of bounded, specific task these systems handle best. The pattern across all of it is consistent — the more specific and supervised the job, the better the tool looks; the more it is asked to stand in for a human in an open-ended, high-stakes conversation, the thinner the ground.

Sources

• [s1] Effectiveness of AI and rule-based conversational agents for depression, anxiety and stress: a systematic review and meta-analysis — npj Digital Medicine (2026)
• [s2] Systematic review and meta-analysis of AI-based conversational agents for promoting mental health and well-being — npj Digital Medicine (2023)
• [s3] Safety Mechanisms and Risk Mitigation in Generative AI Mental Health Chatbots: A Systematic Scoping Review — Healthcare (2026)