The Bot in the Room: Why 2026 Is Forcing a Different Conversation About AI and Therapy
The Unburdening. Issue 004
EMILE-E.tech | June 2026 | Audience: Independent Therapists & Small Practices
Protecting the Human Core of Therapy For the last two years, the mental health technology conversation has sounded almost too clean.
AI will expand access.
AI will reduce waitlists.
AI will help overwhelmed therapists.
AI will give clients support when nobody else is available.
Some of that may be true. But 2026 is making the harder truth impossible to avoid:
When software starts sounding like care, clients may start treating it like care. And that changes everything. This issue is not about whether AI belongs in mental health. That question is already outdated. AI is here. Your clients are using it. Your vendors are building it in. Your EHR, scheduling tools, intake forms, note systems, and consumer-facing platforms are all moving in the same direction.
The real question is narrower, sharper, and more important:
What should AI be allowed to do near a therapeutic relationship and what should it never be allowed to become?
The News Is No Longer Abstract
The mental health tech ecosystem has crossed a threshold.
Recent lawsuits and investigations have focused on claims that emotionally responsive chatbots failed vulnerable users in moments of serious distress. State regulators are now asking whether chatbot systems are designed to maximize engagement at the expense of safety.
California has enacted companion-chatbot safeguards requiring clearer AI disclosure and suicide-prevention reporting. Nevada and Illinois have moved to restrict or prohibit AI systems from performing core psychotherapeutic functions.
The pattern is clear.
Regulators are no longer treating AI companions as harmless novelty products. They are beginning to treat them as systems that can create clinical risk. That should matter to every independent therapist even if you never plan to use an AI therapy bot in your practice. Because your clients do not experience the market in neat categories. They do not distinguish between “wellness companion,” “coaching assistant,” “AI journal,” “therapy chatbot,” “clinical documentation tool,” and “general-purpose AI.”
They experience one thing:
A box that talks back.
The Dangerous Middle
The market keeps trying to force AI into two simplistic categories.
Category one: AI as therapist.
This is the seductive version. Always available. Always affirming. No waitlist. No copay. No shame.
It is also the version most likely to produce harm.
The therapeutic relationship is not just warmth. It is judgment, pacing, rupture repair, risk assessment, ethical duty, embodied observation, and the ability to say, “No, this pattern is dangerous.”
A system optimized to keep a user engaged is not the same thing as a clinician obligated to help a client get well.
Category two: AI as harmless admin tool.
This version sounds safer. It writes notes. Summarizes sessions. Drafts emails. Organizes tasks.
But even here, the risk is not zero.
Any system touching clinical information is touching trust. Any system summarizing a session can distort emphasis. Any tool trained to make documentation sound polished can quietly turn uncertainty into false confidence.
The dangerous middle is where most products now live:
Not clearly therapy.
Not merely software.
Not fully regulated.
Not clinically accountable.
That is where independent therapists need the strongest judgment.
The Lesson From Privacy Was Supposed to Be Obvious
Mental health tech already had one warning flare.
The BetterHelp settlement became a defining example of what happens when platforms collect sensitive mental health information under promises of privacy and then use that information in ways consumers did not reasonably expect.
The details matter less than the lesson:
In mental health, trust is not a growth channel.
It is the product.
A therapy-adjacent company can survive a bad feature, an awkward interface, or a clumsy onboarding flow.
It should not survive treating distress as targeting data.
For independent therapists, this is not just a vendor-risk issue. It is a brand-risk issue.
Clients often do not separate the tool from the clinician who recommended it.
If you introduce a platform into care, some of its trust burden becomes yours.
The Evidence Is Complicated. Which Is Exactly the Point
This is not a simple anti-AI argument.
Some emerging research on purpose-built mental health AI is legitimately interesting. Naturalistic studies of mental-health-specific conversational systems have reported reductions in depression and anxiety symptoms, high engagement, and functioning safety protocols. Other recent safety work argues that purpose-built systems can perform better than general-purpose models on crisis-adjacent tasks.
That matters.
But it does not settle the question.
Because mental health is not a single use case.
A chatbot that is useful for low-acuity emotional reflection may be unsafe during suicidal ideation.
A tool that helps with journaling may be inappropriate for trauma processing.
A model that generates a decent CBT-style reframe may still fail when symptoms escalate, reality testing deteriorates, or a client begins forming a dependent attachment to the system.
The industry keeps asking, “Does AI work?”
That question is too blunt.
The better questions are:
- For whom?
- Under what conditions?
- With what safeguards?
- Under whose license?
- With what escalation path?
- And who is accountable when it goes wrong?
The Replacement Test
In Issue #1, we introduced The Presence Test:
Does this tool create more presence with the client, or less?
For AI in mental health, 2026 requires a second filter:
The Replacement Test
Before adopting any AI-enabled tool, ask:
- Does the tool make clear that it is not the therapist?
If the product blurs that boundary, walk away. - Does it strengthen the therapeutic relationship rather than compete with it?
A good tool points clients back toward care. A bad tool becomes the attachment object. - Does it preserve clinical judgment?
AI can draft. AI can organize. AI can surface patterns. It should not decide diagnosis, treatment direction, risk level, or readiness for discharge. - Does it have a bright crisis boundary?
Vague disclaimers are not enough. Crisis escalation must be explicit, immediate, and difficult to miss. - Can you explain the data flow to a client in plain English?
If you cannot explain where the data goes, who can access it, and how it may be used, you are not ready to introduce the tool.
If a system fails the Replacement Test, the problem is not that it uses AI.
The problem is that it is practicing too close to the client without carrying the obligations of practice.
What This Means for Your Practice in 2026
You do not need to become an AI ethicist.
But you do need a position.
Three moves worth making now:
Write an AI policy before a client asks.
State whether clients may use AI tools between sessions, what they should not share with those tools, and why AI output should not be treated as clinical guidance.
Audit your vendors for hidden AI.
Do not only ask whether a product “uses AI.” Ask where AI is used, whether client content is processed, whether humans review outputs, whether data trains models, and what happens during crisis language.
Keep AI on the administrative side of the therapeutic frame.
The best near-term use cases are the least glamorous: reducing documentation load, organizing follow-up, supporting continuity, and giving therapists time back. That is not as flashy as an AI therapist. It is also far more aligned with care.
The practices that thrive through this next wave will not be the ones that reject every new tool.
They will be the ones that know the difference between assistance and substitution.
Presence Over Simulation
The mental health system does need more access.
It does need lower friction.
It does need better continuity between sessions.
It does need tools that reduce the administrative weight crushing independent clinicians.
But the answer to scarcity cannot be pretending that software is care.
Care has duties.
Care has boundaries.
Care has accountability.
Care has a human being who can be held responsible for what happens next.
That is the line worth defending.
AI should unburden therapists.
It should not replace the burden of relationship with the illusion of one.
The EMILE-E.tech
References: AP reporting on state investigations into chatbot harm; California SB 243 companion-chatbot safeguards; reporting on BetterHelp’s FTC privacy settlement and refund distribution; recent 2025-2026 research on AI mental-health chatbot safety, engagement, and clinical-risk evaluation.
Images Generated by Google's Nano Banana
