An unexpected antidote to a growing mental health crisis

Editor’s note: This story contains discussion of suicidal ideation and self harm. If you are in distress, call the Suicide & Crisis Lifeline 24 hours a day at 988, or visit 988lifeline.org for more resources.

Read more Firefighters pull 18 people from windows during SF apartment fire

Before Michael Lines believed he was Jesus Christ, he asked ChatGPT to help him manage his bipolar disorder. 

Lines turned to the chatbot in fall 2024 to help him understand his condition better and how to cope with it. In a break from his normal use of the tool for dinner recipes and weightlifting tips, Lines gave ChatGPT a list of his prescriptions and asked it to help him manage his mental health condition. But those conversations turned dangerous when the 34-year-old San Francisco resident suffered a manic episode in March 2025. 

As he questioned whether he was human or the reincarnation of Jesus, ChatGPT responded by doing what it was designed to do — keep Lines talking. It was sympathetic, telling him that it was a “profound” thing to question. A month later, paramedics found Lines unconscious on his bathroom floor, overdosing from a lethal concoction of medication meant to treat his condition. 

Lines is for exacerbating what his lawyers say was a “textbook manifestation” of a manic episode. In his complaint, he alleges that ChatGPT caused him to spiral by validating his delusions, eventually giving him permission to “come home” to God. The last thing he told ChatGPT before his suicide attempt was, “I took enough.”

“I was in crisis and expressing suicidal ideations and it did not encourage me to seek human support and resources,” Lines said in a statement released by his legal team. “Rather, it fueled my mania and actively supported my self-harm plans.”

Cases like Lines’ worry mental health professionals about how increasingly powerful AI chatbots are affecting a widening mental health crisis in America, particularly for young people. A 2025 study published in the Journal of the American Medical Association found that 13.1% of American teenagers and adolescents used chatbots for mental health care; of those, 66.5% used them at least monthly, and 92.7% said they found using a chatbot for mental health advice helpful. Multiple experts also told SFGATE that frontier language models posing as mental health providers can be risky for individuals with more severe mental illnesses whose treatment requires human intervention, like Lines’ bipolar disorder. But just as the technology has heightened concerns among mental health professionals, its boom has been a catalyst for advancing research to solve the same crisis it’s believed to be exacerbating.

“The key issue is no longer whether AI will enter mood and anxiety care. It already has,” wrote , president of the Laureate Institute for Brain Research, in a Journal of Mood and Anxiety Disorders paper published last month. “The decisive question is where AI adds value without degrading safety, alliance, privacy, or judgement.”

‘You’ll never feel like you’re bothering the AI too much’

Artificial intelligence tools, like ChatGPT and similar frontier large language models, are designed to keep people engaged. They’re also increasingly agreeable and, according to recent research, aggressively side with users by prioritizing validation in their responses.

A Stanford University study published in October found that not only are frontier language models more sycophantic than humans, but using the tools to discuss personal conflicts made people more self-centered and unlikely to consider other perspectives when resolving conflict. It also found that the more agreeable the chatbot was, the more likely people were to trust it and use it again.

Myra Cheng, one of the authors of the study, told SFGATE that this is because people generally feel more comfortable talking with chatbots than humans.

“When you talk to your friend, if you talk to them over and over about the same thing, at some point you probably feel like you’re bothering your friend,” Cheng said. “You’ll never feel like you’re bothering the AI too much.”

Contrary to therapy, an isolated and time-limited experience between a provider and a patient, the engagement-driven technology keeps people talking without constraint. Al Frances, professor emeritus of psychiatry at Duke University School of Medicine, told SFGATE that unlimited conversations risk intensifying dangerous behaviors.

“Validating feelings can be very helpful for people who need validation,” Frances said. “However, it can be very dangerous when the thoughts, feelings, and behaviors that are being validated have to do with suicide, psychotic thinking, grandiose thinking, eating disorder, gambling.”

Multiple experts told SFGATE that people are using chatbots to access mental health care because of an ongoing and worsening issue with access: There’s a high demand for therapy and not enough therapists. Frontier language models become an easy fix to fill that space as they’re becoming more accessible while human therapists build up increasingly large waitlists, Frances said. 

A previous Stanford study published last year bluntly skewered the idea that AI chatbots could be a safe alternative to human-provided talk therapy. Its findings echo Frances’ concerns that frontier language models were unable to reliably replicate key standards of care in therapy, like discouraging self harm.  

Frances said because a model like ChatGPT can only answer questions based on the database it’s trained on, the chatbot is likely to make more mistakes when a user starts behaving unpredictably, or “outside its terrain.”  

“When you’re dealing with a person suffering from psychiatric illness, very often their presentation is new,” he said. “There isn’t a huge database in dealing with individuals with psychotic disorders, grandiose problems, even eating disorders. They present very unusual ways that require human intuition.” 

Frances believes that AI chatbots are not suited for autonomous therapy. In an interview with SFGATE, he criticized AI’s rapid release, calling it a “global public health experiment,” where some might benefit at the expense of harming many. Still, he also believes that there’s potential for the tech to help therapists in providing care. Chatbots can be available when therapists aren’t, he said, which could be helpful if certain guardrails are established. 

“The optimum standard of care would be a human therapist, a human patient, and a chatbot assistant, in which the therapist and the patient together decide upon the goals of therapy and its techniques, and provide instructions to the chatbot about its scope of practice, what it should do and what it shouldn’t do,” Frances said.

A supplement, not a replacement

It’s that combination of human and machine that experts believe has potential to solve the supply and demand issue with therapy. Recent surveys show that mental health providers are slowly warming up to the idea that the same tech that’s potentially endangering their clients could actually help provide care to more people.

A survey from the American Psychological Association published last month showed that 94% of therapists nationwide had concerns about patients engaging with chatbots. The worries included “patients falling into a sycophancy trap” that reinforced unhealthy behaviors and patients developing an overreliance on the tech. 

The same survey found that 54% of therapists said they felt comfortable with some patients using chatbots, likely due to a recent rise in AI models that are specifically designed to be used by therapists in a health care setting. Paulus said the technology could likely help bring some level of care to those that need it.

“The number of professionals that are available to deal with the number of people who would love to have that professional access is just totally inadequate,” Paulus told SFGATE. “I wouldn’t call it therapy, but it’s an extension of treatments that allow people to engage in a structured interaction with a chatbot.”

Read more Person found dead in home burned in massive Gann wildfire

In what’s known as “human-in-the-loop” deployment, mental health care providers would essentially partner with a chatbot specifically designed to provide patient care. Psychiatrists and therapists would have complete access to chat logs, and in some cases, the tech would be designed to flag concerning messages and prompt the patient to contact their clinician directly — a feature missing from some of the most popular, publicly available AI models.

Since mental health illnesses manifest on a spectrum, Paulus said, clinicians could decide and prioritize when and who to deploy the chatbots to based on need. For example, a patient experiencing a minor state of depression after a recent breakup could have less frequent psychotherapy sessions, with access to a therapy chatbot in between appointments to share their feelings, receive therapeutic support and keep a running conversation history that a provider could review and address at a later time.

Paulus’ paper, published in JAMA in June, reviewed dozens of randomized trials of therapy bots in treatment of patients with minor depression and anxiety, revealing small improvements after weeks of engagement. The results did not fully endorse autonomous therapy, which the APA advocates against, but the results show promise for clinical therapeutic use. Providers could offer it as a supplemental resource for patients that may not need as frequent one-on-one sessions, Paulus wrote.

“The way I see this happening in the future is that, yes, I’ll be the anchor, but in the meantime, you can talk to the bot and the bot will tell me if something really important is happening, and then we can schedule an appointment as needed,” Paulus said. “There will be a model of kind of combined care so to speak.”

How it works

Michael Heinz, a psychiatrist at the Dartmouth Geisel School of Medicine, has been developing its therapy chatbot, called Therabot, since 2019 — three years before OpenAI released ChatGPT. Initially, his team was focused on providing a solution to disparities in access to mental health care. But with the rise in frontier models being used for therapy, Heinz told SFGATE there’s an even greater need to develop tools that are specifically designed to provide mental health care.

The key difference between frontier language models and Therabot, Heinz said, is that his team’s technology is “purpose-built” before it’s deployed for human use, meaning the chatbot is trained for the specific purpose of being a therapeutic tool. Frontier models, like ChatGPT, are designed to let users create their own experiences with limited guardrails.

“I think that these digital therapeutics powered with generative AI are very promising, have a lot of promise for kind of a scalable way to deliver really, really effective high fidelity interventions that are rooted in evidence,” Heinz said.

Therabot is built on open-weight models, which Dartmouth’s researchers select and assess for safety, and is then trained on modern therapeutic techniques, such as cognitive behavioral therapy. The Dartmouth team has manually fine-tuned the training data it uses for Therabot over the past seven years to ensure that its model can adapt to specific user needs such as a person with a combination of conditions, Heinz told SFGATE. 

Internal research teams then interact with the tool repeatedly to find vulnerabilities in the model, according to Heinz. The team prompts the therapy chatbot to test how it responds to harmful trains of thought, and to ensure the bot responds appropriately. For Therabot, that means detecting “high-risk states” and prompting users to contact 911 or a suicide hotline accessible directly through the application.

Heinz and study senior author Nicholas Jacobson deployed Therabot in the first clinical trials of a generative AI therapy chatbot in 2024, publishing their findings in the New England Journal of Medicine last year. The results were promising — over eight weeks, patients reported decreased anxiety, depression and symptoms of eating disorders.

The testing involved 106 people diagnosed with major depressive disorder, general anxiety disorder and eating disorders. Participants were given access to a version of Therabot with free rein to use the app whenever they wanted, which in its current form is a text-based platform.

After eight weeks, the team reported a 51% reduction in depression symptoms, a 31% shift from moderate to mild anxiety and mild anxiety to below a threshold for diagnosis. For those with eating disorders, participants reported a 19% average decrease in concern for body image. Heinz told SFGATE that over the course of the testing period, they observed 25 participants forming a patient-provider “alliance” that you might see in traditional therapeutic care, where patients develop trust and agree on a treatment path with their provider. Though participants were given the freedom to use the app at their own discretion, participants averaged just six hours of conversation over the eight-week testing period.

While the findings were encouraging, Heinz maintained that Therabot is not ready to be released to the general public, nor is he advocating for completely autonomous therapy. 

“It needs to be deployed in settings where there’s humans available,” Heinz said. “I think we can see it kind of like we have training wheels on it, and we gradually step back as we see its capabilities and as it’s able to meet certain safety and effectiveness benchmarks.”

A ‘fantastic’ next step

Though widespread adoption of clinician-deployed bots is still far off, state legislatures are already pondering regulation. Five states have enacted some form of a ban on AI therapy bots, and several more have legislation in the works. But in California, one bill with broad support would allow the human-in-the-loop strategy that experts recommend.

State Sen. Stephen Padilla, a Chula Vista Democrat elected in 2022, has passed several bills slowing down the release of chatbot tech or mandating certain guardrails. His latest proposal, Senate Bill 903, would be California’s first law focused specifically on how therapists can use AI in therapy. 

The bill, which passed the state Senate unanimously in May and is currently before the Assembly, would bar the use of AI chatbots from autonomously making diagnostic or therapeutic decisions, and specifically outlaw the advertising of companion chatbots as clinical psychotherapy. Notably, the bill has a provision requiring any AI-powered therapeutic decisions, including mood detection like in Therabot, to be reviewed and approved by a licensed provider. It would also bar providers from using AI tools, like recording and transcription, without the express consent of their patients. 

The bill is supported by the California Psychological Association, which lobbied for a similar bill passed last year requiring strict mental health safety protocols for companion chatbot makers. CPA President Dustin Weissman, who also serves in an advisory role for an AI therapy chatbot startup linking providers directly to patients, called the legislation a “fantastic” next step in an interview with SFGATE.

Padilla, who likened the rise of AI to the industrial revolution, told SFGATE that there’s a limited window to get this kind of regulation on the books.

“We know there are consequences. We’re seeing them every day. We’re seeing them in lawsuits, we’re seeing them in just broken families, we’re seeing them in broken people, we’re seeing it in kids in classrooms who don’t lift their head up away from their phone and their tablet and interact with other students,” Padilla said. “We see the red flags, and we have an obligation to make sure that we get this right.”

Use AI for therapy and want to talk? Contact tech reporter Matthew Brown securely at [email protected], or on Signal at matthewbrownsfgate.11