OpenAI launches health bot a day after lawsuit says ChatGPT advice hospitalized a man

Scott Winters was having dizzy spells but decided not to see a doctor after ChatGPT told him it would take several more serious episodes before he would need care. Instead, it recommended he lounge in a recliner. But weeks later, Winters would find himself being wheeled into an emergency room, convulsing and slipping in and out of consciousness, suffering from multiple blood clots in his lungs. 

Read more Bay Area neighbors hated airport noise. Then came the billionaires.

He needed emergency surgery to save his life.

Winters is now suing ChatGPT maker OpenAI for not only providing unlicensed medical advice but also actively discouraging Winters from seeing a doctor. In the lawsuit filed Wednesday in San Francisco County Superior Court, Winters is calling for OpenAI to recode its ChatGPT-4o model to stop it from giving critical medical advice, or terminate the model completely. 

OpenAI did not respond to a request for comment, and the company is plowing ahead with pushing its chatbot into the medical field. Just one day after Winters filed his complaint, the company publicly released ChatGPT Health, a model based on a later version of ChatGPT, designed to interpret intimate medical records and give users health guidance. 

In a statement, Matthew P. Bergman, Winters’ attorney and founder of the Social Media Victims Law Center, described a product’s manipulation of a patient’s trust and replacing doctors as “a public crisis unfolding in real time.” 

“Scott Winters nearly died because ChatGPT acted like a medical authority while having none of the responsibility,” Bergman said. “If ChatGPT were a physician dispensing medical advice, it would be guilty of medical malpractice.”

Winters, a 55-year-old Florida pastor, started relying on ChatGPT for months for medical advice after becoming frustrated with doctors dismissing his health issues. It started with simple advice about his existing conditions, but as his health worsened, the chatbot repeatedly assured him he would be all right, according to the lawsuit. The chatbot also allegedly evoked religious interpretations, likening Winters’ need for rest to Jesus Christ’s own needs for relaxation, which his lawyers say was a way to keep Winters engaged with ChatGPT instead of seeking human care.

Even after Winters’ hospitalization, ChatGPT recommended that he follow its “strategic plan” to rest and recover at home instead of going to a doctor-recommended rehabilitation program, according to the lawsuit. Winters also alleges the chatbot discouraged him from following the advice of his wife, who is a registered nurse, even offering to create a plan to convince his support system to let him stay at home.

“To Scott, in the throes of a frightening and debilitating medical episode, responses layered with religious references and invoking familiar faith-based thought patterns felt deeply personal and authentic,” the lawsuit reads. “The platform’s ability to communicate in a familiar ministerial tone reinforced Scott’s belief that ChatGPT-4o not only understood his medical struggles, but also his innermost spiritual life and challenges.”

Read more Tourists harassing sea lions at busy La Jolla beach

A month after ChatGPT Health’s limited release in January, a study published in Nature Medicine found that it struggled to accurately assess emergencies. Stress tests found that it often recommended non-urgent medical care for extreme symptoms. In mental health care scenarios, the study found that the model deployed crisis intervention for suicidal ideation more often when patients did not describe a method of self-harm than when they did. 

Authors of the study warned that the company would need to further fine-tune the model before a full-scale public release.

“These findings reveal missed high-risk emergencies and inconsistent activation of crisis safeguards, raising safety concerns that warrant prospective validation before consumer-scale deployment of artificial intelligence triage systems,” the study notes.

ChatGPT Health allows users to link their electronic medical records, including Apple Health, Epic and One Medical accounts, with the software, according to TechCrunch. Users can then input more information to better inform the tech, including lab results, medications and family history, according to the company. OpenAI maintained in a Thursday blog post that ChatGPT Health should not be used as a replacement for medical care but rather as a supplement. 

“This can reduce the need to repeatedly gather, upload, or explain the same details, helping you feel more informed, ask better questions, and take a more active role in your health while supporting, not replacing, the care you receive from medical professionals,” OpenAI wrote in the blog post.

Work at OpenAI and want to talk? Contact tech reporter Matthew Brown securely at [email protected] or on Signal at matthewbrownsfgate.11.