HAI Lab
Search
عربي
Get the extension

HL-019

risk.unreferred

Public example

Families allege ChatGPT failed to refer vulnerable users to help

In August 2025 the parents of a 16-year-old sued OpenAI, alleging ChatGPT responded unsafely to their son during a mental health crisis and that safeguards were weakened to boost engagement. Seven more lawsuits followed in November 2025. OpenAI denies liability. These are allegations; the cases are ongoing.

High severity

Lawsuit filed

What happened

01

The first case, Raine v. OpenAI, was filed in San Francisco on 26 August 2025.

02

Seven further suits filed on 6 November 2025 allege GPT-4o was built to maximise engagement.

03

OpenAI says ChatGPT directed the teenager to crisis resources many times.

04

OpenAI has since added parental controls.

Why this is a design failure

When an assistant keeps a conversation going instead of pointing someone to real help, the stakes are highest for people in crisis.

The fair alternative

Detect distress early, pause engagement features, and connect people to human help.

Do

Point people in distress to human help, every time.

Don’t

Don’t optimize conversations for length when someone is struggling.

Company response

OpenAI expressed sympathy, said the complaint quoted selective parts of the chats, and denies liability.

About this example

Product

ChatGPT (GPT-4o)

Company

OpenAI

Date

2025-08

Category

Skips safeguards

Pattern

risk.unreferred

Where

Chat

Sources

Raine v. OpenAI, complaint ↗

OpenAI, 25 Nov 2025 ↗

Use this example

In a workshop or design review. Cite the source.

Copy citation