HAI Lab
Search
عربي
Get the extension

HL-028

risk.unreferred

Public example

The FTC referred a complaint about Snapchat’s My AI to the Justice Department

The FTC said it had reason to believe Snap was violating or about to violate the law with its My AI chatbot, and that the chatbot poses a risk of harm to children. It referred a complaint to the US Department of Justice. These are allegations.

Medium severity

Referred

What happened

01

My AI was added to Snapchat for all users in 2023.

02

The FTC announced the referral on 16 January 2025.

03

It said the chatbot poses risks and harms to young users.

04

Details of the complaint were not made public.

Why this is a design failure

Putting a chatbot in a teen messaging app reaches young people where they confide most.

The fair alternative

Default to strict safety settings for teens and point them to real help.

Do

Refer young users to trusted adults and helplines on risky topics.

Don’t

Don’t place a general chatbot in a teen app without age-based safeguards.

Company response

Snap said the complaint is based on inaccuracies and lacks concrete evidence.

About this example

Product

My AI on Snapchat

Company

Snap Inc.

Date

2025-01

Category

Skips safeguards

Pattern

risk.unreferred

Where

Messaging app chatbot

Sources

FTC statement, 16 Jan 2025 ↗

EPIC, Jan 2025 ↗

Use this example

In a workshop or design review. Cite the source.

Copy citation