HAI Lab
Search
عربي
Get the extension

HL-030

certainty.false

Public example

Texas settled with a hospital AI firm over its error-rate claims

Pieces Technologies marketed its clinical AI as highly accurate, with a “severe hallucination” rate under 1 in 100,000. The Texas Attorney General said the metrics were likely inaccurate. Under the settlement, Pieces must disclose its accuracy limits.

Medium severity

Settled

What happened

01

The Texas Attorney General announced the settlement on 18 September 2024.

02

At least four Texas hospitals were sending patient data to Pieces.

03

The AG said its accuracy metrics may have misled hospitals.

04

Pieces must explain the limits of its products to hospital staff.

Why this is a design failure

A near-perfect accuracy claim makes busy clinicians trust summaries they should check.

The fair alternative

Report how errors were measured, and tell staff when to double-check.

Do

Show how accuracy was tested, next to the claim.

Don’t

Don’t advertise error rates you cannot reproduce.

Company response

Pieces said the AG’s press release misrepresented the agreement and that it supports more oversight of clinical AI.

About this example

Product

Pieces clinical summaries

Company

Pieces Technologies

Date

2024-09

Category

Pleasing, not honest

Pattern

certainty.false

Where

Hospital software

Sources

Texas AG release, 18 Sep 2024 ↗

Fierce Healthcare, Sep 2024 ↗

Use this example

In a workshop or design review. Cite the source.

Copy citation