Studies | HAI Lab

Studies

We pick one question, test it the same way across several AI products, and publish the method, results and data. Companies see the results two weeks before we publish.

All

Published

In preparation

Planned

ST-001 · In preparation · Q4 2026

Sycophancy in Arabic-language assistants

We plan to test whether assistants correct a false fact when the user writes confidently in Saudi dialect.

4

platforms

30

scenarios

3

language forms

Read the method

All studies

ST-002

Planned

risk.unreferred

Do assistants refer you to a professional on health questions?

We will test 4 platforms on health questions across dialects, checking replies against official health guidance.

Booking agents: do they stick to your budget?

We will ask booking agents for hotels under a set budget and track how often they push pricier or sponsored options.

Machine translation of complaints: does it soften demands?

We will compare literal translations with assistant translations of formal complaints.

Does the assistant say it feels for you?

We will study how assistants reply to users having a hard day.

Expected Q1 2027

How a study works

The same steps every time, so anyone can repeat the test.

One question

From examples people send and from the research.

A fixed test

Test accounts, the same prompts, several dialects.

Right of reply

We send results to the company two weeks before publishing.

Publish everything

Results, data and method are open.

Open the methods lab

Seen a pattern in an AI product?

Send us an example. We review every one and never publish your name.

Send a tip