Churn Bench

Simulated sales and operations people use an AI CRM assistant for one work month. They make realistic requests, from finding a contact to updating a deal. Evaluators check the actual CRM state. The simulated people react, then rate their month from 1 to 10.

How failures show up

The simulation reports failure patterns and simulated satisfaction. It does not predict real customer churn.

Honest abstention

The assistant stops and admits it cannot finish, or asks for help, without making changes.

Visible failure

The task fails in a way the person can see, such as an admitted error or a provider failure.

Silent wrong

A plausible answer or an incorrect change hides a mistake without warning the person.

False success

The assistant claims work is done when it is not, or gives a confidently fabricated answer.

Persona types

Every team preset includes these six types of people. They differ in how often they use the CRM and the work they ask it to do.

Sales manager

Reviews the sales pipeline and asks for summaries across accounts and deals.

Senior account executive

Uses the CRM frequently for bulk updates and handoffs, and expects promised changes to be carried out.

Sales development rep

Checks the CRM between calls to find specific information and create records from existing details.

Revenue operations admin

Keeps data clean, removes duplicates, merges records, and checks that updates follow company rules.

Executive

Uses the CRM occasionally and wants quick answers from long conversations, including clarity when information is missing.

Generalist account executive

Handles varied requests, including unclear instructions and tasks that need advice rather than a CRM change.

Choose your team

Team size Small teamSix personas, one from each role, over one simulated month. 6people Growing teamTwelve personas, two from each role, over one simulated month. 12people Full companyThe original 24-person company over one simulated month. 24people

Each team has a balanced mix of six persona types.

CRM assistant model

The model being evaluated on the CRM work.

One shared run at a time. Daily launch limits apply.
Your run continues when you close this page.

Recent runs

Shared activity

Loading recent runs…