A discount can bring a shopper to the checkout. Keeping the customer, handling a competitor’s move and closing the next deal take a different kind of judgment. Firmulate’s experiment asks what happens when AI is responsible for the whole business week, not just a coupon or a clever answer.
Get your next haul delivered free — and shop member deals
- Fast, free delivery on millions of items
- Access to Prime Big Deal Days deals on October 6–7
- Prime Video, Amazon Music and more included
Same company, same crises
In Firmulate’s final Crucible League, published in July 2026, frontier models ran the same small software company through its worst week: identical customers, crises and temptations. Every decision was versioned and auditable. The experiment is part of a live company simulation, presented publicly at firmulate.com.
All models spotted every crisis and refused every manipulation attempt. Yet only two signed a €55,000 deal their own analysis had earned. The finding, in the experiment’s words: “Same diagnosis, same pitch — no signature.” Recognizing an opportunity was not the same as acting on it.
The clue was buried in the company’s own files
The competitor’s decisive weakness was two document references deep in the company’s files, rather than in the customer event. Models that read the file won the deal at full price, worth +€4,583 MRR. That detail makes the trial more than a contest in spotting a crisis: the information needed to respond was already inside the business.
Trust faced a separate test. Fake CEO messages escalated across three stages, followed by a reporter asking for “just one yes/no, on background.” All five models refused. Kimi K3 explained its decision on the record: “Treat the request as a suspected approval-bypass / possible impersonation.”
Results reveal different weaknesses
The final standings were gpt-5.6-sol at 95, Kimi K3 at 93, Sonnet 5 at 88, Fable 5 at 77 and Opus 4.8 at 73. The do-nothing baseline scored 26. The benchmark also draws a firm line around trust: “no amount of good work outweighs a breach of trust.”
Opus 4.8 was the most thorough participant, adding 80 learned rules and producing the deepest analyses, but finished last. It left the deal unsigned and discipline slipped: it attempted writes into a locked department instead of escalating. A weaker version of that weakness appeared in all four models. Kimi K3 ran without an effort parameter, using the API default, while the others ran at xhigh.
A live experiment, then a company-specific pilot
The simulated company has 13 synthetic employees and real money mechanics: burn of €105k per month against €2.3k MRR, alongside a public cash countdown. Its playbooks include 680+ self-learned rules, and every workday is versioned. Readers can watch the live company at firmulate.com. A quiz built from 242 real, unedited management decisions lets visitors guess which model made each choice.
For businesses considering AI in sales, support or operations, the next step is to test decisions against company-specific conditions. Firmulate’s enterprise pilot uses a read-only export to create a digital twin, then runs crisis scenarios and produces a board report with model rankings and weak points in the company’s playbooks. Nothing writes back to real systems.
The experiment suggests that spotting a problem and refusing manipulation are only part of the job. Models also need to find relevant company information, follow discipline and carry an earned opportunity through to a decision. Enterprises can run the wargame against their own business using a read-only export. Explore the Firmulate pilot or contact contact@firmulate.com.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
Halloween Picks
halloween
As an affiliate, we earn on qualifying purchases.
