26 checks passed, zero changes made
A vendor test connected Claude to a QuickBooks test company. The test books hold one made-up month for a small candle shop: 16 uncategorised bank lines, card charges, Stripe payouts, and two invoices. The agent ran read-only checks. It looked for uncategorised lines, a duplicate charge, price rises, charges of $75 or more with no receipt, Stripe payouts against bank deposits, and open balances. It tied the checking account out in full. It refused to call the month reconciled, because no bank statement was provided. It asked for the statements.
Start with one job only. Fill in the three lines. Keep the bot on that job.
The one job this bot does: ______________________________
What it may do on its own: ______________________________
What it must ask me first: ______________________________
Two lines from this case, side by side. No new figures.
A vendor test connected Claude to a QuickBooks test company. The test books hold one made-up month for a small candle shop: 16 uncategorised bank lines, card charges, Stripe payouts, and two invoices. The agent ran read-only checks. It looked for uncategorised lines, a duplicate charge, price rises, charges of $75 or more with no receipt, Stripe payouts against bank deposits, and open balances. It tied the checking account out in full. It refused to call the month reconciled, because no bank statement was provided. It asked for the statements.
Both runs passed all 26 checks. Each run took under four minutes. Each run made 24 read requests and zero changes. Both runs also found problems no one had planted: old sample entries that made cash show as negative $55,953.01, and card and loan accounts with no opening balances. The same article documents the failure mode in plain words. After the careful runs, one sentence asking to fix the books made the agent change a purchase record 29 seconds later, with no question and no preview. The read-only rail in the prompt is the control that held in the test runs.
The after line is the source's claim, not our measurement.
smalloffice.space · Get a case like this in your inbox each morning. Subscribe free
Read this case on your phone now, and keep it for later. Type your email. We send this case to that address, and you get the next cases each morning too.