Five of six planted errors, caught
A Reddit poster built a fake month of bookkeeping for a business that does not exist: real-looking invoices, vendor payments, and a churn curve. He first wrote an answer key, then planted six errors: a duplicate invoice, a vendor rate that had drifted, a sign-off that never happened, a customer segment that looked fine but lost money, revenue leakage that only showed in the churn month, and a vendor payment that was never approved. He handed the books to his own AI agent setup, with no product or model named.
Start with one job only. Fill in the three lines. Keep the bot on that job.
The one job this bot does: ______________________________
What it may do on its own: ______________________________
What it must ask me first: ______________________________
Two lines from this case, side by side. No new figures.
A Reddit poster built a fake month of bookkeeping for a business that does not exist: real-looking invoices, vendor payments, and a churn curve. He first wrote an answer key, then planted six errors: a duplicate invoice, a vendor rate that had drifted, a sign-off that never happened, a customer segment that looked fine but lost money, revenue leakage that only showed in the churn month, and a vendor payment that was never approved. He handed the books to his own AI agent setup, with no product or model named.
The agents caught five of six errors unprompted (poster's report). The missed one was a sign-off rule the grading logic itself had not counted, and two agents flagged that gap on their own. The whole exercise cost about $5.50 and took under half an hour. Catch: this is a test on a fake business, not a real firm's books, so it shows error-catching form, not a live bookkeeping result.
The after line is the source's claim, not our measurement.
smalloffice.space · Get a case like this in your inbox each morning. Subscribe free
Read this case on your phone now, and keep it for later. Type your email. We send this case to that address, and you get the next cases each morning too.