Skip to content
Rocky G
Go back

The Project Where I Banned AI

Table of contents

Open Table of contents

An uncomfortable decision

I spend most of my time building LLM systems. A WhatsApp AI receptionist for clinics. A multi-agent forecasting engine. So when I started CPA Desk, an advisory tool for real-estate channel partners in Bangalore, everyone assumed the interesting parts would be AI.

I banned it instead. Zero LLM calls anywhere near the numbers.

CPA Desk tells a partner which client to call today, which properties to pitch, and whether a client should hold and rent or sell and reinvest. Every one of those outputs moves someone’s money. And that changes the engineering question completely.

The question that decides it

Here is the test I now run on every feature: what does a wrong answer cost, and can I explain the answer afterwards?

A chatbot suggesting a restaurant: wrong answer costs a mediocre dinner, no explanation needed. LLM away.

A tool telling a family to sell a property and reinvest: a wrong answer costs lakhs, and “the model felt bullish” is not an explanation anyone can stand behind. When a client asks why, the advisor needs to walk them through the exact reasoning, line by line, and get the same answer twice.

LLMs are probabilistic by construction. Same question today, a slightly different answer tomorrow, with no guarantee the reasoning in between stays fixed. For creative work that variance is a feature. For financial reasoning it is disqualifying, and not because the model is dumb. It is disqualifying because the output cannot be audited or reproduced. You cannot walk a client through a chain of reasoning that will not be the same chain next week, and you cannot write a test for a function whose answer drifts. Money needs determinism twice over: once to explain the number, and once to trust the number will not move on its own.

what does a wrong answer cost? and must I explain it afterwards? cheap + no audit needed → use the LLM, add guardrails chat, drafts, summaries, intent expensive + must be explained → deterministic engine, sourced money, medicine, legal, ranking

What I built instead

The unglamorous answer: engines. Plain, deterministic, testable TypeScript, built as pure functions. Same inputs, same outputs, no hidden state, and no network call anywhere in the path that produces a number. That property is the architecture, not a detail of it, because a pure function is one you can audit by reading it and pin down with a test. The moment an LLM enters that path, both of those guarantees are gone.

A matching engine that scores properties for a client using nine weight matrices, one for each combination of intent and risk appetite, with a five-level relaxation cascade when nothing fits perfectly. A returns model where every input is written down: cash out of pocket as the denominator, amortised interest over full tenure, stamp duty at possession, a post-possession slowdown factor, capital-gains tax at the documented rates. The model’s output was verified against a fully worked real case before anything shipped, and that check is only possible because the engine is deterministic. A fixed set of inputs produces exactly one output, so the whole calculation can be pinned in a test that fails loudly the day someone changes a rate by accident. Determinism is not only about trust for the client. It is what makes the thing testable at all.

And one page I insisted on: a public methodology page. Every assumption, every rate, every threshold, with its source. When the tool says “sell and reinvest crosses over in year 4,” the advisor can click through to exactly why.

Some rules we enforced that sound almost rude in 2026:

That last one matters most. The temptation with LLMs is to let them paper over gaps in your data. The gap is information. Hiding it is lying with extra steps.

Where the judgment actually lives

People read “banned AI” as anti-AI. It is the opposite. It is the same discipline that makes my AI systems trustworthy.

Caira, my clinic receptionist, answers only from verified facts and escalates the rest to humans. Kshetra tags evidence as verified or directional and ships its own dissent. CPA Desk takes the logical next step: where answers must be reproducible and auditable, the right amount of LLM is zero.

One system, three positions on the same dial. The dial is not “how much AI can I add.” The dial is “how much unexplainable variance can this decision tolerate.” Chat tolerates a lot. Triage tolerates some, behind guardrails. Money tolerates none.

The most senior engineering skill in AI right now is not prompting, not fine-tuning, not agent frameworks. It is knowing where to set that dial, and having the spine to set it to zero when the domain demands it, even when AI is the reason clients came to you in the first place.

Every tool wants to be a hammer this year. The judgment is in knowing which problems are not nails.


Share this post on:

Previous Post
Building Caira: a WhatsApp AI Receptionist a Hospital Can Trust
Next Post
Seven AIs Walk Into an Argument