— answered about my own work first. If I'd ask it of you, it's only fair I answer it about myself.
Six layers, grown from shipping an AI judge. A golden dataset before trusting anything: 1,000+ tickets already rated by in-charges, compared dimension by dimension — nothing shipped until the system cleared 90%+ agreement with the human judgments. 500+ regression cases run before every release, with a staged rollout: a specific real-user group first, then the real world. Self-consistency as a tripwire: status and rating must correlate, so a broken correlation surfaces edge cases automatically, and errors alert the involved parties by email and Slack — failures announced themselves. Structural checks before quality checks: validate → ask → execute → notify caught malformed actions mechanically. Wrong outputs became permanent tests, never fixed-and-forgotten. And a live appeal loop as ongoing evaluation: challenged ratings turn into new parameters and rules — the users' pushback became the eval. The suite was mine as tech lead — if the person who owns the architecture doesn't own the definition of 'working', the two drift apart.
The retry storm. V1 let agents retry freely; one malformed input, one night, and retries multiplied into duplicated downstream business actions. Fixed within a week — Redis-backed idempotency keys on every action, plus a circuit breaker, because retries assume failure is transient and a malformed input is not. Changed permanently: every integration since starts with the failure path.
At Wiffy: both, and settled — they solve different problems. RAG answers what the data says; fine-tuning answers what a question even means in our application, so domain interpretation lived in the weights of LLaMA and Mistral — a call that also tipped on practical grounds: data privacy, cost at volume, and latency. In my own products it's a live debate I re-argue per product, and the tiebreaker is unit economics — any answer that only works at API prices isn't an answer at consumer scale.
If things go right: VisionAI on real homes and schools, Kept at wedding scale, GameMind earning its first real-time commentary. What breaks first at 10x, in order: the human review loop, then inference cost, then retrieval quality — which degrades quietly, and quiet degradation is the dangerous kind. What doesn't break, by design: the event-driven backbone — 10x should be a provisioning problem, not an architecture problem.
Days 1–30: understand before touching — map the system, read the incidents, ship one small real improvement to earn trust. Days 31–60: own one outcome end to end with a number attached — the thinnest slice that touches production and carries real risk; I want the pager, not just the pull request. Days 61–90: make it repeatable — regression set, documented decisions, onboarding. One production outcome by day sixty, the system that keeps it healthy by day ninety.
Three real fires, one playbook. The retry storm: stabilize first, blame never — the malformed input that caused it lives in the regression set forever, and the incident became a design principle: the failure path comes first. The 10-month mandate: pressure gets absorbed at the top, not passed down — the quality bar stayed up through design, code, and functional reviews with CI regression on every merge, and release velocity still rose 30%. Calm is a leadership deliverable. Growing six engineers into event-driven production under deadline: shrink the blast radius until mistakes are affordable — small services, replayable events, and at Classplus, validation in headless environments before every 50,000-viewer event. The through-line: build the system, human and technical, where mistakes are survivable and nobody is afraid to report breakage.
If you'd answer these six differently about your own systems — that's exactly the conversation I want to have.
START THAT CONVERSATION — BHAGAT99198@GMAIL.COM →