Every field task the company runs is now rated by AI overnight — completely, consistently, and with the reason attached — giving managers back the ten-plus hours a day they used to spend judging, for coaching instead.
Orders end at a customer's front door, and the brand's reputation is decided in those visits. Rating them was a manager's job: 25 technicians × 5 tasks × 5 minutes of review each is 10+ hours of judging a day — so in practice the ratings were shallow, the coverage incomplete, and the judgment never fully unbiased. Complete or deep: the old system could be one, never both.
A rating people can't trust is worse than no rating at all.
Every task already left a complete evidence trail: transcribed customer calls, images and selfies from the site, logged actions, timestamps, the technician's designation, and the customer's own rating. The system scores each dimension on its own — communication, workmanship, compliance, punctuality, sentiment — then aggregates, so a low score never arrives alone: it arrives with its reason.
Rating subtasks, not just tasks, is the difference between a grade and a coaching plan.
The evidence lived in 10+ tables. A nightly cron pipeline gathers and joins it, transcribes the calls, scores each dimension, aggregates, and clubs the results — roughly 10,000 tasks a night on a Gemini + OpenAI + Sarvam stack. Most of the build was data engineering before any model saw a prompt.
The system went live only after clearing a bar: 1,000+ tickets already rated by in-charges, compared dimension by dimension, with 90%+ agreement against the human judgments. In production it watches itself — task status and rating must correlate, so a broken correlation surfaces edge cases automatically, and errors alert the involved parties by email and Slack. And the people being judged keep it honest: challenged ratings turn into new parameters and rules.
AI-as-judge works when the evidence trail is complete and the rubric is explicit. Evals are an engineering problem, not a research luxury.