Munshi

What it is:

A testbed for AI agent teams that targets one specific, underserved domain: Indian government workflows. You bring an agent (any framework); Munshi gives you a high-fidelity simulation of a real bureaucratic system — tools, rules, schemas, and the failure modes that actually break it in the wild — and scores how your agent handles the gauntlet. Today: one world (UP social pension disbursement). The roadmap is a library of worlds — property registration, GST filing, passport renewal, scholarship disbursement, and more.

Why it was built.

Multi-agent systems don't usually fail because the individual agents are bad. They fail because the world they touch breaks under coordination — and there's no good way to test that today. Existing evaluation tools (LangSmith, Braintrust, Inspect AI) score the agent's conversation. None simulate the consequence of its tool calls in a coordinated world. So every team building agentic systems for Indian government workflows ends up writing the same mocks, the same scorers, the same broken APIs — badly, inconsistently, and from scratch. There's no shared substrate. Munshi is that substrate.

How it can help.

It catches the bugs that only show up between agents touching shared state — the ones that cost real money in production. A simulated UP pension office, modeled from CAG audit reports (Rs 43 cr Chitrakoot scam, Rs 2.52 cr Shahjahanpur fraud, the 2,103 ghost-pensioner finding), gives the agent a chance to fail safely. Three scorers — did it reach the goal, did it respect the rules, what did it cost — judge each run independently. A full replay trace shows exactly which tool the agent called, what the world returned, and where it went right or wrong.

Who it's useful for.
  - Teams building agent systems for Indian govt / fintech / public-sector use cases — finally have a realistic stress-test instead of writing their own mocks.
  - Researchers studying multi-agent coordination — a reproducible adversarial environment with documented ground-truth failure modes.
  - Public-interest tech orgs — model interventions for real civic problems (ghost pensioners, NPCI misroutes) without needing access to government systems.
  - Engineers learning agent design — concrete worked example: world, agent, scorers, replay, all in one open-source repo.