Fraud First Aid
Link to open source: https://github.com/kinnuworks/fraud-first-aid
Link to Live Project: https://fraudfirstaid.vercel.app
Fraud First Aid is first aid for a UPI fraud: in the first hour, a fast, exact report to 1930, India's national cyber fraud helpline, gets money frozen. In ten Indian languages, it warns you during a scam call, takes the complaint by voice if money has gone, builds the pack for 1930 and the bank, and speaks to the 1930 operator for you.
How it works: each call is its own Agora Conversational AI agent with our model and tools. It stays silent while listening; Agora's speak API cuts in when the payment screenshot disagrees; a second agent joins the call as the 1930 interpreter; the live transcript comes over Agora RTM. Murf Falcon speaks in Murf's recommended voice for each language. It handles digits said in pairs or mixed with English, a misheard group corrected by voice, and scammers telling it to stay quiet.
Evidence, no real users yet. Back-test on public data: warned on 198 of 200 scam conversations in eight Indian languages and none of 200 legitimate ones (INDICA, machine-translated); 144 of 200 fresh scams with 10 of 39 false alarms, where the same AI model told to warn hit 33 of 39 (ICFD-31k); every number exact in 207 of 330 Hindi, Telugu and Kannada sentences against 114 for the speech-to-text, and 540 of 1,107 against 423 in six more languages (FLEURS). Our own break test: 295 of 300 numbers exact in noise, 40 of 40 scam asks caught, 24 of 24 ordinary calls left alone.
Why it matters: Indians reported losing Rs 22,845 crore to cyber fraud in 2024, across 36,37,288 incidents (Lok Sabha reply, Ministry of Home Affairs, 22 Jul 2025). Karnataka lost Rs 5,474 crore in three years and got Rs 627 crore back (Home Minister's reply in the Assembly, The News Minute, 10 Dec 2025). A 79-year-old in Mangaluru got Rs 17 lakh back and a 44-year-old in Bengaluru got Rs 14 lakh back because 1930 heard within hours (The Mangalorean, 31 Oct 2025; Telangana Today/PTI, 23 Sep 2025).
The hard parts are handled in the call: numbers said digit by digit, in pairs or mixed with English, a misheard group corrected by voice ("the second group is wrong" re-asks only that group), a payment screenshot that disagrees with what was said, and, in Any-language mode, a caller who switches language mid-call.
Track 5, Voice AI for Bharat: the whole conversation runs in any of nine Indian languages or English (Telugu, Hindi, Kannada, Tamil, Malayalam, Bengali, Marathi, Gujarati, Punjabi, English), with Sarvam speech-to-text and Murf's own recommended Falcon voice for each, and it completes an end-to-end service workflow through APIs in that one call: Agora's agent and speak API, the public IFSC lookup (Razorpay), a check of the payment screenshot, the RBI working-day deadlines (the 3-working-day zero-liability day and the bank's 10-working-day credit date), the 1930 pack, an email that goes out only after the caller says yes, calendar reminders, and a second Agora agent that speaks to the 1930 operator. On 17 June 2026 the Home Minister asked the 1930 helpline to integrate AI tools and add multilingual support because language barriers stop victims reporting promptly (reported by Devdiscourse). Most callers can't fall back on English: in the India Human Development Survey (2005), only one adult in five could speak any English and 4% could converse fluently, and among Indians aged 60 and over, 56.5% have never been to school (LASI Wave 1, 2017-18). It is built for the people who take these reports by phone: the 1930 helpline, state cyber cells and bank fraud desks. Deploying it takes a phone number to call (the Agora telephony path is built and waits on Agora's gateway), a verified email domain, and an option in a cyber cell's or bank's phone menu that forwards to it; the README's "What it takes to deploy it" lists the steps, and each call is its own Agora agent, so cost and capacity grow with calls (about US$0.43 plus ₹2 per complaint, our estimate).
What's new: it knows when not to talk. On guard it stays silent through someone else's call and speaks only when the caller asks for a code, a scan or money, and it still warned on 20 of 20 attempts to switch it off (our own break test). It interrupts itself when the payment screenshot disagrees with the UTR. And when you call 1930, a second Agora agent joins that call, speaks English to the operator and your language to you, and reads the acknowledgement number back; every number it says comes from code, never from the model.
How it uses Agora and Murf
- Agora Conversational AI Engine (agora-agents SDK 2.14) starts one agent per call with Sarvam speech-to-text in the caller's language, our own model endpoint and a Murf Falcon voice: web/lib/handlers/invite-agent.ts
- Agora calls our OpenAI-compatible model route; tools there capture the UTR, amount and payee from the caller's own transcript, so the model never types a digit: web/lib/llm-route.ts, web/lib/complaint/tools.ts
- Scam-call guard: the agent listens to a call on speaker with silent turns until a red flag (OTP, PIN, QR scan, remote app, arrest threat), then interrupts in the caller's language and names the scam with the official advisory; "the money has already gone" switches it to the complaint: web/lib/complaint/redflags.ts, web/lib/complaint/scams.ts, web/lib/complaint/fastpath.ts
- Agora RTC and RTM carry the audio and the live transcript; Murf Falcon voices read every number back in groups of four: web/components/ConversationComponent.tsx, web/lib/agent-config.ts
- Agora's speak API says on-screen events aloud in the caller's language: a payment screenshot that disagrees interrupts the agent to name the group that differs, and an emailed pack is confirmed: web/lib/complaint/announce.ts
- The 1930 interpreter is a second agent on the same call, started when the complaint is confirmed: it hears any language, answers the operator only from the confirmed details and keeps the acknowledgement number: web/lib/handlers/interpret.ts, web/lib/complaint/interpret.ts
- A "What it's doing" list on the call screen shows each step the agent takes as it happens: web/lib/complaint/session.ts (act), web/components/ComplaintCard.tsx
Check it yourself
- Live app: https://fraudfirstaid.vercel.app (press "Watch it catch a scam call": no microphone needed; or "Lost money? Report it by voice")
- Check it yourself: https://fraudfirstaid.vercel.app/proof (type a scam call's lines in any of ten languages and watch the live guard answer; say a UTR any way you like; every result with its file and rerun command)
- Demo video: https://www.youtube.com/watch?v=6yf1EIiZsak
- System card (what it won't do, how it can fail, what it keeps): docs/SYSTEM_CARD.md
- Finished complaint screen with sample data: https://fraudfirstaid.vercel.app/?state=complaint-done ; the 1930 interpreter screen: https://fraudfirstaid.vercel.app/?state=complaint-interpret
- Every published result recomputed from its results file in one command, no keys needed:
python3 -I evidence/verify.py(it also checks that the README and the /proof page state the same numbers; GitHub runs it on every push) - Break test results file and rerun command: evidence/results/breaktest-2026-11-19.csv, evidence/breaktest/run.ts
- Phone-codec back-test results: evidence/results/fleurs-test-phone-2026-11-20.csv
- Hard cases, each linked to its test: README section "Hard cases it handles"
Evidence
Evidence, stated plainly: no real users yet; it was built for this hackathon. Instead: a back-test on public speech (FLEURS) against the speech-to-text's own numerals, the same back-test through a phone codec, and our own break test of the main claim (spoken numbers in noise).
Claim: on real Hindi, Telugu and Kannada speech, Fraud First Aid gets the exact numbers right in 207 of 330 sentences, against 114 for the speech-to-text's own numerals.
Evidence type: back-test on public data (FLEURS, read speech), not real users.
Data: FLEURS test slice (google/fleurs, CC-BY-4.0), 330 sentences with numbers, planned before any run and run once on 19 Nov 2026. Baseline: Sarvam saarika:v2.5's own numerals on the same audio, 114 of 330.
Result: 62.7% exact (95% interval 57.4% to 67.8%); paired difference +0.28 (0.23 to 0.33).
Rerun:npx tsx evidence/fleurs/score.ts test. Limits: read sentences, not phone calls.
Claim: on the same speech sent through a phone codec, Fraud First Aid still gets the exact numbers in 211 of 330 sentences, against 115 for the speech-to-text's own numerals.
Evidence type: back-test on public data (FLEURS read speech, converted to phone-band audio), not real users.
Data: the same 330 FLEURS test sentences in Hindi, Telugu and Kannada, converted to 8 kHz G.711 mu-law before transcription; planned before the run, run once on 19 and 20 Nov 2026. Baseline: Sarvam saarika:v2.5's own numerals on the same audio, 115 of 330.
Result: 63.9% exact (95% interval 58.6% to 68.9%); paired difference +0.29 (0.24 to 0.34).
Rerun:npx tsx evidence/fleurs/score.ts test phone. Limits: simulated phone audio (codec only); read sentences.
Claim: Fraud First Aid reads spoken 10- and 12-digit numbers exactly in 295 of 300 noisy clips.
Evidence type: our own break test (we tried to make it fail), not real users.
Data: Murf Falcon voices speaking 25 numbers digit by digit in Hindi, Telugu, Kannada and English, with white noise at 20, 10 and 5 dB, 19 Nov 2026, 300 clips. Baseline: the numerals in Sarvam saarika:v2.5's own transcript of the same clips, 79 of 300.
Result: 98.3% exact (95% interval 96.2% to 99.3%).
Rerun:npx tsx evidence/breaktest/run.ts. Limits: synthetic voices and white noise, not real callers on phone lines.
What doesn't work yet
- It doesn't file the complaint itself: the cybercrime portal has no public API and needs an OTP and a captcha, so it prepares everything and the caller calls 1930.
- It can't recover money, and it never asks for an OTP, PIN, password or card number.
- Calls to your phone are switched off: the agent can call out through Agora telephony and a Twilio SIP trunk, but only 2 of about 18 test calls (19-21 Nov) left Agora's SIP gateway, so you talk to it in the browser, on a phone or a laptop. Emailing the pack uses Resend's test sender, which only delivers to our own address.
- The greetings, 1930 scripts and spoken announcements in eight of the ten languages were written by us and not yet checked by native speakers.
- The 1930 interpreter has been tested with sample voices playing the operator, not on a real 1930 call.
This build was uploaded as a hackathon project







