Aarogya: Voice-First Health Companion That Speaks Your Language
Link to open source: https://github.com/TaufeeqAi/arogya-ai
Link to Live Project: https://reverence-haziness-apprehend.ngrok-free.dev/
Aarogya (आरोग्य) is a voice-first, multilingual health companion for Indian families, built for PS11: Breaking Language Barriers Through Agentic Voice AI.
THE PROBLEM
Millions of Indians receive lab reports and prescriptions they can't read, in a language they don't speak at home. Health apps are English-first, and chatbots stop at "please consult a doctor". The patient is left alone to understand the report, find a doctor, remember medicines and follow up.
WHAT AAROGYA DOES
Speak in Hindi, English or Hinglish, even switching mid-conversation, and Aarogya replies in your language, by voice and on screen. It closes the loop from report to consultation:
• Understands your reports: remembers a year of lab history (4 reports, 33 tests, 8 body systems), explains values in plain language, compares trends and cites the source PDF for every number
• Health tab: a body-system overview showing what needs attention, with detail and trends for each system
• Upload a new report: AI extracts the values, and you review and confirm them before anything is saved
• Books the doctor: finds a specialist and offers slots, and books only after your explicit "yes". If the slot is taken, nothing is booked and it offers alternatives
• Medicines: daily and weekly reminders and refill orders. It flags unclear or stopped medicines instead of guessing
• Shares records with a doctor for a stated purpose, revocable anytime, and prepares a doctor brief
• Home dashboard, care timeline, and notification preferences with quiet hours
HOW IT'S BUILT
• Agora Conversational AI: speech recognition, turn-taking and barge-in over Agora RTC on web and Android. Agora calls our custom LLM endpoint
• Murf Falcon TTS: a natural bilingual female voice. Numbers, dates and times are spoken in Hindi when the reply is in Hindi
• The reply language is detected on every turn, even when speech recognition writes English words in Devanagari
• A LangGraph agent on Gemini Flash-Lite (with model fallback) uses 18 typed tools. Results stream live to the app as generative UI cards
• FastAPI, Postgres (health values with their source), Qdrant (semantic search), Redis (live events)
• Expo / React Native: one codebase for the web app and the Android app
SAFETY THE MODEL CAN'T BYPASS
• NeMo Guardrails input rail (runs locally, about 30 ms) blocks diagnosis, dose changes, jailbreaks and requests for other patients' data
• Emergency rules in Hindi, Hinglish and English trigger a fixed 112 escalation
• Consent gate in code: every action needs an explicit "yes" in a later turn. The approval is tied to the exact proposal, expires, and works only once
The demo uses a synthetic patient. Clinic booking and pharmacy are simulated sandbox services.
Live: https://reverence-haziness-apprehend.ngrok-free.dev/
This build was uploaded as a hackathon project









