Nov 21, 2026

Aarogya: Voice-First Health Companion That Speaks Your Language

fastapi langgraph agora murf nemoguardrails

Aarogya (आरोग्य) is a voice-first, multilingual health companion for Indian families, built for PS11: Breaking Language Barriers Through Agentic Voice AI.

 

THE PROBLEM

Millions of Indians receive lab reports and prescriptions they can't read, in a language they don't speak at home. Health apps are English-first, and chatbots stop at "please consult a doctor". The patient is left alone to understand the report, find a doctor, remember medicines and follow up.

 

WHAT AAROGYA DOES

Speak in Hindi, English or Hinglish, even switching mid-conversation, and Aarogya replies in your language, by voice and on screen. It closes the loop from report to consultation:

• Understands your reports: remembers a year of lab history (4 reports, 33 tests, 8 body systems), explains values in plain language, compares trends and cites the source PDF for every number

• Health tab: a body-system overview showing what needs attention, with detail and trends for each system

• Upload a new report: AI extracts the values, and you review and confirm them before anything is saved

• Books the doctor: finds a specialist and offers slots, and books only after your explicit "yes". If the slot is taken, nothing is booked and it offers alternatives

• Medicines: daily and weekly reminders and refill orders. It flags unclear or stopped medicines instead of guessing

• Shares records with a doctor for a stated purpose, revocable anytime, and prepares a doctor brief

• Home dashboard, care timeline, and notification preferences with quiet hours

 

HOW IT'S BUILT

• Agora Conversational AI: speech recognition, turn-taking and barge-in over Agora RTC on web and Android. Agora calls our custom LLM endpoint

• Murf Falcon TTS: a natural bilingual female voice. Numbers, dates and times are spoken in Hindi when the reply is in Hindi

• The reply language is detected on every turn, even when speech recognition writes English words in Devanagari

• A LangGraph agent on Gemini Flash-Lite (with model fallback) uses 18 typed tools. Results stream live to the app as generative UI cards

• FastAPI, Postgres (health values with their source), Qdrant (semantic search), Redis (live events)

• Expo / React Native: one codebase for the web app and the Android app

 

SAFETY THE MODEL CAN'T BYPASS

• NeMo Guardrails input rail (runs locally, about 30 ms) blocks diagnosis, dose changes, jailbreaks and requests for other patients' data

• Emergency rules in Hindi, Hinglish and English trigger a fixed 112 escalation

• Consent gate in code: every action needs an explicit "yes" in a later turn. The approval is tied to the exact proposal, expires, and works only once

 

The demo uses a synthetic patient. Clinic booking and pharmacy are simulated sandbox services.

 

Live: https://reverence-haziness-apprehend.ngrok-free.dev/

This build was uploaded as a hackathon project

Hackathon

Build with Agora - BLR

View All Projects
Give a star to encourage!Discussion
Start a new conversation!
Login to join the discussion
Updates
  • Added a medicine cabinet (reminders, refills, guards for stopped and unclear medicines) and safety tests: 12/12 end-to-end, 9/9 red-team.
    Saturday, Nov 21st, 2026
  • Added a Health tab (8 body systems, 4 lab reports, 33 tests), home dashboard, care timeline and notification preferences.
    Saturday, Nov 21st, 2026
  • Prototype complete: voice agent on web and Android (Agora Conversational AI + Murf Falcon TTS) with lab-report memory and doctor booking behind a consent gate.
    Saturday, Nov 21st, 2026