Sahaay AI
Link to open source: https://github.com/junjaanand/ShaayAi
Link to Live Project: https://shaay-ai-ubvz.vercel.app/
SahaayAI (सहाय-AI) is a real-time multilingual voice AI customer support and triage agent designed to make customer assistance faster, more natural, accessible, and context-aware. Built on Agora’s Conversational AI Engine, SahaayAI enables customers to communicate naturally in English, Hindi, and Hinglish without navigating traditional keypad-based IVR systems.
Traditional customer support systems often create frustration through rigid menus, language barriers, repetitive conversations, and disconnected escalation processes. SahaayAI addresses these challenges by providing a human-like conversational voice experience that understands users even when they switch between Hindi and English during the same conversation.
The system listens to the caller in real time, understands their intent, extracts important information through dynamic slot filling, and identifies details such as the caller’s name, contact information, issue category, issue description, location, and urgency level. This information is continuously structured during the conversation so that customers do not have to repeatedly explain their problems.
SahaayAI also incorporates safety and ethical guardrails. It avoids providing unsafe medical diagnoses, legal opinions, or financial advice and can identify emergency situations and guide users toward appropriate emergency services. This makes the system more reliable for real-world customer support environments.
One of SahaayAI’s key features is its Explainable AI (XAI) escalation system. When an issue requires human intervention, the agent does not simply transfer the caller. Instead, it creates a structured escalation package containing the conversation context, confirmed information, missing details, reason for escalation, issue category, and urgency. This allows human support representatives to understand the situation immediately without asking the customer to repeat everything.
The system also connects voice AI with enterprise workflows. Escalated issues can automatically create tickets in Linear and send real-time alerts to support teams through Slack. Alongside the caller-facing voice interface, SahaayAI provides an internal Support Agent Dashboard where supervisors can monitor conversations, transcripts, agent states, extracted information, and escalations.
The core architecture combines Agora RTC, Agora Conversational AI Engine, Agora RTM, Deepgram multilingual speech recognition, OpenAI GPT-4o-mini, and modern TTS technology to create a low-latency conversational experience. Agora handles real-time audio communication and AI agent orchestration, while RTM enables live transcript and agent-state synchronization with the dashboard.
The ultimate goal of SahaayAI is to bridge the gap between AI-powered automation and human customer support—delivering the speed and scalability of AI while preserving empathy, safety, transparency, and meaningful human intervention when it matters most.
This build was uploaded as a hackathon project
