Aug 29, 2026

Relay-ai

voiceai customersupport realtimeai #smartsupport #aiinnovation multilingualai #voiceai #humanescalation #customersupport #conversationalai #codeswitching #aiforgood

Relay AI β€” Knows When It Shouldn’t

Relay AI is a real-time multilingual voice support agent designed for situations where callers may be stressed, speaking in mixed languages, interrupted, or unable to clearly explain their issue.

Instead of simply trying to answer every question, Relay AI focuses on reliable resolution. It understands Hindi, English, and Hindi-English code-switched conversations, captures the essential information from the caller, confirms important details, and continuously evaluates the reliability of the conversation.

The core innovation is Confidence-Aware Human Handoff. When Relay AI detects uncertainty around an important detail or determines that human judgment is required, it does not blindly guess or provide potentially incorrect information. It can escalate the conversation to a human while preserving the context collected so far.

 

<iframe width="320" height="200" src="https://www.youtube.com/embed/G-0WXOm64gw"></iframe>

During escalation, Relay AI creates a structured case context containing:

  • πŸ“ Conversation summary
  • πŸ“Œ Confirmed information
  • ❓ Missing or uncertain information
  • 🌐 Detected language
  • 🚨 Priority
  • πŸ”„ Reason for escalation

This means the human agent receives the context instead of a cold transfer, reducing repetition for the caller and helping the human resolve the issue faster.

Key Features

πŸŽ™οΈ Real-Time Voice Interaction
Natural voice conversations powered by Agora's real-time communication infrastructure.

🌐 Multilingual & Code-Switched Support
Handles Hindi, English, and conversations where callers naturally switch between languages.

🧠 Conversation Intelligence
Extracts important information such as issue, location, name, contact details, intent, and priority from the conversation.

πŸ“Š Confidence-Aware Routing
Uses conversation confidence as a decision signal rather than treating every AI response as equally reliable.

πŸ™‹ Human Escalation
When AI confidence is insufficient or human judgment is needed, the conversation can be routed toward human assistance.

πŸ“‹ Context-Preserving Handoff
The human receives a structured case brief instead of forcing the caller to explain everything again.

πŸ”Š Interruption-Aware Conversation
Designed around real voice conversations where users may interrupt, correct themselves, or provide incomplete information.

Technology / Architecture

Caller β†’ Agora RTC β†’ Voice AI Agent β†’ Speech Recognition β†’ LLM/Conversation Intelligence β†’ Text-to-Speech β†’ Confidence & Routing β†’ AI Resolution / Human Escalation

Agora forms the real-time communication layer of the experience, enabling the voice interaction between the caller and the AI agent.

Why Relay AI?

Most voice AI systems focus on one question:

β€œCan the AI answer?”

Relay AI asks a more important question:

β€œShould the AI answer?”

The system is designed around a simple principle: AI should automate what it can handle reliably and involve a human when uncertainty matters.

Our goal is not maximum automation.
Our goal is reliable resolution.

Real-World Applications

Relay AI can be adapted for:

  • Customer support
  • Public information helplines
  • Service/request registration
  • Utility complaints
  • Local government assistance
  • General non-clinical support

The prototype demonstrates how multilingual voice AI can make support interactions more accessible while maintaining a clear boundary between AI automation and human judgment.

Relay AI β€” We don't just transfer the call. We transfer the context.

 

This build was uploaded as a hackathon project

Hackathon

EchoSphere

View All Projects
Give a star to encourage!Discussion
Start a new conversation!
Login to join the discussion

Created by

Teammates

More Builds by Sourabh

buildwithswytchcode