Where developers grow together

Vocal AI Studio

VocaAI Studio is an AI-powered real-time voice interaction and video voiceover platform. It allows users to communicate naturally with an AI assistant through their microphone while also providing a separate feature for adding AI-generated voiceovers to uploaded videos. The main objective of the project is to combine real-time conversational AI with AI-based voice generation in a single, user-friendly platform. The real-time conversation feature is built using Agora Conversational AI, which provides the real-time communication infrastructure between the user and the AI agent. When the user starts a conversation, their voice is captured through the microphone and transmitted through an Agora channel. Deepgram Speech-to-Text (STT) converts the user's speech into text, which is then processed by the OpenAI Large Language Model (LLM) to understand the user's request and generate an appropriate response. The generated response is then converted back into natural speech using MiniMax Text-to-Speech (TTS) and delivered to the user in real time. This creates a two-way conversational experience where the user can speak to the AI naturally and receive an immediate voice response. The application also provides a customized conversation dashboard where users can view the ongoing conversation and transcript, monitor the AI interaction, control their microphone, and end the conversation. The interface is designed specifically for the project rather than simply presenting the default Agora demonstration interface. This allows Agora to function as the underlying real-time conversational technology while the project provides its own customized user experience and functionality. In addition to real-time conversations, VocaAI Studio provides an AI video voiceover feature. Users can upload a video through the application and preview it before processing. They can provide a voiceover script and select an AI voice. The system uses Edge TTS to convert the provided script into an AI-generated voice. The generated audio is then combined with the uploaded video using FFmpeg, producing a new video containing the original visual content along with the generated AI voiceover. Overall, the project combines real-time communication, speech recognition, large language models, speech synthesis, and video processing into one platform. It can be used for applications such as AI assistants, educational content, video narration, tutorials, presentations, accessibility solutions, and AI-powered content creation. The main innovation of the project is the integration of a real-time conversational AI experience with an additional AI video voiceover workflow, making the platform useful beyond a conventional voice-assistant demonstration.

ARIA - Adaptive Real Time Instructional AI

🧠 ARIA: Voice AI Co-Teacher What is it?Have you ever been in an online class where the teacher is moving way too fast, half the students are lost, and nobody wants to unmute to ask a question? That’s what I wanted to fix.  ARIA is an AI co-teacher that literally joins your live video call as a participant. She doesn't replace the human teacher; she acts as a smart teaching assistant. She sits in the meeting, listens to the lesson, and pays attention to the chat. If she realizes a student is struggling, she waits for a natural pause in the conversation and actually speaks up to help them out. She can explain concepts, throw out quick quizzes, and even give the teacher a summary of learning gaps after the class ends. How I built it (and why I chose this stack) Building an AI that actually talks in a live meeting is tricky because of latency. If the AI is even a second too slow, it ends up talking over the teacher, which ruins the whole flow.  To make it feel natural, I had to piece together a very specific tech stack: * **The Video & Audio Engine (Agora):** I built the entire meeting room on the **Agora WebRTC SDK**. I knew I had to use Agora instead of standard WebSockets or open-source WebRTC wrappers for a few reasons. First, their global network keeps latency incredibly low, which is non-negotiable for human-AI conversations. But the real lifesaver was Agora's custom audio track API. When my AI generates a response, I can grab that raw audio buffer and inject it directly into the Agora call as a virtual microphone using `createCustomAudioTrack()`. To the students, ARIA just looks and sounds like another person in the Zoom call. * **The "Brain" (Groq + LLaMA):** I needed the AI to think fast. I hooked the backend up to Groq because their inference speeds are insane. It allows ARIA to read the classroom context and decide in milliseconds whether she should speak, take a silent note, or just wait.* **The State & Sync (Supabase):** I used Supabase Realtime to keep everything perfectly in sync. Instead of paying for expensive cloud transcription, I use the browser's native Web Speech API to transcribe what people are saying locally. Those transcripts—along with the chat and the teacher's controls—are instantly fired into Supabase so the AI always has real-time context of what's happening.* **The Frontend (Next.js + Tailwind):** I wrapped the whole thing in a Next.js 14 app. I spent a lot of time on the UI with Tailwind CSS, using frosted glass effects and fluid animations so it actually feels like a polished, modern product like Google Meet or Zoom.  It was a tough challenge getting all these real-time systems to play nice together, but seeing the AI actually jump into a voice call and help a student made it totally worth it!

ARGUS — Agora Powered Voice Web Automation

 ARGUS — Agora Powered Voice Web Automation ARGUS autonomously interacts with websites to execute tasks without human intervention, seamlessly handling actions like booking products, filling out forms, and summarizing site content  by commanding with the Voice. Focus on efficiency: Operating without manual intervention, ARGUS autonomously navigates web browsers to execute complex tasks like booking products, completing forms, and extracting website information. Focus on capability: ARGUS empowers autonomous web interaction by seamlessly executing steps—such as booking products, filling out forms, and navigating sites—entirely without human involvement.     Real-World Scenario Currently, manually executing routine web tasks and filling out forms is highly inefficient and time-consuming. To solve this, we are pitching a seamless voice-to-execution pipeline powered by Agora SDK + LLM + Backend Logic. This creates a direct workflow: Voice Capture → Intent Understanding → Autonomous Execution → Goal Complete. For example, when a user wants to purchase an e-commerce product, they simply speak their intent to ARGUS. The Agora API captures the audio, triggering ARGUS to autonomously navigate the site, click the necessary buttons, and populate input fields. When sensitive Human-In-The-Loop (HITL) intervention is required—such as entering an OTP, payment details, or confirming specific options—ARGUS generates a secure popup to collect the data. Once the transaction is finalized, ARGUS instantly notifies the user that the task is successfully completed. The Link for how it book the product:Product Booking Watch how ARGUS autonomously fills web forms: https://youtu.be/xOJCvNjIdJs (Note: Recorded exclusively to showcase form-filling capabilities using synthetic, mock PII data). The Next Stages Of The ARGUS  STAGE 2: Handling Complex Navigation Currently, ARGUS—Agora Powered Voice Web Automation—operates on a foundational LLM without fine-tuning or a Retrieval-Augmented Generation (RAG) architecture. To eliminate AI hallucinations and system overwhelm, this stage introduces a robust RAG framework. By actively observing contextual data, ARGUS will flawlessly manage complex web navigations and autonomous task executions. Furthermore, it will deliver real-time voice feedback to the user, verbally confirming exactly what action was completed at every single step.   STAGE 3: Advancing Payment Execution via x402, ACP, and AP2 Currently, delegating direct payment authority to an LLM poses significant security risks, requiring ARGUS to rely on Human-in-the-Loop popups for sensitive financial details. In our next evolution, we will fully automate secure transactions by integrating advanced agentic frameworks: x402, the Agentic Commerce Protocol (ACP), and the Agent Payments Protocol (AP2). By deploying cryptographically signed mandates to establish verifiable trust between transacting parties, ARGUS will safely and autonomously execute payments without compromising user security es.   Final Expected Outcomes We are bridging the gap between human thought and digital execution through voice commands with ARGUS—Agora Powered Voice Web Automation. By leveraging Agora's real-time voice capabilities, ARGUS seamlessly transforms spoken intent into autonomous action, driving the entire workflow from initial voice capture to independent task completion. Here are a few strong alternatives of similar length: Focus on autonomy: ARGUS seamlessly bridges human thought and computer execution using intuitive voice commands. Powered by Agora, the system autonomously translates spoken intent into direct action, effortlessly driving the entire process from initial voice capture to final task completion. Focus on the technology: By integrating ARGUS—Agora Powered Voice Web Automation, we bridge human intent with digital execution. Agora's technology captures the user's voice, empowering ARGUS to make autonomous decisions that seamlessly transition from understanding a spoken command to independently completing the goal The Technical architecture is present in the GitHub link