EdexConvoAI — The Adaptive Voice Tutor for Real Learning
Link to open source: https://github.com/akanksha-dot-dev/agent-quickstart-nextjs.git
EdexConvoAI is a real-time, voice-native AI tutoring platform that adapts to every student through Socratic dialogue, Bloom's Taxonomy progression, cross-session memory, and four distinct learning modes — all powered by Agora Conversational AI Engine.
3️⃣ Problem Being Solved
Traditional EdTech is broken in a fundamental way: it delivers information but doesn't build understanding. Students watch videos passively, read static text, or answer MCQs with instant hints — none of which trains the deep retrieval and explanation skills required in exams, vivas, or real jobs.
The core problem:
-
Passive learning doesn't stick. Students need to explain, not just recognise.
-
One-size-fits-all content ignores where each student actually is cognitively.
-
No continuity — each session starts from scratch, wasting time.
-
Viva and oral exam preparation has no scalable solution.
-
Study groups have no AI facilitator that can probe, challenge, and adapt.
EdexConvoAI solves this with a voice-first, adaptive, memory-aware AI tutor named Lexi that meets every student exactly where they are.
4️⃣ Target User
|
User |
Scenario |
|
University students |
Viva preparation, exam revision, concept exploration |
|
School students |
Homework help, subject mastery across 7 core subjects |
|
Self-learners |
Structured learning without a human tutor |
|
Teachers / Mentors |
Assign Lexi as a 24/7 study partner; receive session reports |
|
Interview candidates |
Technical interview practice (CS, ML, algorithms) |
|
Language learners |
Spoken practice with adaptive feedback |
Primary market size: 1.5 billion students globally; 50M+ preparing for oral/viva examinations annually.
5️⃣ How Agora Conversational AI Is Used
EdexConvoAI is built natively on Agora's Conversational AI Engine — Agora is not just transport; it is the intelligence layer:
|
Agora Component |
How it's used |
|
Agora RTC (agora-rtc-react) |
Real-time voice channel between student and Lexi |
|
Agora RTM (agora-rtm) |
Bidirectional messaging for transcript, agent state, metrics, and error events |
|
Agora Conversational AI Engine |
Managed STT → LLM → TTS pipeline; handles turn detection, barge-in, and silence detection |
|
agora-agent-client-toolkit |
AgoraVoiceAI for agent lifecycle; transcript helpers; agent state events |
|
agora-agent-uikit |
AgentVisualizer at the core of the LexiAvatarPanel; MicButtonWithVisualizer for mic UX |
|
RtcTokenBuilder.buildTokenWithRtm |
Server-side dual-token generation for RTC + RTM access |
|
VAD (Voice Activity Detection) |
Configured via turnDetection.config.start_of_speech and end_of_speech — enables natural barge-in |
|
Managed Agent Sessions |
/api/invite-agent uses Agora's agent SDK to start a session with a 2,000+ word adaptive system prompt |
|
Agent Stop |
/api/stop-conversation gracefully ends sessions; 400 ErrConflict treated as silent success |
The system prompt (lib/edex-prompt.ts) is built dynamically from the learner's profile and chosen mode — injected at agent invite time via the Agora Agents API.
6️⃣ Demonstrated Conversational AI Capabilities
|
Capability |
Implementation |
|
✅ Natural real-time voice conversation |
Sub-second response via Agora RTC + managed LLM pipeline |
|
✅ Student barge-in / interruption |
VAD-based turn detection — student can interrupt Lexi mid-sentence |
|
✅ Session memory |
Mistakes tracked and referenced: "Earlier you said X — do you still agree?" |
|
✅ Cross-session memory |
previousSessionSummary and previousSessionMistakes persisted in localStorage and injected into next session's system prompt |
|
✅ Adaptive difficulty |
Bloom's Taxonomy L1→L6 progression within a session; level changes based on answer quality |
|
✅ Dynamic questioning |
No fixed script — Lexi generates each question from context, past answers, and detected gaps |
|
✅ 4 distinct learning modes |
Study Partner, Viva Prep, Quiz Mode, Revision Sprint — each with radically different conversation contracts |
|
✅ Structured external outcome |
SessionReportCard with grade (A–F), strengths, weak areas, 3 revision recommendations — exportable |
|
✅ Frustration detection |
Prompt detects signs ("I don't know", long silence, repeated failure) and de-escalates |
|
✅ Analogy personalisation |
Student picks analogy domain (sports, music, cooking, etc.); Lexi uses it throughout |
|
✅ Knowledge graph visualisation |
Live KnowledgeGalaxyPanel maps topics as constellation nodes |
|
✅ Achievement system |
Real-time achievement toasts (streak milestones, Bloom level-ups) |
|
✅ Honesty handling |
Prompt instructs Lexi to state uncertainty rather than confabulate |
|
✅ Human control |
Session report exportable for teacher review; teacher can restart with a different mode |
7️⃣ External Action by the Agent
"At least one external action or structured outcome" — ✅ Multiple:
-
Session Report Card — After every session, Lexi's conversation produces a structured JSON-grade report (strengths, weak areas, recommended revision topics, performance grade A–F). This report is exportable via clipboard and persisted to localStorage for the teacher.
-
Cross-session memory write — The session summary is serialised into the learner profile (previousSessionSummary, previousSessionMistakes) and injected into the next session's system prompt via the Agora Agents API — creating a genuine memory loop without a backend.
-
Dynamic system prompt construction — /api/invite-agent builds a 2,000+ character adaptive prompt from live learner state (Bloom level, mastery scores, past mistakes, chosen mode) and calls the Agora Agents API to start an agent session with that context.
8️⃣ List of External APIs, LLMs, Speech Providers & Services
|
Service |
Role |
|
Agora Conversational AI Engine |
Core AI voice pipeline (STT + LLM + TTS, managed) |
|
Agora RTC |
Real-time audio transport |
|
Agora RTM |
Real-time messaging for transcript and agent events |
|
Next.js 16 (App Router) |
Full-stack framework; API routes; SSR |
|
React 19 |
UI rendering |
|
agora-agent-client-toolkit |
Transcript helpers, AgoraVoiceAI lifecycle |
|
agora-agent-uikit |
AgentVisualizer, MicButtonWithVisualizer |
|
agora-agents (server SDK) |
Managed agent session management |
|
Tailwind CSS |
Utility-first styling |
|
Lucide React |
Icon system |
|
pnpm |
Package management |
|
TypeScript |
Type safety across all layers |
|
localStorage |
Client-side learner profile and cross-session memory persistence |
No third-party LLM API key required. The LLM, STT, and TTS are all managed by Agora's AI Engine.
9️⃣ Known Technical Limitations
-
No persistent backend — Learner profiles and session memory are stored in localStorage only. Data is lost if the browser cache is cleared or a different device is used.
-
Mastery scoring is heuristic — Topic mastery is estimated from conversation patterns (keyword detection), not from a formal assessment engine.
-
Single student per session — The current architecture supports one-to-one tutor/student. Multi-student collaborative sessions (study groups) are not yet implemented.
-
Offline unavailable — Requires an active Agora network connection; no offline mode.
-
Language — Currently English only; multilingual mode is architecturally planned but not wired.
-
Session length — Very long sessions (>45 min) may hit Agora's managed agent timeout; the app handles this gracefully via the ErrConflict 400 pattern.
-
Next.js 16.2.6 stale version warning — Cosmetic; does not affect functionality.
🔟 AI Limitations & Safety Considerations
|
Concern |
Mitigation |
|
Hallucination |
Prompt instructs Lexi: "If unsure, say so plainly. Never invent information." |
|
Age-appropriate content |
Prompt enforces: "Keep all content educationally appropriate. Politely redirect off-topic requests." |
|
Over-reliance on AI |
Viva mode explicitly tells students they are practising, not being assessed for real marks |
|
Data privacy |
Zero PII leaves the device — no backend, no user accounts, no cloud storage of transcripts |
|
Bias in feedback |
Session report is based on mastery scores, not subjective AI opinion; grade is purely computational |
|
Teacher/parent control |
Session report is exportable; teacher can review full performance and restart with different mode |
|
No medical/legal advice |
Prompt scope-locks Lexi to the selected educational subject |
1️⃣1️⃣ System Architecture
┌─────────────────────────────────────────────────────────────────────┐
│ BROWSER (Next.js Client) │
│ │
│ ┌──────────────────┐ ┌──────────────────────────────────────┐ │
│ │ EdexOnboardingCard│ │ ConversationComponent │ │
│ │ (4 learning modes,│ │ ┌─────────────┐ ┌──────────────┐ │ │
│ │ subject, Bloom, │ │ │LexiAvatarPanel│ │TranscriptPanel│ │ │
│ │ analogy domain) │ │ │(orbital rings,│ │(live turns, │ │ │
│ └─────────┬─────────┘ │ │ state glow) │ │ agent state) │ │ │
│ │ │ └──────┬───────┘ └──────────────┘ │ │
│ ▼ │ │ │ │
│ ┌──────────────────┐ │ ┌──────▼──────────────────────────┐ │ │
│ │ LearnerProfile │ │ │ AgoraVoiceAI (toolkit) │ │ │
│ │ localStorage │ │ │ RTM: TRANSCRIPT_UPDATED │ │ │
│ │ - Bloom level │ │ │ AGENT_STATE_CHANGED │ │ │
│ │ - masteryScores │ │ │ AGENT_METRICS │ │ │
│ │ - mistakes │ │ └──────┬──────────────────────────┘ │ │
│ │ - past summary │ │ │ │ │
│ └──────────────────┘ └─────────┼────────────────────────────┘ │
│ │ Agora RTC + RTM │
└────────────────────────────────────┼────────────────────────────────┘
│
┌────────────────────────────────────┼────────────────────────────────┐
│ NEXT.JS API ROUTES (Server) │
│ │ │
│ /api/generate-agora-token ←──────┤ RtcTokenBuilder.buildTokenWithRtm │
│ /api/invite-agent ←──────┤ Agora Agents SDK (managed session)│
│ └─ buildLexiPrompt() │ 2000+ char adaptive prompt │
│ /api/stop-conversation ←──────┤ Agora Agents API │
└────────────────────────────────────┼────────────────────────────────┘
│
┌────────────────────────────────────┼────────────────────────────────┐
│ AGORA CONVERSATIONAL AI ENGINE │
│ │ │
│ [STT] → Whisper / Agora ASR ────┤ │
│ [LLM] → Managed LLM (prompt) ────┤ VAD: start_of_speech / │
│ [TTS] → Agora managed TTS ────┘ end_of_speech │
└─────────────────────────────────────────────────────────────────────┘
1️⃣2️⃣ Future Evolution
|
Phase |
Timeline |
Feature |
|
v1.1 |
1 month |
Multi-student study rooms — Agora channel with 2-4 students + Lexi as AI facilitator |
|
v1.2 |
2 months |
Persistent backend — Supabase/Firebase for cross-device profile sync |
|
v1.3 |
3 months |
Teacher dashboard — Class-wide view of Bloom levels, weak topics, session durations |
|
v2.0 |
6 months |
Multilingual — Hindi, Spanish, Mandarin via Agora's language model switching |
|
v2.1 |
6 months |
Formal assessment mode — AI-proctored oral exam with timestamped evidence log |
|
v2.2 |
8 months |
Curriculum alignment — Map questions to IB/GCSE/AP/JEE syllabus topics |
|
v3.0 |
12 months |
Institutional SaaS — School/university licensing with teacher control panels |



