Pillion

Pillion — the AI that rides with you

India's delivery and bike-taxi riders run their whole job on a phone clipped to the handlebar: the next drop, the customer who keeps calling, "5 minute mein pahunch raha hoon", today's earnings. Every glance at that screen is a risk. And when a rider crashes alone at night, often nobody knows.

Pillion is a hands-free voice co-pilot for them, as a native Android app. Earphones in, phone in pocket: the rider just talks in Hindi, English or Hinglish, and Pillion answers, acts, and keeps them safe.

WHAT IT DOES
• Talks like a real co-pilot: live voice in Hindi, English and Hinglish, even switching mid-sentence. Interrupt it anytime and it stops instantly (barge-in).
• Acts, not just talks: next-drop ETA, nearest petrol pump, puncture shop or ATM, today's earnings vs yesterday, weather with heat and rain warnings, and an SMS or call to the customer, only after a spoken "yes" that my server double-checks with Jev (TypeSafe AI).
• Reads the order for you: share a screenshot of the delivery app's order screen, and on-device OCR (Hindi + English) fills in the customer's name, number and drop address. Masked numbers are detected, and the customer's number never leaves the phone.
• Detects crashes: accelerometer, gyroscope and GPS run on the phone. After a hard impact and a fall, Pillion asks "Aap theek ho?", sounds an alarm and counts down 20 seconds. No answer means an SMS with the rider's location goes to up to three emergency contacts, with updates every 2 minutes until the rider taps "I'm OK". Saying "SOS bhejo", or one tap on the SOS button, does the same.
• Works without internet: crash detection, the alarm and the SOS SMS run entirely on the phone, over the SIM. If GPS is weak, it falls back to network location.
• Live Guardian: the SOS SMS carries a link. Family opens it in any browser, no app needed, hears the rider live, holds a button to talk straight into the rider's earphones, and sees them on a live map. Pillion steps aside ("आपके घरवाले line पे हैं") and comes back when they leave.
• Glanceable on a moving bike: a glass globe that reacts to the voice, English subtitles under every Hindi line, a fatigue reminder after two hours of riding, a trip summary, earnings charts, a safety log, and automatic dark mode after sunset.

AGORA AT THE CORE, NOT BOLTED ON
• Conversational AI Engine: one agent per ride, with Sarvam speech-to-text and text-to-speech for Indian languages, Agora-managed gpt-4o-mini, turn detection tuned so road noise doesn't cut in, and filler words in the rider's language while tools run.
• Custom tools: eight tools (ETA, nearby, earnings, weather, SMS, call, confirm, SOS) call my backend, which reaches the phone through server-sent RTM messages and returns real data to the LLM.
• RTC with AI noise suppression and AI echo cancellation against traffic and wind; volume indication drives the globe.
• RTM for live transcripts, agent state and per-turn latency metrics.
• Speak API for safety prompts that cut through whatever the agent was saying.
• Web SDK with RTC-only tokens for the family page, plus an agent stop-and-restart handoff when a human joins the line.

BUILT FOR REAL RIDERS
Voice-first and hands-free, big 88 dp controls, TalkBack labels, 200% font support, reduced motion and bilingual alerts. The voice needs a network; safety never does.

I built it solo: native Kotlin with Jetpack Compose, a Node backend live on Render, 94 unit tests, and a signed APK anyone can install. Along the way I measured the latency at every stage and wrote down 21 pieces of honest Agora SDK feedback in the README.

Pillion is the friend on the pillion seat: it talks with you, works for you, and makes sure someone knows if you fall.