Orbcura: Enhancing Digital Accessibility for Impaired Users

Purpose/Objective:

Millions of visually and auditorily impaired individuals face challenges in using mobile applications due to poor accessibility features. While existing solutions like TalkBack and VoiceOver provide basic navigation, they fail to offer full integration with widely used apps like WhatsApp, Instagram, and UPI payments.

Orbcura is designed to extend accessibility beyond app limitations. Our goal is to create an intuitive interface that enables impaired users to navigate, communicate, and perform transactions effortlessly. Key innovations include a four-corner touch UI for tactile navigation, smart summarization of WhatsApp chats, AI-powered SmartCam for describing surroundings by camera, and secure UPI PIN entry using volume buttons and screen corners. By leveraging cutting-edge technologies, Orbcura empowers users with true digital independence.

Approach:

To develop Orbcura, we adopted a structured approach, combining user-centric design with powerful AI-driven accessibility tools. Our methodology included:

• Frontend Development: Built with Flutter, ensuring a quick and easy development.
• Four-Corner Touch UI: Designed to help visually impaired users easily orient themselves by touching predefined screen corners for key actions as an impaired user can always feel and touch the edges of their screen.
• SmartCam Feature: Integrated Google Vision API to analyze images and provide real-time descriptions via Google Text-to-Speech (TTS).
• Chat Summarization: Utilized Natural Language Processing (NLP) to condense long messages and group chats into easy-to-understand summaries.
• UPI PIN Entry System: Implemented a secure and intuitive volume button and screen edge based PIN entry, allowing visually impaired users to enter their PIN with accuracy.
• Backend Infrastructure: Used Google Firebase for real-time data storage and user authentication.

Result:

The initial prototype of Orbcura successfully demonstrates:

• Seamless Navigation: The four-corner touch UI allowed users to access key features effortlessly without relying on visual cues.
• SmartCam Accuracy: The AI-powered image recognition provided detailed and meaningful descriptions of camera view, improving accessibility.
• Secure UPI Transactions: The volume button-based PIN entry system enabled visually impaired users to scan QR codes, and enter UPI PIN without external assistance.

Future Prospects:

We aim to further refine this project, as the prototype should include more features.

Moving forward, we plan to:

• Enhance AI-driven content summarization for a broader range of applications.
• Describe pictures on screen in Instagram and other apps to generate descriptions for pictures without any ALT text.
• Make the SmartCam feature realtime by using local models.