Website created by Mumbai teen aids illiterate individuals in navigating medical treatments: Explains its operation
In a groundbreaking development, a 17-year-old named Som Sengupta has created KnowUrMedicine, a web app designed to make medicine information accessible to India's less literate populations. The app, which operates entirely in the cloud, uses AI, Optical Character Recognition (OCR), and Text-to-Speech (TTS) to deliver medicine details in languages like Hindi and Marathi.
The OCR system, the app's backbone and Som's proudest achievement, accurately extracts dosage, side effects, and other details from diverse medicine packaging through image processing uploaded by users. For AI translation, a simplified approach based on ChatGPT's API was used, ensuring efficient, scalable, and fast translation of medical information into regional languages.
The TTS component automatically plays the translated text aloud, making the medicine information accessible to those who cannot or prefer not to read. This streamlined integration of OCR, AI translation, and TTS overcame accuracy, usability, and data privacy challenges, enabling vernacular-speaking and less literate populations in India to access critical medicine information effectively.
The frontend was optimized for mobile browsers with a clean interface allowing users to scan a QR code, select their language, and upload an image effortlessly. By keeping all processing in the cloud without storing user data, the app maintains user privacy and scalability.
During development, Som collaborated with over 20 healthcare professionals and 3-4 chemists for real-world feedback. The app cross-references extracted data with online sources to verify details, ensuring reliability.
Initially, the TTS module had issues with sounding robotic, particularly in Hindi and Marathi. However, Som used a modified version of Google Voice to iterate and achieve a conversational tone for the TTS module. The Optical Character Recognition (OCR) system was also refined through iterative testing and field tests with over 200 customers, improving its accuracy.
A study shows that 45% of patients misinterpret dosage or frequency without native-language instructions, posing health risks. KnowUrMedicine addresses this issue by providing medicine information in the user's native language, reducing the risk of misinterpretation.
Despite facing challenges in mastering FastAPI and AI integration during development, Som Sengupta has successfully developed a solution that could potentially revolutionize the way medicine information is accessed in India.
[1] Source: KnowUrMedicine Official Website and Interviews with the Developer.
