Smart AI Dictation Device for Electronic Health Records
2nd Place, MIT Grand Medicine Hackathon, 2024
Problem
Nurses spend 4+ hours per shift on documentation, reducing patient care. Patients struggle to understand their medical notes.
Solution
WingNote—an AI-powered hardware solution that automates nurse documentation and provides clear, personalized patient summaries with a simple RFID tap.
Detailed Orthographic Drawings
Challenges
The original design changed with when the electrical components and its dimensions changed. The electrical engineer of the previous design was done by Venkadesh Eswaranandam. The biggest design challenge was creating a compact housing for the electronics, primarily dictated by battery size. I worked closely with electrical engineer Aditya Bangde to precisely fit his components—RFID, camera, and microphone—using calipers to match the components to the CAD. The design focused on being compact, lightweight, and sturdy, with an internal structure that secured components despite movement.
Collaborative Work
Teammates from Stanford Hackathon: Ben Cullen, Elsa Bis, Aditya Bangde
Teammates from previous MIT Hackathon: Eric Swidler, Jerome Andres, Tenzin Chophen, and Venkadesh Eswaranandam
The software, hardware, and CAD design for each hackathon were created separately, following the specific rules of each event, with no reuse of previous work.
Hardware
- RFID-based patient identification – Scans patient wristbands to log patient ID.
- Power efficient audio & video syncing – Captures speech & patient behavior for every interaction throughout the day.
- Xiao ESP32-S3 smart badge – Microphone, camera, and Wi-Fi enables wireless transmission of real-time data.
- Custom 3D-printed housing – Encases ESP32, RFID reader, and patient wristbands for hospital use.
AI-powered software
- Speech recognition (Deepgram API) – Converts nurse-patient dialogue into detailed, diatorized text.
- Vision analysis (Gemini 2.0 Flash) – Identifies helpful details about the patient and room to enhance notes.
- EHR structuring (Mistral AI) – Transforms raw conversation into Epic-compatible notes.
- Doctor’s review portal (Next.js/Vercel v0) – Physicians edit & approve AI-generated documentation.
- Patient summary app (Next.js/Vercel v0) – Provides multilingual, easy-to-read summaries of doctor's notes.
- Patient Q&A (Perplexity Sonar API & Postgres SQL) – Patients can receive reliable medication answers grounded in their doctor's notes and web data.
WingNote fully automates the documentation pipeline, improving workflow efficiency and patient care.