Real-time object detection system using Meta Ray-Ban AI Glasses and YOLOv8 Nano. The glasses stream live video to Instagram, which is then captured and processed on a Mac M1 to identify objects in the user's field of view. Originally built to assist with Arduino-powered robotics projects and later expanded as a general visual AI research tool.
Technical Details
How to Run on Ray-Ban Meta
- 1Open the Meta View app on your phone and connect to your Ray-Ban Meta glasses.
- 2Navigate to the display settings and enable the developer web browser.
- 3Enter the web app URL: https://github.com/ghsaboias/glasses-ai
- 4The app will load directly — no authentication required.
- 5Use gestures or voice commands as described in the app's documentation.
Comments (3)
Sign in to leave a commentLatency on detection was around 200ms in my tests which is impressive given the stream goes Instagram → Mac. The M1 Neural Engine really helps here.
Would be amazing to see this extended with SAM2 for segmentation. The groundwork here is solid. Anyone trying to do CV research with these glasses should start here.
Clever use of the Instagram live stream as a video source — it's a bit hacky but it genuinely works. YOLOv8 Nano is fast enough on M1 to keep up with the 1fps feed from the glasses.