The Next Interface: Your Field of View
Forget keyboards. Forget screens. The next computing interface is ambient, voice-first, and always-on.
Enter VisionClaw — a provocative new open-source project that combines three emerging technologies into something that feels like science fiction made real:
- Meta Ray-Ban Smart Glasses — stylish wearable cameras and speakers
- Google Gemini Live — real-time multimodal AI with native audio
- OpenClaw — the local AI agent that can actually do things
The result? An AI assistant that sees what you see, hears what you say, and takes actions on your behalf — all without pulling out your phone.
What It Actually Does
Put on the glasses, tap a button, and talk:
"What am I looking at?"
Gemini streams your glasses camera at ~1 frame per second, analyzes the scene, and describes it: "You're looking at a café called 'Blue Bottle' with outdoor seating. There's a chalkboard showing today's specials."
"Add milk to my shopping list"
The request delegates to your local OpenClaw instance, which adds the item via your connected task app — Todoist, Notion, or Apple Reminders.
"Send John a message that I'll be 10 minutes late"
OpenClaw routes through your configured messaging channels — WhatsApp, Telegram, iMessage, or Signal — and sends the text.
"What's the best coffee shop within 3 blocks?"
Gemini triggers a web search via OpenClaw, processes the results, and speaks back: "Based on reviews, Blue Bottle has 4.8 stars and is 200 meters ahead on your right."
All of this happens hands-free, through your glasses, while you continue walking down the street.
How the Pieces Fit Together
The architecture is elegant in its simplicity:
Your Eyes (via Ray-Ban Glasses)
↓
~1fps video + microphone
↓
iOS App Bridge
↓
Gemini Live API (WebSocket)
↓
├─→ Audio responses (spoken to you)
└─→ Tool calls ──→ OpenClaw Gateway
↓
56+ available skills:
• Web search
• Messaging (WhatsApp, Telegram, iMessage, Signal)
• Smart home control
• Notes & reminders
• Calendar management
• And more...
Key technical decisions:
Gemini Live over STT: Unlike traditional voice assistants that convert speech-to-text before processing, Gemini Live uses native audio streaming over WebSocket. This means lower latency, better prosody understanding, and more natural conversation flow.
The 1fps Compromise: Video streams at just one frame per second — enough for scene understanding and object recognition, but bandwidth-efficient enough for mobile. For static scenes (reading a menu, examining a device), that's sufficient. For dynamic scenes (sports, traffic), it struggles — but that's the tradeoff for always-on operation.
OpenClaw as the Action Layer: Gemini provides the reasoning and conversation. OpenClaw provides the agency — the ability to actually send messages, control devices, search the web, and modify your digital life. Without OpenClaw, Gemini would be a chatty oracle. With it, the system becomes a capable assistant.
iPhone Mode: Testing Without $300 Glasses
Not ready to buy Ray-Bans? VisionClaw includes an iPhone mode that tests the full pipeline using your phone's back camera instead of glasses. Same code, same APIs, same experience — just holding a phone instead of wearing it.
This democratizes development. Anyone with an iPhone and a Gemini API key can prototype ambient AI experiences without investing in smart glasses.
The Vision: Computing Disappears
VisionClaw points toward a future where computing becomes invisible:
Current state: You pull out your phone, unlock it, find an app, navigate to a function, type or tap, wait for results, then put your phone away. The device demands your attention.
VisionClaw state: You're walking down the street. You see an interesting restaurant. "What's the rating?" you ask. The AI sees the restaurant, searches for reviews, and tells you: "4.2 stars on Google. Recent reviews mention excellent pasta but slow service." You never broke stride.
This is ambient computing — technology that waits in the background, available when needed, invisible when not.
The OpenClaw Advantage
What's particularly clever about VisionClaw is its use of OpenClaw as the action layer. This gives the system several advantages over closed alternatives:
Local-First: Your OpenClaw gateway runs on your hardware — Mac Mini, VPS, or homelab. Your conversation data and action history stay on your infrastructure, not in Google's or Amazon's cloud.
Extensible: OpenClaw's 56+ skills can be extended. Want to control your specific smart home setup? Write a skill. Need to query your company's internal API? Write a skill. The system grows with your needs.
Cross-Platform: Because OpenClaw handles the action layer, VisionClaw can send iMessages (requires Mac), WhatsApp messages (requires phone or WhatsApp Web), Telegram, Signal, Slack — whatever you've configured in your OpenClaw setup.
No Vendor Lock-in: The entire stack is open source. If Google changes Gemini's API terms, swap to Anthropic's Claude. If Meta discontinues the glasses, the code adapts to whatever wearable comes next.
The Challenges
VisionClaw is impressive as a prototype, but significant hurdles remain:
Battery Life: Ray-Ban Meta glasses get 3-4 hours of active use. Always-on AI streaming reduces this further. The "ambient" promise requires battery technology that doesn't exist yet.
Social Acceptance: Talking to your glasses in public remains socially awkward. The "glasshole" stigma from Google Glass hasn't fully faded. Until this behavior normalizes, adoption will be limited to early adopters.
Privacy Paranoia: A camera that's always on, always streaming to AI servers, raises obvious privacy concerns — both for the wearer and those around them. Expect regulatory scrutiny and public pushback.
The 1fps Limit: One frame per second is sufficient for static scenes but useless for dynamic environments. A true "seeing" assistant needs 10-30fps, which requires orders of magnitude more bandwidth and compute.
Latency: Even with Gemini Live, there's a 1-2 second delay between asking and answering. For casual queries, fine. For time-sensitive tasks ("Is that car stopping?"), too slow.
The Broader Implications
VisionClaw is a harbinger, not a product. It demonstrates where computing is heading:
From Pull to Push: Today's computing is pull-based — you go to information. Tomorrow's will be push-based — information comes to you when contextually relevant.
Multimodal as Default: Text-only AI was 2023. Voice + text was 2024. Vision + voice + action is 2025-2026. The most useful AI agents will process and generate across all modalities.
The Interface Disappears: We went from command lines to GUIs to touchscreens to voice. The endpoint is no interface at all — just intention and action, mediated by AI.
Agents Need Hands: Pure conversational AI is limited. Useful agents need tools — the ability to send messages, make purchases, schedule appointments, control devices. OpenClaw provides these hands.
Competition and Context
VisionClaw isn't alone in this space:
- Meta's native AI: Ray-Ban glasses already have Meta AI built-in, but it's limited to Meta's ecosystem and capabilities.
- Humane AI Pin: A wearable screen-less device with similar ambient AI goals, but poorly received due to latency and reliability issues.
- Rabbit R1: Another dedicated AI device, struggling with the same "why not just use my phone?" question.
VisionClaw's advantage is leveraging existing infrastructure: Glasses you might already own, a phone you definitely own, and an OpenClaw setup you can customize. It's not a new device — it's a new layer on devices you have.
The Open Question
Will we actually want this?
There's a plausible future where ambient AI assistants become as essential as smartphones are today — always there, always helpful, quietly handling logistics while you live your life.
There's also a plausible future where the social awkwardness, privacy concerns, and technical limitations relegate this to niche hobbyist use — cool demos that never achieve mainstream adoption.
VisionClaw lets us test that question. For the price of a Gemini API key and some open-source code, you can live in 2027 today. Whether you'll want to stay there — that's the experiment.
Getting Started
Want to try it?
- Get the code:
git clone https://github.com/sseanliu/VisionClaw.git - Get a Gemini API key: Free at Google AI Studio
- Configure OpenClaw: Follow the OpenClaw setup guide
- Start with iPhone mode: Test the experience before buying glasses
The future is open source, wearable, and surprisingly close.
VisionClaw was created by Sean Liu and is available under the MIT License. It is not affiliated with Meta, Google, or the OpenClaw project — it's a community experiment at the intersection of wearables, multimodal AI, and agentic systems.