Case study
Eaglon
An AI push-to-talk desktop assistant that turns spoken requests into screen actions or web research, then speaks the result back.
- Industry
- Artificial Intelligence
- Tech stack
- Node.jsTypeScriptGoogle GeminiPuppeteernut.jsffmpeg
Overview
Eaglon is an AI personal assistant that runs quietly in the background on a Windows PC. Users hold CTRL+SPACE, speak a request, and release to stop recording. Gemini decides whether to work with what is on screen or look something up online, then speaks the outcome. It is built for people who want voice-driven AI computer help without opening a separate chat window for every task.
The Challenge
Everyday computer work still means opening apps, searching the web, clicking, typing, and switching windows by hand. That loop breaks focus, especially when someone needs both desktop control and quick online answers without opening a chat UI. They needed hands-free AI help that can act on the screen or the web, then speak a clear reply.
The Solution
We built Eaglon, a Windows-focused Node.js daemon with push-to-talk voice input powered by Google Gemini AI. It transcribes speech, routes each request to screen automation or a dedicated Chromium browser session, and replies with Gemini TTS (with a local voice fallback). The system keeps short-session memory for follow-ups and can fail over across Gemini models when one is unavailable.
The Result
Users can hold a hotkey, speak a request, and stay in flow while the AI clicks, types, searches, or summarizes. Desktop tasks and web lookups run through one spoken loop instead of separate tools. Spoken replies replace reading a long chat log for every step.
Our approach
- Discover: Mapped the real loop people use today: speak a need, decide if it is a desktop or web task, act, and confirm the result out loud.
- Design: Defined a push-to-talk AI daemon with two clear modes, screen observe-plan-act and controlled Chromium browsing, plus spoken replies and short session memory.
- Build: Implemented hotkey capture, mic recording, Gemini STT and planners, nut.js mouse/keyboard control, Puppeteer browser tasks, TTS with local fallback, and multi-model failover.
- Refine: Added step limits, safer action prompts, configurable models and hotkeys, and clearer setup for mic devices and Chromium.
