Work/Earlier
JARVIS MK II
Gesture and voice, no keyboard
- 2026
- Concept
- Mac Studio
- Design & engineering
A no-type, no-click operating environment. Two screens: a dashboard holding a slowly turning globe, and THE STAGE — the machine’s own canvas, where it works while you are not looking. You control all of it with your hands in the air and your voice, and it answers every time you speak.
It starts with two claps
Black screen, faint white dots. A single circle sits in the centre with one thin ring around it. Clap twice and the circle ignites — concentric rings orbiting a core, the whole thing expanding outward, then collapsing back into the middle and fading to dots.
Directed by Gresz Industries. Three spinning dots. JARVIS OS launching. The dashboard fades in.
None of that is necessary. It is there because an operating system you enter should feel like being let in.
Why there is no keyboard
A keyboard is a request form. You translate an intention into a syntax the machine already understands, and the machine does exactly that and nothing else. It works, and it has shaped forty years of assumptions about what a computer is allowed to be.
If the machine can see your hands and hear you, typing is a downgrade. So the whole interface is two channels:
| Gesture | Action |
|---|---|
| Slide | Move the cursor |
| Pinch | Click |
| Pinch and move | Drag |
| Two fingers | Scroll |
| Fingers spread → in | Zoom in |
| Fingers in → spread | Zoom out |
A faint trail follows your fingertip across the screen, because a system that reads your body has to show you what it thinks your body is doing. Without the trail, a missed pinch feels like the machine ignoring you. With it, you can see you were half an inch off.
Voice is the primary channel, and once the OS is up there is no wake word. The cost of a wake word is that you have to remember the machine exists before you can use it.
A place, not a list
The dashboard is a globe. Not an illustration of one — the actual surface you work on. Spin it by hand. It has modes: weather across a region you are about to travel to, routes drawn as arcs, news networks surfacing over the part of the world a story came from, radio tuned by spinning the planet until you land on a country.
Everything a conventional OS would bury in a menu, this puts somewhere.
Port Calvera
Off the globe there is an island — volcanic, mid-Atlantic, entirely fictional, with three biomes and three cities. Click it and you drop to a helicopter view. Click a city and you fall into a black-and-white wireframe drone shot moving between the buildings.
Every building is a real subsystem. That is the whole conceit, and it is not decoration.
Nova Calvera, the capital, sits in a volcanic basin — towers above, arcades mid-level, data centres underground. It is where new agents are designed and built, where the system files public bulletins about what it is currently doing, and where the island’s own laws and history get written.
Alto Norte is a citadel bored into cold mountains above the cloud line. It holds the live task inspector — everything running right now, click any of it for the full plan — alongside the repair agent that spends its day fixing the system, and a reasoning collective of dozens of nodes for problems a single model cannot hold in one piece. Nova Calvera talks for the island. Alto Norte thinks for it.
Porto das Velas is the port: beaches, jungle, a harbour. Everything that flows in and out passes through here. It is deliberately the warmest of the three.
The reason for the fiction is practical. A system running dozens of agents is unreadable as a list — you scroll it and learn nothing. As a city it is legible from across the room: you can see which district is lit up and busy, and walk there.
The night
Say you are signing off and the second screen renders the day back to you. Then the machine keeps working:
- Dreaming — replaying the day, filing what happened into memory, keeping what matters
- Planning — building tomorrow, and the brief that will be waiting when you wake
- Pondering — researching the things you said that it did not fully understand
- Stimming — firing at random intervals to check nothing has been left undone
- Activating — anything dangerous or urgent rendered faintly, for a room with nobody in it
How it is built
The order matters more than the parts, and it is the opposite of the fun order.
Nothing that touches the world was written until four things existed: a vault, so no secret is ever stored in plaintext; a budget governor that meters every model call against a real ledger and refuses to spend past a ceiling; a permission gate that funnels every consequential action through one chokepoint and waits for a spoken yes or no, where silence counts as no; and a rollback spine that snapshots before any self-modification, applies it, health-checks the result in a fresh interpreter, and reverts itself automatically if the system comes back broken.
None of that is visible on screen. All of it had to exist before a single agent was allowed to touch anything, because the interesting version of this project is one where the system modifies itself — and that is only interesting if it cannot quietly destroy itself doing so.
Then the shell: a single-page app in a full-screen kiosk window, wired to the running system over an event bus, so the machine can push a banner, open a page or drop a note onto the second screen mid-sentence.
Then hands. Camera capture into a hand-tracking model, feeding a gesture state machine that is unit-tested on its own, away from any camera. The hard part is not recognising a pinch. It is that a pinch has to survive your hand being at a bad angle, hold steady while you drag, release cleanly when your hand leaves frame, and never fire twice — which is smoothing, hysteresis and a lot of tuning, none of which looks like anything in a demo.
Where it is
The boot sequence, the voice shell and hand tracking all run. The globe is up with several of its modes working. The island is next, and it is the part with real scale to it — a persistent world is a multi-year project, not a sprint, so nothing else in the system is allowed to depend on it being finished.
The story so far On the timeline →
The MK II plan
Written into the repo, with a live camera feed behind a minimal greeting.
Live aircraft on a globe
Real flights from OpenSky, with routes when you pick one.
Follow JARVIS MK II as it’s built
New moments land on the timeline first, and the big ones in the newsletter.