How it works

Nearshell is designed with low latency as the first goal. This page explains the trade-offs it makes to get there — no implementation code, just the design.

Control plane vs. realtime plane

A core design decision is to split ordered, reliable control from low-latency, droppable realtime data into two paths:

• Control plane (reliable) — terminal toggles, mode switches, capture preferences, system actions; carried on a reliable, ordered channel so nothing is lost or reordered;
• Realtime plane (low latency) — video, mouse / keyboard input, cursor position, terminal text; optimised for minimum latency, free to drop stale events.

The two have nearly opposite requirements; mixing them makes both worse. Split apart, each can be as good as it gets.

Making latency imperceptible

Beyond the channel split, Nearshell uses a few techniques specifically to fight round-trip latency:

• 8ms bounded event coalescing — the sender never backs up on stale move / scroll events while preserving fractional scroll remainder;
• Local cursor prediction — the iPhone predicts and draws the cursor locally on the frame instead of waiting for the Mac to echo it back, so the pointer feels immediate;
• No reconnect on window switch — switching between full-screen and “follow the focused window” doesn’t restart the media connection, so the transition is seamless.

Direct-first, nearest fallback

The connection tries peer-to-peer (P2P) direct first; only when the two networks can’t reach each other directly does it fall back to relaying through our nearest TURN node. Either way the content is end-to-end encrypted — a relay node only ever forwards ciphertext.

The diagnostics view labels the current path Direct or TURN, so you can always see which one you’re on.

The path of a frame

ScreenCaptureKit  →  encode (60fps · long edge 1920 · end-to-end encrypted)
        │                          │
     Mac capture           direct / nearest TURN
                                   ↓
                       iPhone decode → Metal render
                                   ↓
                     locally predicted cursor overlay

Focused-window mode

Beyond full-screen capture, Nearshell can capture and control just the current focused window: less visual and bandwidth load, and a tighter focus on a small phone screen. Switching doesn’t restart the media connection; with no focus it freezes the last frame and pauses input.

More on security in the Security model; privacy boundaries are covered in the Privacy section on the home page.