Browser Runtime

The problem

Browser automation becomes unreliable when it confuses seeing a browser target with being allowed to control it. Stale sessions, multiple profiles, reconnects, user-owned tabs, and interrupted workflows can all turn a convenient script into an unsafe control surface.

What existed before

Early browser workflows could accomplish real tasks, but the hard problem was not clicking elements. It was proving which Chrome profile and tab a workflow currently owned, keeping that authority fresh across navigation and reconnects, and cleaning it up safely when work stopped unexpectedly.

The hypothesis

A browser control layer can stay useful without becoming an unrestricted remote-control API if authority is explicit and short-lived, interactions are exposed as bounded semantic primitives, and failures close rather than silently falling back to a different control path.

The first test

Build one current Chrome runtime, prove cross-profile isolation on real browser profiles, install it through the normal product path, then use it on real workflows. Treat every real-use failure as evidence: classify whether it belongs in the generic runtime, make the minimum fix, and re-run live acceptance before freezing the core again.

What we built

A Rust-based local runtime with a Chrome extension, explicit profile authority, task sessions, execution epochs, tab leases, bounded semantic observe/scroll/activate/type interactions, reconnect and freshness rules, and runtime-managed expiry for idle authority. Site-specific workflows remain outside the core.

Evidence from use

The installed v0.2 path passed live cross-profile isolation and real Chrome connectivity checks. Real long-timeline use exposed the need for a bounded generic scroll primitive, which was added and then used to continue through an X profile timeline without adding X-specific commands to the core. A later interrupted workflow exposed orphaned browser authority; runtime-managed idle expiry and safe reclamation were implemented and accepted live, after which the core returned to usage-driven development.

Where it stands

The browser runtime now has a bounded authority and lifecycle model that has survived real usage rather than only synthetic demos. It is still a building product, not a finished automation platform: new core capability is added only when repeated real work demonstrates that the missing behavior is genuinely browser-generic.

What we learned

• Discovery is not authority. A browser target being visible does not mean a workflow is entitled to act on it.
• Reconnect should restore transport, not silently transfer control from one browser authority context to another.
• Real usage is a better capability backlog than speculative feature completeness; generic primitives should be added only after the owner of a failure is clear.
• Interrupted automation needs a lifecycle policy that can reclaim stale authority without guessing that the caller is dead or closing user-owned tabs destructively.

Capabilities

macOS product engineering · Browser automation · Agent infrastructure · Rust · Safety-oriented systems design

The hard part was not the click

A browser can expose a list of profiles, windows, and tabs very easily. That list is useful for discovery, but it is a poor definition of control.

The product question became: what exactly is this workflow allowed to touch right now, and what should happen when that answer becomes stale?

That led to a model where profile and tab authority is acquired explicitly, bound to a task and execution window, and invalidated when the surrounding browser state changes.

Keep the core smaller than the workflows

The runtime exposes a small set of semantic operations such as observing a bounded part of a page, continuing one viewport at a time, activating a freshly observed target, and typing into a freshly observed field.

It deliberately does not contain commands for X, ChatGPT, publishing platforms, or other individual sites. Repeated site knowledge belongs in adapters and workflows above the runtime unless real evidence shows that a missing capability is browser-generic.

Real use changed the design

A long profile-timeline task exposed that observation alone was not enough; the runtime needed a bounded continuation primitive. That became a generic scroll operation rather than an X-specific feature.

Later, an interrupted workflow left valid browser authority behind after the caller disappeared. Instead of building a caller-death detector, the runtime moved to bounded, runtime-owned authority lifetime: old authority can expire and be reclaimed safely while user-owned browser state remains protected.

The current development rule is therefore deliberately conservative: use it on real work, capture the failure, identify the correct owner, make the smallest generic change, accept it live, then freeze the core again.