An experiment in server-driven interfaces

Stream the interface,
not the page.

XUL-J is a JSON version of Mozilla's XUL, sent as a stream of small operations. A server, an MQTT device or an unmodified desktop application describes what to show and what can be done. Any client decides how it looks: a browser, a terminal, or something else. Clicks go back as intents.

Try the live demo Have a model build one Read the code

This runs the real XUL-J renderer. A small script in the page plays the server's part, so the operations on the right are exactly what would arrive over the wire. Watch the log placeholder fill in, a plugin add a Rollback button after the fact, pick production to get a modal confirmation, switch View › Theme, and right-click log lines (Ctrl/Shift-click selects several). Menus open locally; only choices are sent.

Let a model build it, and run it

The protocol is small enough for a language model to write as it goes. In the generator you describe an interface and a model streams it as XUL-J, one JSON line at a time. Every line is checked against the schema before it is rendered, so you watch the UI assemble itself, and anything malformed is rejected rather than displayed.

Then the model becomes the app's backend. Typing stays in your browser. Clicking an action sends {"do":"cmd_add","inputs":{…}}, and the model answers with operations that update rows, totals and messages, or open a confirmation dialog. You can also ask for changes in plain words.

Bring your own key

The generator calls OpenRouter directly from your browser, with any model you choose. Sign in with OpenRouter issues a key you control, or you can paste one. It is kept in this tab (or on this device, if you ask), sent only to openrouter.ai, and can be forgotten with one click. There is no XUL-J server in between. The generated interface is data, never code: the renderer runs nothing it receives.

AI agents, without screenshots

An interface streamed as XUL-J is already what an agent needs: a labelled tree with explicit commands, enabled and disabled state, tables and dialogs. The XUL-J MCP server hands that tree to any MCP client. Put it in front of a bridge, and a 2005 WinForms or Swing app becomes something an agent can operate reliably: no screen scraping, no clicking at coordinates.

claude mcp add legacy-orders -- \
  node xul-j/mcp/server.js --url http://127.0.0.1:8092
// do_command cmd_addButton
+ label "Customer is required." [errorLabel] (danger)
~ label "Order not added" [statusLabel]

// open_context_menu ordersGrid
item "Duplicate order" [duplicateItem] → cmd_duplicateItem
item "Delete order…" [deleteSelectedItem] → cmd_deleteSelectedItem

// do_command cmd_exportItem, then Save
~ label "Exported 1 orders to agent-export.csv" [statusLabel]
~ enabled again: file, edit, help and 14 more

How it works

The server sends operations, one JSON object per line. Each one is valid on its own, so a cut-off stream is never a broken document. It is just a shorter one.

// server → client
{"op":"command","id":"cmd_deploy","label":"Deploy","key":"ctrl+enter"}
{"op":"node","in":"root","tag":"window","id":"win","label":"Deploy console"}
{"op":"node","in":"win","tag":"toolbar","id":"tb","children":[
  {"tag":"toolbarbutton","command":"cmd_deploy"}]}
{"op":"node","in":"win","tag":"pending","id":"log","hint":"tree","flex":1}
{"op":"replace","id":"log","tag":"tree","rows":{"source":"log"},…}
{"op":"rows","source":"log","append":[{"t":"12:01","msg":"build ok"}]}
{"op":"broadcast","id":"busy","value":true}

// client → server
{"op":"do","command":"cmd_deploy"}
{"op":"input","id":"filter","value":"error"}

XUL already had the right ideas, and each one turns out to suit streaming:

Stable ids and overlays
Every element can be addressed, so anything can be patched later. A node whose parent hasn't arrived yet simply waits for it.
Commands
Intents are separate from widgets. Disable cmd_deploy once, and its button, menu item and shortcut all follow.
Broadcasters
Named values that attributes observe: observes: {"disabled": "busy"}.
Boxes and flex
A half-arrived layout still looks reasonable, and a pending placeholder holds space until its widget arrives.
Themes as tokens
A producer can send colours, radius, density and font, never CSS. The viewer's dark mode, contrast settings and WCAG checks win.
Data apart from structure
A tree is declared once and rows stream into it. 100,000 rows render as about 30 DOM nodes.
XUL-J architecture Producers (a server app, an MQTT device, WinForms and Swing apps through bridges) emit XUL-J operations over SSE or MQTT to renderers (a browser, a terminal). Intents flow back. Server app (Node) MQTT device WinForms app + net-bridge Swing app + java-bridge XUL-J opsSSE · per session, resumable XUL-J topicsMQTT · retained = current UI Browser renderer Terminal / headless intents

Two transports, one protocol

SSE

The server keeps a numbered log of operations for each session. When a client reconnects it sends Last-Event-ID and gets only what it missed. A new client rebuilds the identical UI from the log. Intents arrive as small POSTs, checked against the current commands, so a disabled button really is disabled.

MQTT

Each widget is a retained topic, xulj/<app>/node/<id>, so the broker holds the current interface. A late joiner gets it instantly with nothing to replay, and the publisher holds no viewer connections. Any device that can publish MQTT can publish its own UI, and its last will marks it offline.

Bridges: old desktop apps, served to browsers

The bridges host an unmodified desktop application. Each browser session gets its own instance of the app's window, kept off-screen. The bridge walks the live widget tree every 50 ms, compares it with the previous frame and streams only the differences. Clicks and edits from the browser become real events on the real controls, so the app's own handlers, validation and timers run as usual.

net-bridge .NET WinForms

xulj-host.exe App.exe. Layout is inferred from Dock, Anchor and the designer's positions, and the designer's column alignment is preserved. &Add becomes Alt+A, and menu shortcuts carry over. MessageBox, file dialogs and InputBox are intercepted (via Harmony) and shown in the browser.

Documentation →

java-bridge Java Swing

java -jar xulj-swing.jar --jar App.jar. Layout comes from the layout managers, or from real bounds for GridBag, GroupLayout and null layouts. Ids come from the app's own field names. JOptionPane and JFileChooser work in the browser, and System.exit ends only that session.

Documentation →

Dialogs in both bridges
The app callsThe browser sees
Message box, confirm, input boxA modal window with icon, message, the same buttons, Enter for the default and Escape to cancel
Open-file dialogA file picker. The browser uploads, and the app reads the uploaded copy
Save-file dialogA file-name prompt. The app writes the file, and the browser downloads it
Print, colour, folder…Cancelled, with a notification

The protocol in one table

Operations are validated against a JSON Schema by producers, and by MQTT viewers since no server sits in between.

opmeaning
nodeinsert an element (with nested children) into in, optionally before a sibling or at an order
replaceswap an element for a new one, typically a pending placeholder for the real widget
set / removechange an element's attributes, or remove it
commanddeclare or update an intent: label, disabled, keyboard shortcut
broadcastset a named value that attributes observe
rowsappend rows to (or clear) a data source bound to trees
themedesign tokens (colours, radius, density, font), with an optional dark palette
resetstart over
download / notifytransient: a file to download, a message to show; never replayed

Widgets: window (optionally modal), vbox, hbox, box, spacer, groupbox, toolbar, statusbar, menubar, menu, menuitem, menuseparator, menupopup, label, description, button, toolbarbutton, textbox, checkbox, menulist, tabbox, tabpanel, deck, tree, progressmeter, filepicker, pending. Menus open and close in the browser; only choosing an item is sent back. The browser answers with intents: do, input, select, activate and contextmenu, plus uploads for file pickers. The full reference is in the xul-j README.

Run it

git clone https://github.com/xul-j/xul-j && cd xul-j
npm install
npm start            # http://127.0.0.1:8080: the deploy console over SSE
npm test             # end-to-end: valid ops, resume, overlays, 100k rows
npm run tty          # the same stream, rendered in a terminal

The MQTT setup, and the bridges' Docker builds (Mono + Xvfb, JDK + Xvfb) and test suites, are described in each repository's README.

Status

XUL-J is a working prototype with end-to-end tests for every transport and bridge, not a finished product. The bridges have so far been tested on Linux (Mono's WinForms, OpenJDK Swing), not yet on Windows. Cell editing isn't mapped yet, custom-painted controls don't render, and there is no authentication: put a bridge behind your own access control. Ideas and issues are welcome on GitHub.