English · 中文

Demo · Quick start · How it decides · Components · Widgets · Extending · Security · Configuration · Quality · Develop

Generative, interactive UI for DeepSeek Harness (DSH). Ask in plain language and the agent decides on its own how to answer:

• plain text when that reads best;
• native components: dashboards, data grids, charts, scatter plots and heatmaps, timelines, live function plots, KaTeX math, Mermaid diagrams, code diffs, file trees, JSON trees, forms and choice cards;
• a fully custom sandboxed live widget: Three.js scenes, canvas simulations, mini-apps.

The UI renders progressively while the answer streams. Sorting, filtering, sliders, rotation, zoom and tabs respond instantly in the browser. Only explicit actions (a button, a form, a choice, Bridge.sendPrompt) go back to the agent, either in the current session or in a new one.

It ports the ideas of CopilotKit's OpenIntelligentUI (Open Generative UI + A2UI-style declarative components) onto DSH's own extension points, with no CopilotKit, LangGraph or external router.

✨ Highlights

🎬 Demo

The real DeepSeek model, end to end

https://github.com/user-attachments/assets/e4eb0adc-659e-4c80-b473-449aa42ca00f

All 11 browser cases of the quality baseline, generated live by deepseek-flash in DSH Web 0.2.0-rc.1 (2026-10-16): text, chart, table, form round trip, Plot, Three.js with "Ask to fix", stop → regenerate, two in-place edits, Fast 3G, CDN offline and phone width. Failures are left in and marked ✗. The DeepSeek whale girl (鲸鱼娘) narrates (credits). Full video: docs/assets/real-model/demo.mp4 (2 min 27 s).

🖼️ Results at a glance (screenshots per case)

Multi-series chart: ✗ (call cut off mid-stream)	Two in-place edits with ui_patch: ✓
The chart is right, but the card says "generation interrupted"	"Make it a line chart", then "add 小程序 30/45/60/80": one card, version 3
Form submit → the agent uses the values: ✓	Damped oscillation Plot with live sliders: ✓
Three.js: ✗, then repaired with "Ask to fix"	Stopped mid-stream → regenerated: ✓
Fast 3G, cache off: ✓	CDN unreachable: ✓ (degrades as designed)
Phone width (390 px): ✓	Sortable table: ✗ (agent researched instead)

GIF version for viewers that cannot play video (e.g. npm): demo.gif.

Feature tour

Recorded live in the DeepSeek Harness desktop app with real DeepSeek models through the magpie gateway; nothing is scripted. The camera follows the cursor and zooms into each interaction; waits for the model are fast-forwarded (the badge shows the speed).

Choice: pick one option, continue here or start it in a new session · DeepSeek-V4.1-Flash

https://github.com/user-attachments/assets/1317993c-17b1-4fca-9356-c365afcc6101

Table as a data grid: typed sorting, search, per-column filters, paging, CSV export · DeepSeek-V4.1-Flash

https://github.com/user-attachments/assets/be9c74e4-2ed3-402c-bd08-d57ea8516eec

Scatter and Heatmap: hover, tap and keyboard inspection; missing data hatched, never 0 · DeepSeek-Flash

https://github.com/user-attachments/assets/cb575946-bab5-4551-9621-489dee55df07

CodeDiff, FileTree and JsonView: code as text, collapsible context, JSONPath copy · DeepSeek-Flash

https://github.com/user-attachments/assets/75900041-3880-4e21-97f8-21bfd41872ad

🚀 Quick start

You need DeepSeek Harness 0.1.7 – 0.2.x (>=0.1.7-rc.1 <0.3.0) and Node.js ≥ 22.13, on Windows, Linux or macOS. Nothing is compiled during install: the package ships its prebuilt browser bundle and has no install scripts.

dsh plugin --profile web add deepseek-harness-intelligent-ui

Restart DSH Web (dsh web) and ask for something visual, for example "compare Q1–Q4 revenue by channel" or "let me explore x²+y²+z²=2Rz in 3D". In the desktop app, use Plugins → Add plugin with the same package name.

The guided installer checks Node, the dsh version, pnpm and the profile first, then runs the same dsh plugin add:

npx deepseek-harness-intelligent-ui install     # --profile <name>, --dsh <path>, --dry-run
npx deepseek-harness-intelligent-ui doctor      # diagnose only

Compatibility. CI installs the packed plugin and runs the full browser E2E on Windows, Linux and macOS against DSH 0.1.7-rc.2, 0.2.0-rc.1, latest, next and alpha, on every push and nightly, so new DSH releases are caught early. Details and local verification history: docs/compatibility.md.

🧠 How the agent decides

The plugin injects a system-prompt section (src/guidance.js) so the model routes each turn itself:

• Plain text for greetings, short facts, explanations, code to copy, and workspace/coding work.
• ui_render for structured answers: comparisons, data to explore, dashboards, processes, formulas, functions, code reviews, choices between alternatives, input collection.
• ui_render_widget for what the catalog cannot draw: 3D, simulations, animations, games, custom tools.
• Local first: anything computable in the browser must update instantly without a message to the model.

/ui <request> forces a UI answer.

🧩 Component catalog (ui_render)

• Validated before it renders. A UI travels as a flat node list with id references (DSH's tool schema has no recursion). The Host rejects unknown types or props, dangling or duplicate ids, cycles, shape mismatches (chart series, heatmap rows), non-finite numbers, uncompilable Plot expressions and oversized data, returning ok:false with paths so the model fixes and retries.
• Text, never HTML. Cells, code, JSON and labels render as React text. CSV export neutralises spreadsheet formula injection and is RFC 4180 with a UTF-8 BOM.
• Accessible and responsive. Charts are focusable and keyboard-inspectable, trees use role=tree, touch targets grow on coarse pointers, and charts lay out at the card's real width so axis text stays legible on phones.
• Math and Mermaid render in the same sandbox as widgets from the CDN allowlist, with a static fallback when the CDN is unreachable, or fully offline with the offline resource package.

Choice and new sessions

Choice shows a question with option cards and two buttons. Continue in this session sends the picked option's prompt as your next message. Start new session creates a session in the same workspace, submits that prompt there and opens it; nothing from the current conversation is copied, so each option's prompt is self-contained. Button can do the same with sessionTarget: "new". The launch is idempotent: a double click, a retry after a failure or a reload never starts the task twice.

Roadmap and design notes for upcoming components (Map, Calendar): docs/component-roadmap.md.

🔁 Updating a UI in place (ui_patch)

When the user asks to change a UI shown earlier ("make that bar chart a line chart; leave the rest"), the agent patches it by component id instead of re-sending the whole tree.

• Identity: every ui_render result reports the UI's uiId and version 1; each accepted patch reports the new version and an id:Type outline.
• Operations: setProps (merge, unset to remove), replaceNode, insertChild and removeNode, applied in order and all-or-nothing. The next version then passes the same checks as a fresh ui_render.
• Concurrency: baseVersion is required; a stale patch is refused with the current version, and patches never run in parallel.
• Scope: the Host stores nothing. It rebuilds the current version by replaying the conversation's own history, so a patch can only address UIs of the same conversation.
• State: components keep their ids across versions, so untouched components keep their sort, filters, tab, form input, submitted lock and slider positions. A replaced component starts fresh.
• Display: while a patch streams, the previous version stays live. On success the new version renders beneath the reply and the older copy collapses to "Updated to version N". Refused, failed or stopped patches change nothing.
• Widgets are re-rendered rather than patched, and keep their UI.state.

🪟 Sandboxed widgets (ui_render_widget)

Fields follow the OpenIntelligentUI streaming contract, in this order: title, initialHeight, placeholderMessages, css, html, jsFunctions, jsExpressions. Placeholder lines show until css is complete; then a preview frame opens and the partial html is morphed in (idiomorph); when the arguments are complete, the same frame is finalized in place and the code runs.

• Bridge: Bridge.sendPrompt({ text }) disables the clicked control until the host answers; Bridge.openLink({ url }) opens links; Websandbox.connection.remote.* is kept for OpenIntelligentUI compatibility.
• Libraries: an importmap maps three, three/addons/…, d3, chart.js/auto, gsap and canvas-confetti to esm.sh.
• Recovery: script errors show a count with Reload and Ask to fix (which sends the errors to the agent).

Where the UI appears, and what survives

• While a turn runs, the UI renders inline in the tool call; one component instance spans preparing → running → result.
• When the turn ends, DSH folds tool calls into the "process" section, so the plugin re-presents each UI beneath the reply and the inline copy shrinks to a pointer.
• Interaction state is kept per UI in memory and in localStorage (bounded), so moving beneath the reply or reloading restores it. The UIs themselves are rebuilt from the durable session log.
• A stopped generation keeps the finished part (a static widget preview or the completed components) with a notice and Regenerate, even when DSH dropped a call that was still streaming.

🔌 Extending

🔒 Security model

• Widget isolation: <iframe sandbox="allow-scripts allow-forms"> with srcdoc and no allow-same-origin. The E2E verifies in a browser that parent.document, localStorage and document.cookie throw SecurityError, fetch is blocked, and window.origin === "null".
• CSP: default-src 'none'; scripts, styles, fonts and images only from cdnAllowlist; connect-src 'none' unless allowNetwork; form-action 'none'; base-uri 'none'.
• Messages: accepted only from the frame's own contentWindow with a per-frame random token and a valid shape; forged messages are dropped (verified in E2E).
• sendPrompt limits: refused for 1 s after mount, at most one per 2 s, capped at sendPromptMaxChars, queued as a normal user message.
• openLink: https only, optionally restricted to openLinkAllowedOrigins.
• Catalog: text renders as React elements, never raw HTML; Plot expressions are compiled over a fixed math table with no eval, Function or property access; new-session launches and host actions need an explicit native click.

⚙️ Configuration

The bundle patch inserts the plugin row automatically. To change settings, add to your profile's cordis.patch.yml:

- id: intelligent-ui
  config:
    allowNetwork: true
    cdnAllowlist: [https://esm.sh, https://cdn.jsdelivr.net]

📊 Real-model quality baseline

A tool call whose arguments parse is not the same as a UI the user can use. npm run eval:real measures three layers separately; a pass in one is not evidence for another.

The long tail comes from large Three.js widgets (10–20 KB of code before the preview starts). The failures are real model and agent behaviour: a call cut off mid-stream, an agent that researched on the web instead of rendering, Three.js scope slips repaired with "Ask to fix", and invalid first attempts the model fixed on retry. Each case, its checks and the full findings: docs/real-model-eval.md.

export DEEPSEEK_API_KEY='<your-key>'               # PowerShell: $env:DEEPSEEK_API_KEY = '<your-key>'
npm run eval:real                                   # mock E2E + API + browser; exits 1 on any failure
npm run eval:real -- --repeat 3                     # per-case success rates and p50/p95
npm run eval:real -- --record && npm run eval:demo  # record and cut the demo video

🛠 Develop and verify

npm install
npm run build            # client-src/ → client/client.js and renderer/renderer.js
npm test                 # unit and component tests (vitest)
npm run typecheck
npm run e2e:all          # build → install into a temp DSH home → scripted model + real DSH Web → browser checks
npm run e2e:controlled   # controlled data/actions in real Chromium
npm run e2e:extensions   # third-party component template in real Chromium
npm run e2e:offline      # offline resource package with the network cut
npm run smoke -- --spec ./deepseek-harness-intelligent-ui-<v>.tgz   # install smoke test of a packed tarball
npm run demo:features    # re-record the feature-tour videos in the DSH desktop app
npm run docs:art         # regenerate the README illustrations

• Layout: src/ is the Host half (plain ESM, no build); client-src/ is the browser half (TSX, bundled with esbuild). client/client.js and renderer/renderer.js are committed and published so npm, file: and git installs need no build step; CI fails if they are stale. Never add a prepare/postinstall script: DSH's pnpm blocks plugin build scripts.
• E2E needs Chrome (CHROME_PATH to override), dsh on PATH (DSH_BIN to override) and runs on Windows, Linux and macOS. --spec <tgz> tests a packed tarball, --record re-records the scripted demo, E2E_DATA_ONLY=1 / E2E_SESSION_ONLY=1 run one group.
• Feature-tour videos: start the desktop app with --remote-debugging-port=9333, select the model, then npm run demo:features (scripts/demo/).

📚 Documentation

Limitations

• The default online package needs reachable CDN resources for Math/Mermaid and imported widget libraries. The offline package covers Math, Mermaid and standard Three.js scenes; other addons, GSAP and external media are not included. Library failures show an explicit fallback or script error.
• Interaction state is per browser (localStorage), not synced across devices.
• A call dropped while streaming is matched to its turn by start/end time; a large clock offset between a remote DSH host and the browser could hide that card.

Credits

Concepts, the streaming widget contract and parts of the prompt guidance come from CopilotKit/OpenIntelligentUI (MIT); see THIRD_PARTY_NOTICES.md.

The DeepSeek whale girl (鲸鱼娘) who narrates the end-to-end demo, in 26 poses, is by Sutera-Diffusus/dsh-whale-musume (MIT © Sutera-Diffusus; illustrations by @SuteraWu). The unmodified files and licence are in scripts/eval/demo/assets/whale/ and are not part of the npm package.

Licensed under MIT.