DSH Plugins Marketplace

DSH Plugins

Plugins

/

dsh-wayland-vdesk

O

dsh-wayland-vdesk

Manifest valid

Per-conversation off-screen Wayland desktops for DeepSeek Harness: run, screenshot, drive and watch GUI programs without touching your real desktop.

UI (client)hasBundlePatch

dsh-wayland-vdesk

A private Wayland desktop for every DeepSeek Harness conversation. Run, screenshot, drive and watch GUI programs off-screen — the windows never touch your real compositor.

License: MIT Platform: Linux DSH Node PRs welcome

English · 中文

The live viewer driving a web app inside an off-screen desktop

A DeepSeek Harness bundle that gives each conversation its own headless sway compositor (Wayland + Xwayland), streams it into the Web UI over VNC, and exposes it to the agent as ordinary tools:

vdesk_create()                                   → open this conversation's desktop
vdesk_run({command: "npm run dev"})              → start a program inside it
vdesk_shot()                                     → PNG straight into the model context
vdesk_input({actions: [{type: "click", x, y}]})  → click / drag / type / press keys
vdesk_apps() / vdesk_kill({pid})                 → see and stop what is running
vdesk_resize({width, height})                    → change the resolution live

Why

An agent that can look at pixels is a much better UI engineer — but pointing a real screen-driver at your desktop is unpleasant: windows steal focus, pop up over your work, and global input injection (ydotool, /dev/uinput) hits whatever session is focused, including yours.

dsh-wayland-vdesk gives every conversation a private compositor instead:

  • Off-screen by construction. Its own XDG_RUNTIME_DIR, its own WAYLAND_DISPLAY, its own Xwayland display. hyprctl clients never shows a thing.
  • Wayland native and X11. sway's Xwayland runs side by side, so GTK 4, Qt, Electron and Wine all work.
  • Safe input. Keyboard goes through the compositor's virtual-keyboard protocol, the pointer through sway's own seat — never through /dev/uinput. The two never meet.
  • Live and two-way. You watch the same framebuffer the agent screenshots, and you can take over with your own mouse and keyboard at any moment.

Screenshots

Overview of every conversation's desktopEmbedded in a DSH conversation
OverviewEmbedded

The overview lives in the sidebar (live thumbnails, open/stop, stop-all). The per-conversation tab shows one desktop full-size; the 程序 popover lists the programs and windows inside it and lets you stop them one by one.

Features

Per-conversation isolationOne sway instance per DSH session, keyed by conversation id.
Any resolutionPick it at creation, change it any time (vdesk_resize, 320–7680 px).
Wayland + X11Xwayland is started inside the desktop; both are captured and injected the same way.
Live viewnoVNC over WebSocket, served from one loopback port, embedded in the DSH Web UI.
Screenshots to the modelvdesk_shot stores a PNG in DSH's attachment store and returns an image block.
AudioEach desktop gets its own null sink; PCM is streamed to the page and played through a GainNode. Muted until you click 🔊.
Program controlList and kill the programs and windows a desktop owns — from the UI or from tools.
CLIThe same daemon is reachable from a shell: dsh-vdesk run, shot, env, exec, …
GPU when availablesway renders through the DRM render node; falls back to software automatically.

Requirements

Linux with a Wayland or X11 session and PipeWire/PulseAudio.

# Arch
sudo pacman -S sway wayvnc grim wtype wl-clipboard novnc

# Debian/Ubuntu (noVNC is packaged as novnc + python3-websockify)
sudo apt install sway wayvnc grim wtype wl-clipboard novnc
ProgramWhy
swaythe headless compositor (Wayland + Xwayland)
wayvncexposes the framebuffer/input over RFB on loopback
grimscreenshots
wtypekeyboard injection through the virtual-keyboard protocol
wl-clipboardpaste-based input for CJK and long text (optional)
wlrctlrelative pointer motion, needed for mouse-look and games (optional)
noVNCthe browser-side RFB client (any core/rfb.js tree; DSH_VDESK_NOVNC overrides the path)

ffmpeg is used to downscale screenshots, xdotool is optional. Node ≥ 22.

Install

  1. Install the bundle. Open Settings → Plugins → Install bundle and point it at this repository, or from a checkout:

    dsh plugin --profile web add /path/to/dsh-wayland-vdesk
    

    then add dsh-wayland-vdesk to dsh.profile.bundles in the profile's package.json. The Plugin manager does both steps for you, so prefer it.

  2. Reload the DSH Web UI. A 虚拟桌面 entry appears in the sidebar, and every conversation gains a 虚拟桌面 view tab.

  3. Check the daemon:

    dsh-vdesk status
    

    It reports the hub port, which capabilities were found, and the current desktops.

Using it

From the Web UI

  • Sidebar → 虚拟桌面 — the overview: every desktop with a live thumbnail, 打开 / 停止, and 全部停止.
  • Conversation → 虚拟桌面 tab — this conversation's desktop, full size.
    • 程序 — programs and windows inside the desktop; ✕ kills a program or closes a window.
    • 停止 — tears the desktop down.
    • 🔊 — audio, muted by default; the slider next to it sets the gain.
    • 鼠标 — capture the mouse for games: moves become relative look steps instead of absolute warps (Esc releases). Absolute pointer events cannot drive a pointer-locked client.

From an agent

vdesk_create({width: 1440, height: 900})
vdesk_run({command: "python -m http.server 5173", wait: false})
vdesk_run({command: "chromium --app=http://localhost:5173"})
vdesk_run({command: "sleep 4", wait: true})
vdesk_shot()
vdesk_input({actions: [{type: "click", x: 320, y: 180}]})
vdesk_input({actions: [{type: "type", text: "hello"}]})
vdesk_shot()

Full parameter reference: docs/tools.md.

From a shell

dsh-vdesk create --conv my-app --size 1440x900
dsh-vdesk run  my-app -- cargo run
dsh-vdesk shot my-app --out /tmp/shot.png
dsh-vdesk env  my-app            # export lines you can eval into your shell
dsh-vdesk click my-app 640 400 && dsh-vdesk type my-app "hello"

Full command reference: docs/cli.md.

How it works

DSH Web (127.0.0.1:3080)
 └── conversation view / overview panel
        │ iframe
        ▼
   dsh-vdesk hub (127.0.0.1:7317, token-guarded)
        ├── /            viewer page        ├── /overview   overview page
        ├── /vnc/<id>    RFB over WebSocket ├── /audio/<id> PCM over WebSocket
        └── /api/*       JSON API
        │
        ▼
   dsh-vdesk daemon (one long-lived Node process)
        └── session s-xxxx  (one per conversation)
              ├── sway  (WLR_BACKENDS=headless, private XDG_RUNTIME_DIR)
              │     ├── wayland-1
              │     └── Xwayland :N
              ├── wayvnc → 127.0.0.1:<random port>
              ├── null sink dsh_vdesk_<id> + parec (on demand)
              └── the programs you started

The daemon is deliberately outside the DSH process: the CLI and the tools share one state, and a DSH restart or a plugin reload cannot orphan a compositor. Details and the reasoning behind each choice: docs/architecture.md.

Security model

  • The hub binds 127.0.0.1 only, and every URL carries a per-daemon random token; the API and both WebSockets reject requests without it. Cross-origin pages cannot read the iframe URL, so they cannot obtain the token.
  • Input never leaves the desktop: no /dev/uinput, no synthetic events into your session.
  • This is display isolation, not a sandbox — programs run as you, with your files. Run untrusted code in a container or VM.

Limitations

  • Linux only (Wayland compositor + PipeWire/PulseAudio).
  • No IME inside the desktop. Use method: "paste" for CJK or long text; wtype types Latin text directly.
  • If DSH is served over HTTPS, browsers block the http:// viewer iframe as mixed content.
  • One noVNC view per desktop is fine; many simultaneous viewers are not a design goal.
  • Software rendering is used when the compositor cannot get a DRM render node.

FAQ

Does this show anything on my real desktop? No. The desktop is a separate headless sway instance with its own runtime dir; nothing is ever mapped onto your session.

Can the agent and I use it at the same time? Yes. The agent's vdesk_input goes through the compositor's seat and your browser's input goes through wayvnc's virtual devices — both land in the same desktop.

Where does audio go? Into a per-desktop null sink. Nothing reaches your speakers unless you unmute the viewer; the gain is applied in the browser, so your system volume is untouched.

How do I stop everything?

dsh-vdesk stop-hub      # daemon + all desktops

Repository layout

index.js              Host half: tools + authenticated routes      web/          viewer + overview pages
client.js             Client half: sidebar entry, panel, view tab   src/          daemon, sessions, audio, hub, CLI
cordis.patch.yml      bundle patch (one row)                        docs/         tool, CLI, architecture docs
locale/{en,zh}.json   Plugin Manager metadata                       docs/images/  screenshots

Contributing

Issues and pull requests are welcome. The repository uses Conventional Commits and signed commits.

node --check index.js && node --check client.js && node --check src/cli.js
dsh-vdesk status

License

MIT

Comments

Loading…

Similar plugins

dsh-pilot

by momasiku

Give DeepSeek Harness hands and eyes: screen_view hands the model a real PNG of your desktop, desktop_control drives the mouse, keyboard and windows, desktop_sequence runs a whole task in one call.

Tools & CapabilitiesManifest valid

★ 205

↓ 1k/wk

MIT

JavaScript

Nov 5, 2026

dsh plugin --profile web add dsh-pilot

by jing-hy

Windows-only computer use for DeepSeek Harness: captures the virtual screen and drives mouse & keyboard through 9 tools built on PowerShell and Win32 SendInput; pairs with picturereader so text-only m

Tools & CapabilitiesManifest valid

★ 5

↓ 1.4k/wk

MIT

JavaScript

Oct 1, 2026

dsh plugin --profile web add computer-user

by Loping151

VibeOS inside DeepSeek Harness: the whole web UI becomes a virtual desktop whose app windows are HTML the model writes as you use them.

UI & ExperienceManifest valid

★ 6

TypeScript

Oct 23, 2026

dsh plugin --profile web add dsh-vibeos

Windows-native computer use for DeepSeek Harness: two tools drive and read any desktop app. One batch call runs find, click, type, key, read, wait and screenshot steps in a single engine pass, so a ta

Tools & CapabilitiesManifest valid

★ 0

dsh plugin --profile web add dsh-win-computer-use

by baldovinmarques391-design

Turn desktop operation into discrete tool calls that each return the frames captured around the call.

Tools & CapabilitiesManifest valid

★ 0

MIT

JavaScript

Oct 3, 2026

dsh plugin --profile web add dsh-computer-control

by 988hj7tczd-oss

Cross-platform Computer Use plugin for DeepSeek Harness: observable desktop automation with an isolated virtual cursor, AX/UIA observation, screenshot vision, and 12 guarded tools.

Tools & CapabilitiesManifest valid

★ 69

↓ 1.8k/wk

MIT

JavaScript

Oct 5, 2026

dsh plugin --profile web add dsh-computer-use