dsh-computer-browser
Manifest validDesktop and browser automation in one tool surface. Background CDP page reads and clicks escalate to foreground keyboard and mouse; screen observation and application control are in the same surface.
dsh-computer-browser
Desktop + browser automation for DeepSeek Harness — one tool surface that drives Windows applications and your real, signed-in Chrome.
Status: both channels live (background CDP + foreground real keyboard/mouse). Measurements and decision rules are below.
How one search task flows
| Action | Which channel | Why (measured) |
|---|---|---|
| Send a query (address bar / URL) | background chrome_page_goto | 1.5 s to the results page on a login-free site |
| Read results / paginate | background chrome_page_read | 0.2–0.3 s per page with connection reuse; cold start eliminated |
| Click a public result link | background chrome_page_click (mode=auto) | navigation within 1.2 s = success; auto-escalates to foreground when it doesn't navigate |
| Operate something that must be visible | foreground chrome_page_click (mode=foreground) | brings the window forward and uses the real mouse, bypassing CDP clicks |
| Type into an in-site search box | foreground chrome_window_focus → keyboard | sites detect JS-assigned values; real keystrokes are steadier |
| Login/subscription sites | foreground (signed-in Chrome + KeePassXC Auto-Type) | the dedicated profile has no login state; forcing it only hits the login wall |
| Captcha / QR / risk control appears | foreground, or hand back to the user | human interaction is the only way |
| Write / submit / pay | foreground + explicit confirmation | irreversible actions |
| Two background clicks with no effect | auto-escalate to foreground | built into chrome_page_click as a fallback |
Rule precedence: interception > login > action type > site allow-list.
Tool surface
Always present (2): computer_do (enter/leave the desktop surface) · screen_observe
Once expanded, these are added:
- Observation:
screen_zoom - Keyboard/mouse:
computer_click/double_click/right_click/type/key/scroll/drag/wait - Applications:
app_list/app_launch - Verification:
computer_verify/computer_wait_for - Other:
computer_clipboard/menu/hover/stop/resume/computer_task - Browser (background):
chrome_page_read/chrome_page_click/chrome_page_goto - Browser (foreground prep):
chrome_window_focus - Login detection:
chrome_login_state(reads page facts — login URL, password field, "sign in" text, user menu, cookie count — never a site list)
Key measurements (2026-11-09)
- Background page read: 0.2–0.3 s (31 ms → 3 ms with connection reuse)
- Background navigation: fixed 1.5 s wait
- Foreground keyboard/mouse: 1–2 s per step, 4–7 s for a full chain
- Windows foreground lock: a bare
SetForegroundWindowis refused; you needShowWindow → ALT → BringWindowToTop → SetForegroundWindow, otherwise keystrokes land in DSH itself - Real mouse clicks:
SetCursorPos + mouse_event, using thescreenX/screenYfromlocate(); wait at least 2.5–3 s before reading the URL, or you misread the result as failure - The tool surface folds itself: two non-desktop tool calls collapse it, so a "bring forward + keys/mouse" sequence must finish inside one expanded window
- The dedicated Chrome dies with dsh (it is in the child process tree), so after a restart bring it back with
--user-data-dir=<profile> --remote-debugging-port=9222
Red lines
- Passwords: only via KeePassXC Auto-Type into the foreground page — never read as text, never logged, never written to a file
- Login-required sites: background tools refuse and suggest the foreground path
- Dialogs: unrelated ones are dismissed automatically; login/QR/captcha, payment/outbound submission, and "the dialog is the task" are the three exceptions that must be offered as choices
Layout
lib/desk/register.js tool registration (includes the two-channel chrome_page_*)
lib/desk/chrome-cdp.js CDP session reuse / dialog handling / locate
lib/desk/policy-rules.js site routing / confirmations / password-manager plan / channel fallback
lib/desk/win-focus.js bring-to-front / real mouse click / window enumeration
lib/desk/surface.js expand/collapse surface
Development
node --check lib/desk/register.js
node --check lib/desk/win-focus.js
node --check lib/desk/policy-rules.js
0.2.0 — three fixes from real user reports (2026-11-15)
Source: three field reports under D:\工作区\工作\产出\工具心得\ (57 calls / Navicat / database sessions).
- The tool surface is flat by default. It no longer folds. In the folded state the action tools were not registered at all, so the host answered
unknown tooland the caller never learned to runcomputer_do enterfirst (9 failures across three rounds).enter/exitremain as compatibility entry points. - Post-action verification.
computer_click/ double-click / right-click now take an AX fingerprint of the target window before and after (role+name sequence, element count, title; 2.5 s timeout), and the receipt says either "window content changed" or "no visible change — re-check recommended". When no fingerprint is available it still reportsunverifiablerather than inventing a conclusion. - Three-state receipts: applied / no visible change / delivered only.
Next version (todo): unify the coordinate space with scale, degraded markers and click refusal, automatic pid for screen_zoom, atomic select_option / wait_for, a single-call desktop_state.
0.3.0 — glue between the desktop and browser tool groups (2026-11-15)
The field reports said the two groups did not hand off smoothly. This release only fixes the loose ends; cross-surface numbering stays for stage two.
- Uniform browser receipts:
chrome_page_read / click / goto / login_statenow returnsurface: 'browser', and navigation-capable ones addurlBefore / urlAfter / changed, matching the desktop group's three-state receipt. - Lightweight task state:
lib/desk/task-state.jsremembers the current surface (desk/web), the window or tab, and the last snapshot;computer_do action="status"appends one line, and cross-surface switches are announced in the receipt. It is a status indicator, not task resumption. - Site policy in code: when
chrome_page_read / clickhits a site classified asforeground(CNKI, campus systems, trading sites), the receipt appends a hint to use real foreground input instead of relying on operator memory.
Tests: 12/12 (four new task-state unit tests).
Version mapping
| npm | Git tag | What it contains |
|---|---|---|
| 0.1.0 | (initial) | Scaffold only: lib/policy.js, lib/surface.js — no desktop engine, no CDP bridge |
| 0.2.x | v0.2.3 | Desktop engine fixes from three field reports |
| 0.3.0 | v0.3.0 | Current release: everything under lib/desk/, lib/web/, client/, tests/ |
The repository tracks the npm package from 0.3.0 onward; master and the published tarball describe the same code.
Comments
Loading…
Similar plugins
DSH 的浏览器:登录一次,它就用你的账号替你办事(查后台/填表/抓列表/多标签);要登录、验证码、弹框会停下来等你,绝不替你点「确定」;你最小化它就不吵你,AI 在哪动手你随时看得见
★ 2
MIT
TypeScript
dsh plugin --profile web add dsh-browser-playwright-codexby laoyingfuhuole
Fully background browser automation for DeepSeek Harness: one offscreen Electron window per conversation, real CDP input, no focus stealing.
★ 0
MIT
JavaScript
Nov 18, 2026
dsh plugin --profile web add dsh-browser-use-automationby dcd887
Self-managed persistent browser for DSH automation — launches its own Chrome/Edge profile, navigates, switches tabs, snapshots with semantic refs, acts by ref, evaluates JS, screenshots, and uploads f
★ 0
MIT
JavaScript
Nov 10, 2026
dsh plugin --profile web add @local/browser-ops8by borncode-app
Gives the agent native browser control through browser-use: the package installs itself into a managed environment, updates itself on every launch, and answers through first-class tools.
★ 0
MIT
JavaScript
Nov 14, 2026
dsh plugin --profile web add @local/dsh-browser-useby wangtheme-coder
Windows x64 only, 0.5.3 preview: two scoped tools (dragon_browser and dragon_desktop) that drive a visible browser through the DOM or CDP and Windows apps through UI Automation, with per-origin and pe
★ 0
MIT
JavaScript
Oct 22, 2026
dsh plugin --profile web add dsh-dragon-controlby gulagala001
Computer use with browser automation, approvals and annotations for DSH; full desktop UI and native monitoring require matching host source adapters, patching and a host build.
★ 3
NOASSERTION
JavaScript
Oct 31, 2026
dsh plugin --profile web add opencu