dsh-mcp-test

Tool-surface snapshot and drift gate for DeepSeek Harness (dsh).

Everything the model can call is a supply-chain surface: first-party tools, plugin tools, MCP tools. This plugin snapshots the full visible tool registry (name / description / parameters via ctx.tools.schemas()), stores a baseline you commit, and diffs every registry change against it — so upgrading a plugin or MCP server that silently adds, removes, or reshapes a tool fails loud instead of slipping through. Zero LLM calls in the trust path.

Family: dsh-read-guard (what agents read) · dsh-pr-gate (what they write) · dsh-attest (what they did) · dsh-mcp-test (what they can call).

Why now (checked 2026-10-12)

The dsh supply-chain lane has seven static-scanner plugins (install-time vetting of package manifests) — none watches the live tool surface. dsh-plugin-observatory audits plugin compatibility at the loader level, not the model-visible schema level. Nobody answers the question that actually matters at upgrade time: "did what my model can call just change?" The schema-diff methodology is ported from mcp-test's upgrade gate, applied to the whole registry instead of one MCP connection.

How it works

boot / tools/change ──> snapshot ctx.tools.schemas()
                   ──> baseline present? diff : establish baseline
                   ──> drift: added / removed / changed(name.description|parameters)
                   ──> warn (default) or rebaseline (explicit opt-in)

• The baseline file (.dsh-mcp-test/baseline.json, name-sorted, stable JSON) is meant to be committed — it is the contract.
• Parameter diffs are key-order-insensitive (reordering a schema is not drift; changing it is).
• ignoreTools excludes your own deliberately-dynamic tools.

v0.1 ships the snapshot + drift pillar. The behavioral probe pillar (functional + injection tests against running MCP servers, mcp-test's headline feature) is on the roadmap.

Install

dsh plugin --profile default add dsh-mcp-test

From npm — no build step, no allowBuilds authorization; git installs are equally prebuilt (lib/ ships in the repo).

Configure

- id: dsh-mcp-test
  name: dsh-mcp-test
  config:
    baselinePath: .dsh-mcp-test/baseline.json
    rebaselineOnDrift: false   # true only when accepting an intended upgrade
    ignoreTools: []

Develop

npm install
npm test

Roadmap

• Probe pillar: functional + injection probes against running MCP servers (port of mcp-test's runner)
• Per-source snapshots (first-party vs plugin vs MCP in separate baselines)
• Drift → CI bridge: surface file usable as a repo-level gate artifact

Kill criteria

If dsh ships first-party tool-surface pinning, or the observatory-style plugins add schema-level drift gating, this freezes at its last working version with a postmortem here.