dsh-context-diet

A global DeepSeek Harness plugin that removes tokens from tool output before that output can enter a model request — without removing anything the model may need, and without changing what the model is able to do.

tests/t0.py::case PASSED
tests/t1.py::case PASSED
…
[... 12235 of 12416 characters omitted by dsh-context-diet; 4 diagnostic lines
 retained, middle elided ...]
______________________ test_charge_card_rejects_expired _______________________
src/payments/charge.py:118: in charge_card
E   CardExpiredError: card expired 2024-01-31
FAILED tests/test_payments.py::test_charge_card_rejects_expired
[…]
…899 tests collected…
(Trimmed 12235 characters. Full normalized result stored at:
 /…/session-…/bash.txt. Use read with offset/limit, or grep this path.)

The problem it solves

Every tool result becomes part of the conversation the moment it is logged, and from then on the model re-reads it on every step of the session. The shipped dsh-compaction-tool-result-pruner only trims once a compaction trigger qualifies, and rewriting a result that late invalidates the prompt-cache prefix that was already established — so trimming late costs extra as well as saving.

dsh-context-diet runs at tools/post-execute, the last waterfall before the registry freezes a result into a tool/result event. A rewrite there belongs to the call it came from: there is no cache prefix to invalidate, no second copy to reconcile, and no durable edit to replay.

What it removes

1. Lossless normalization (on by default)

Reduces output to what a terminal would actually display. A terminal shows no glyph for an escape sequence, so removing one cannot change meaning.

Transform	Example	Result
Escape sequences	\e[31merror\e[0m	error
OSC hyperlinks and titles	\e]8;;http://x\a link	link
Carriage-return overlays	10%\r50%\r100%	100%
Backspace erasure	ab\bc	ac
Trailing whitespace	a \n	a\n
Blank-line padding	a\n\n\n\n\nb	a\n\nb

2. Annotated compaction (on by default)

A run of at least repeatThreshold (8) byte-identical consecutive lines that is also at least repeatMinRunChars (1000) characters long becomes one line naming the exact repeat count:

npm warn deprecated inflight@1.0.6: This module is not supported…  [x400 identical lines omitted by dsh-context-diet]

The run is collapsed only when the annotation is strictly shorter than the run it replaces, so this pass can never enlarge a result. The count is preserved, so the run is exactly reconstructible from the annotation and nothing is dropped silently — which is why this pass needs no recovery file.

3. Recoverable, structure-aware trimming (on by default above 32,000 characters)

A result still larger than maxInlineChars is replaced by:

1. its head (headChars),
2. an omission marker stating exactly how much was dropped,
3. the diagnostically significant lines from the omitted middle — errors, failures, exceptions, assertions, stack frames, diff hunks and headers, and file:line:col references (significantChars), and
4. its tail (tailChars),

followed by a notice naming a spill file that holds the complete normalized text, with the backend's own retrieval guidance.

Blind head/tail cutting discards the middle unconditionally. Keeping the diagnostic lines makes the same budget strictly more informative, not less — measured on a 900-test pytest -v run, every one of CardExpiredError, charge.py:118, test_charge_card_rejects_expired, and 1 failed, 899 passed survives with this plugin and is lost by a head/tail cut of the same size.

Why quality is not affected

• Nothing is dropped without a record. Every removal is either a control character a terminal never displays, or something that is counted, annotated, and written verbatim to a spill file the notice tells the model how to read.
• read is exempt by default. File-content output is line-addressed: blank-line or trailing-whitespace rewriting would corrupt content a model may quote back at an editor. This matches the shipped dsh-spill-policy, which exempts read for the same reason.
• The default budget is conservative. No result under 32,000 characters — roughly 8,000 tokens — is touched at all, except by the lossless passes.
• It cannot alter capability. It registers no tools, removes none, changes no prompts, and changes no model or reasoning settings. Tool schemas, the system prompt, and the tool catalog are byte-identical to a run without it, so the prompt cache behaves exactly as before.
• It never makes a model call and never adds a request.
• It fails open. Any unexpected shape, missing service, or thrown error returns the incoming result untouched and logs a warning. verify.mjs covers each of those paths.

Measured effect

On a synthetic but realistic bash-heavy corpus (900-test pytest -v, npm install with 400 repeated deprecation warnings, 2,400-line cargo build, a curl progress bar, and a 3,000-entry find), with shipped defaults:

Result	Before	Model sees	Removed
pytest -v, buried failure	48,002	12,555	74%
npm install, 400 repeats	33,711	236	99%
cargo build, buried error	158,997	20,218	87%
curl progress bar	4,572	61	99%
find over a large tree	130,890	12,218	91%
Total	376,172	45,288	88%

The lossless tier alone accounts for 15% of that; the rest is annotated compaction and recoverable trimming. Character counts are not billed tokens — the figures are indicative, and /diet reports the real numbers for the session you are in.

Verified against a running Harness

Both passes were exercised live, through the real tools/post-execute waterfall of a running dsh web profile, using the real dsh-spill-local backend:

$ for i in $(seq 1 30); do echo "SAME LINE PADDING PADDING PADDING PADDING PADDING"; done
SAME LINE PADDING PADDING PADDING PADDING PADDING  [x30 identical lines omitted by dsh-context-diet]

$ awk '… 570 lines with an error at line 285 …'
download asset 0000 of 0570 ================================================
…
[... 31890 of 43967 characters omitted by dsh-context-diet; 1 diagnostic lines retained, middle elided ...]

src/engine/pipeline.rs:204:7 - error TS2345: UNIQUE_BURIED_DIAGNOSTIC_MARKER

[...]
…
download asset 0569 of 0570 ================================================

(Trimmed 31890 characters. Full normalized result stored at:
 /…/c1cbcd405966-bash.txt. Use read with offset/limit, or grep this path to search within it.)

The 43,967-character result reached the model as 12,077 characters (72% smaller), the diagnostic line that a head/tail cut would have dropped was retained, and the recovery file was verified to hold all 571 lines of the original output.

Install

# 1. install the package into a profile
dsh plugin --profile web add link:/path/to/dsh-context-diet

2. Enable the bundle: Settings → Plugins → Installed → dsh-context-diet → on. An agent driving the plugin manager does both steps at once with install_bundle.

The bundle inserts one global Host row, so it applies to every session in the profile — not to a single session. It needs no credentials and no other plugin; the tools/post-execute seam is part of dsh-tools, and the spill backend comes from dsh-spill-local, both of which the shipped compositions provide. Without a spill backend the plugin still normalizes and compacts, and simply declines to trim, because trimming without a recovery file would lose information.

Configuration

Every field is optional; omitting it keeps the shipped default. Override any of them in the profile's own cordis.patch.yml, which is applied after bundle layers and therefore survives plugin upgrades:

- id: context-diet
  name: 'dsh-context-diet'
  config:
    maxInlineChars: 64000     # 0 keeps only the lossless and annotated passes
    exemptTools: ['read', 'read_image']
    repeatThreshold: 12       # only collapse very long identical runs

Field	Default	Meaning
normalize	true	Master switch for the lossless passes
stripAnsi	true	Remove terminal escape sequences
resolveCarriageReturns	true	Reduce \r overlays to the final visible line
trimTrailingWhitespace	true	Remove trailing spaces and tabs
collapseBlankRuns	true	Cap runs of blank lines
maxConsecutiveBlankLines	1	How many blank lines to keep
collapseRepeatedLines	true	Annotate long runs of identical lines
repeatThreshold	8	Minimum identical lines in a collapsible run
repeatMinRunChars	1000	Minimum characters in a collapsible run
maxInlineChars	32000	Trim above this size; 0 disables trimming
headChars	8000	Leading characters retained by a trim
tailChars	4000	Trailing characters retained by a trim
significantChars	8000	Budget for retained diagnostic lines
significantPattern	built in	Regex selecting the diagnostic lines to keep
exemptTools	['read']	Tools the plugin never touches
recoveryNoticePattern	built in	Recognizes a result that already names a recovery file

maxInlineChars must leave room for the omission markers and the recovery notice: headChars + tailChars + significantChars + 700 must not exceed it. A config that violates this is rejected at activation with that arithmetic in the message, rather than producing a trim that cannot converge.

Asking what it saved

Run /diet in a session:

dsh-context-diet — 138,204 characters removed (51.2%)
  results inspected        93
  characters in            270,163
  characters out           131,959
  removed, lossless        12,308
  removed, annotated       4,120
  removed, trimming        121,776 across 4 recoverable result(s)
  estimated tokens saved   34,551 (chars / 4)

/diet reset clears the session's counters. The counters are plugin memory only: nothing is written to the session log, so replay, fork, and resume are unaffected.

Interaction with the shipped plugins

• dsh-spill-policy (shipped with maxInlineTokens: 12500) registers with prepend: true, so it is the outermost listener and gets the last word. This plugin runs after it and produces at most maxInlineChars characters — about 8,000 tokens, under that cap — so spill-policy finds nothing left to bound. The recoveryNoticePattern check additionally makes this plugin stand down on any result that already names a recovery file, so the two can be reconfigured without producing two spill files for one result.
• dsh-compaction-tool-result-pruner still runs under compaction pressure. It has strictly less to do here, which means character pressure is relieved earlier and summarization is triggered later — the expensive part of a long session.
• dsh-token-meter measures the request surface; this plugin makes that surface smaller before the meter ever sees it.
• run_code / PTC sub-dispatch logs (tools/ptc-dispatch-log) are not handled. PTC presentation is off in shipped compositions, and the extra seam would double the maintenance surface for no benefit today.

How it stays working across Harness upgrades

The plugin is written to depend on as little as possible, and to degrade rather than break:

• No Harness imports at all. index.js imports only ./normalize.js. There is no peer dependency on dsh-tools, dsh-llm, dsh-token-meter, or @deepseek-ai/schemastery that a version bump could invalidate, and no build step, so no compiled artifact can go stale.
• Only stable seams. The tools/post-execute waterfall, the optional spillStore service (contract: saveText(input) -> { locator, retrievalHint }), the optional commands service, and ctx.logger. Each is a published Service or Event contract, and each is used through ctx.get / ctx.inject so its absence is a normal state rather than an activation failure.
• Config via the Standard Schema protocol. Config implements ~standard.validate, which cordis validates natively, so no schema library is loaded. Unknown or non-object config never throws inside apply.
• Fail-open everywhere. The handler awaits next() first, so it never swallows another plugin's decision, and it replaces only the rendered projection — the registry merges it over result, so a tool's structured value is untouched. A non-accept decision, an outcome with no text blocks, an outcome whose shape is unexpected, or any thrown error returns exactly what it received.
• No durable state and no new event types. The plugin writes no session events, so a session it touched reopens, forks, and replays like any other.

Verifying it

node verify.mjs

52 checks cover the text transforms, the trim invariants, the config protocol, and the plugin wiring — including the identity-preserving no-op path, the recovery-file path, and every fail-open path (no spill backend, throwing spill backend, malformed result, unknown session owner, missing command registry). The script has no dependencies and does not need a Harness profile.

Editing the plugin while a Harness is running

A Harness process caches one module generation per plugin specifier for its whole lifetime. Editing index.js, reinstalling, moving the package, disabling and re-enabling the bundle, and even changing the package's exports map all leave the already-loaded generation running; only a specifier the process has never imported gets a fresh generation (install_bundle does load a brand-new package live). So after changing this plugin:

• restart the Host (dsh web) to load the new code, or
• install it under a new package name to load it immediately.

This is stock DeepSeek Harness behavior — "replacing an installed package requires restart to load a fresh JavaScript module generation" — not something specific to this plugin. If you install a new plugin and it appears inert, check this before debugging the plugin.