dsh-ml-workbench
Manifest validDSH adapter for MLflow, Optuna and scikit-learn: track, tune, branch, compare, predict and reproduce local CSV experiments
DSH ML Workbench
Installation and scope · Reviewable evidence · Prepared catalog submission
中文说明 · API · Validation · Upstream reuse
A DeepSeek Harness plugin adapting MLflow, Optuna and scikit-learn into an agent-accessible experiment workflow. Version 0.4.0 provides 20 tools for CSV inspection, experiments, durable tuning, comparisons, prediction and reproducible delivery. Training algorithms, sampling, model serialization and tracking come from established libraries. See the candid capability review for the remaining gap to full upstream platforms.
Ask DSH to train several repurchase classifiers, then say: “Keep the third experiment's fitted preprocessing, try a faster model, and compare.” The branch gets a new MLflow run, copies the exact fitted transformer, retains the same validation rows and leaves the parent untouched.
Install
Requires Python 3.11+ (the lock file was tested on 3.12.0), Node 24.9+, and DSH 0.2.2 (tested against 0.2.2-alpha.3). This is a local, single-user backend. Install the Python backend separately from the DSH bundle.
Download the Python wheel, requirements.lock and plugin .tgz from Releases. Initialize DSH Desktop once, then fully quit it before installing into its profile. On Windows, from a dedicated folder:
py -3.12 -m venv .venv
.\.venv\Scripts\python.exe -m pip install -r requirements.lock .\dsh_ml_workbench-0.4.0-py3-none-any.whl
dsh plugin --profile desktop add .\dsh-ml-workbench-0.4.0.tgz
Alternatively enable Custom plugin index in DSH's experimental plugins, open Discover, and add:
https://raw.githubusercontent.com/shinerain1/dsh-ml-workbench/main/market/index.json
This is a published third-party index. DSH currently has no bundled central index; this is not a claim of official endorsement. The index pins an immutable Git commit and the Git package includes compiled JavaScript. Python must still be installed as above.
Set the plugin's pythonExecutable to the absolute venv interpreter path and root to an absolute writable data directory. The plugin settings also accept this DSH patch:
- id: ml-workbench
config:
pythonExecutable: C:/ml-workbench/.venv/Scripts/python.exe
root: C:/my-project/.dsh-ml-workbench
executionMode: dsh
allowCustomCode: false
Call ml_workbench_health first. Use the same root after restarting DSH to recover runs. On macOS/Linux use .venv/bin/python; these platforms have not been verified for this release.
Alternatively run the installation helper from this checkout or an extracted plugin package directory:
py -3.12 scripts/install.py --venv C:/ml-workbench/.venv --workspace C:/my-project --wheel C:/downloads/dsh_ml_workbench-0.4.0-py3-none-any.whl
It installs locked dependencies, checks backend/plugin version equality and local SQLite writes, and writes ml-workbench.config.json inside the venv. Apply that snippet to the plugin; the helper never overwrites a DSH profile. In dsh mode the root must be in the Session workspace or an explicitly approved writable root. No extra grants are added. Run python -m dsh_ml_workbench doctor --root PATH independently to diagnose an installation; only an actual DSH tool call verifies OS sandbox access.
Workflow
ml_dataset_inspectreports missing values, types, duplicates and possible identifiers.ml_experiment_createthen snapshots CSV, code, parameters, environment and validation indices. Arequest_keymakes creation retries idempotent.ml_experiment_runcombines create and start; the two original tools remain available. Useml_experiment_waitfor bounded waits and DSH PTC to compose calls. Local workers survive DSH shutdown; DSH-mode workers stop with their owning plugin/process range.ml_experiment_listrestores run IDs after a session restart. A dead worker is reported asinterrupted, never successful.ml_experiment_branchcreates a new experiment. By default it reuses a successful parent's exact fitted preprocessing. Setreuse_preprocessing: falseto change processing or restart a failed fit.ml_experiment_diffexplains configuration, code and evaluator changes.ml_experiment_comparerefuses incompatible conditions. Classification adds ROC-AUC, average precision and log loss when defined; reports include a dummy baseline and confusion matrix.ml_search_create/step/get/canceladapt persistent Optuna tuning. Each step starts at most one asynchronous trial; repeating calls is safe, and the same study resumes after a DSH restart. Search contract.ml_model_predictapplies a verified model to another local CSV, checks required features and writes predictions/probabilities plus a manifest.- Select one model using validation only, then call
ml_experiment_finalizeto evaluate its reserved final test. New runs default to 60/20/20 train/validation/final-test partitions (group/time constraints may adjust row counts). ml_experiment_exportverifies snapshots and MLflow metrics before delivering a ZIP with model, code, HTML/Markdown report, environment and standalone reproduction. CSV is omitted unlessinclude_data: true.
Example creation arguments:
{"dataset_path":"C:/data/customers.csv","target":"repurchased","task":"classification","name":"forest","model":"random_forest","params":{"n_estimators":100},"preprocessing":{"drop_columns":["customer_id"]},"request_key":"customers-forest-001"}
Example branch arguments:
{"run_id":"<third run ID>","changes":{"name":"third preprocessing, linear model","model":"logistic_regression","params":{}},"reuse_preprocessing":true,"request_key":"customers-linear-004"}
changes replaces specified top-level fields, including the whole params object. No run ID is reused for a changed experiment. Branching is restart-from-data, not optimizer checkpoint continuation.
Choose evaluation boundaries explicitly. split: {"strategy":"group","column":"customer_id"} keeps a customer entirely on one side; split: {"strategy":"time","column":"event_date"} puts later ISO timestamps in validation without splitting equal timestamps. The split column is automatically excluded from features. Time mode does not implement a forecasting pipeline or an embargo gap; grouped holdout does not guarantee class balance.
When upgrading from 0.1.0, keep the same root. Old records remain readable/exportable. A fitted-preprocessing branch retains the parent's snapshotted recipe and evaluator; cross-version evaluators are not silently ranked together. To use new metrics or tuning, create a branch with reuse_preprocessing:false to refit using the current recipe.
Delivery and MLflow
Unzip a delivery into a dedicated directory and use a fresh virtual environment:
python -m pip install -r inputs/requirements.txt
python reproduce.py --verify-only
python reproduce.py --data /absolute/path/to/original.csv
Omit --data when the export includes its CSV. Reproduction checks the data hash, environment, snapshot checksums and reproduced metrics. The original workbench package and tracking database are not needed. Load a trusted exported model with mlflow.sklearn.load_model("output/model"); its preprocessing is included.
The backend uses native MLflow tracking. ml_workbench_health returns the tracking_uri. To browse it locally:
mlflow ui --backend-store-uri sqlite:///C:/ml-workbench/records/mlflow.sqlite3 --host 127.0.0.1
Do not edit tracked metrics or snapshot files: consistency validation will reject mismatched deliveries.
Custom code and scope
DSH can author a scikit-learn transformer factory (examples/preprocessor.py) or estimator factory (examples/estimator.py). Enable allowCustomCode only for reviewed local Python. executionMode: dsh routes control and training through DSH sandbox, sandboxPolicy and subprocess services with no local fallback. local preserves durable host execution. Windows DSH reports partial file confinement: this is not a read restriction, network sandbox or hostile-code boundary. Models use Python serialization; load only trusted artifacts. A root is for one local user, not an untrusted multi-tenant server.
The default is an IID holdout, with explicit group/time alternatives. Forecasting, GPU training, nested cross-validation, automatic leakage detection, DVC storage, model serving and RD-Agent orchestration are not implemented. Optuna runs sequentially; DSH must poll step to advance, while each new fit has an independently enforced background deadline. Remove identifiers and post-outcome columns explicitly. Repeated model selection can overfit validation. Finalize the chosen model once on the reserved final partition; the same partition can select only one run in this local root. This is a workflow guard, not access control over the original CSV. Legacy two-way splits remain unchanged; final_test_fraction=0 explicitly opts out. Local synthetic results do not establish business value or improved agent productivity.
Development and evidence
py -3.12 -m venv .venv
.\.venv\Scripts\python.exe -m pip install -r requirements.lock
.\.venv\Scripts\python.exe -m pip install -e ".[dev]"
npm ci --ignore-scripts
npm run check
.\.venv\Scripts\python.exe -m pytest tests -q
.\.venv\Scripts\python.exe scripts/demo.py --root .verification/new-demo --report docs/validation-results.json
.\.venv\Scripts\python.exe scripts/demo-v2.py --root .verification/new-demo-v2 --report docs/v2-results.json
Use a new demo root each time. Validation reports actual runs, failure injection and limitations. Architecture explains which guarantees the adapter adds. MIT license for this adapter; upstream licenses remain with their projects.
Session recovery and exact-call retry identity: DSH integration. Controlled comparison: experiment report.
Real-data evaluation: UCI retail and bike workflow experiments, with replay commands and DSH prompts. These evaluate workflow safeguards and reproducible delivery; they do not establish better prediction accuracy or Agent productivity.
Versions
| Latest version | Published | Size |
|---|---|---|
| 0.4.0 | — | — |
Comments
Loading…
Similar plugins
by CDeZT
Native DSH foundation bundle that registers a persistent default workspace, provides paged read-only inspection of plugins, skills, memory, sessions, storage, settings, credentials, and DSH_HOME files
★ 0
MIT
JavaScript
Sep 2, 2026
dsh plugin --profile web add better-basicfunby onclaw-dev
`dsh-workflow-designer` 是面向 DeepSeek Harness 的提示词优先工作流设计插件。它收集当前 Agent 可见的本地 Skill、原生 Tool 与 MCP Tool,提供必要的可视化编排和约束编辑能力,并导出稳定的 YAML 中间表示。后续可由大模型及对应框架把 YAML 生成 Python、`workflow.mjs` 或其他代码工作流。
★ 0
↓ 198/wk
AGPL-3.0
JavaScript
Sep 10, 2026
dsh plugin --profile web add dsh-workflow-designerby dingguangyi0
Migrate WorkBuddy sessions, memories, MCP drafts, and automation tasks into DSH — preview-first, read-only on the source
★ 0
TypeScript
Sep 12, 2026
dsh plugin --profile web add dsh-plugin-migration-workbuddyA local workbench with model metrics, cost estimates, prompt presets, and optional request auditing for DeepSeek Harness.
★ 0
MIT
JavaScript
dsh plugin --profile web add dsh-atelier-pluginby xiangshu2026
Scheduled report generator for DeepSeek Harness: computer status, AI model news, plugin update check and a DSH plugin weekly digest, rendered to Word. Windows batch + PowerShell + Python, privacy-redacted by default.
★ 1
MIT
Python
Nov 3, 2026
dsh plugin --profile web add dsh-scheduled-reportsby 58catmiao
dsh-sciwork 把「科研模式」装进 DeepSeek Harness:一个落在自己硬盘上的科研库、一套技能与一块面板。课题、图、草稿、文献、数据表、期刊档案、投稿台账各有其位,写入都经库脚本落盘并登记,面板只读为主、自己不出图。技能把「怎么画、怎么写、怎么引」固化成纪律:出图交给 Origin 链路,写作、引用、投稿格式各有专门技能,外加 12 份期刊档案兜底。零 npm 依赖,路径运行时解析,换台机器照样跑。
★ 4
MIT
JavaScript
Nov 14, 2026
dsh plugin --profile web add dsh-sciwork