dsh-eval-harness avatar

dsh-eval-harness

DSH 插件评测工具:YAML 用例驱动真实 agent 回归评测 + baseline 对比 PASS/WARN/FAIL 门禁|Regression eval harness for DeepSeek Harness plugins

PluginDeveloper
Verification
L4 · Install tested
Failed
Risk
Low
Health
Active
Trust
Silver

Install verification: sandbox install ok, web smoke failed (L5) · tested on dsh 0.1.7-rc.2 · last checked Install verification details →

What it doesAI

YAML-case-driven regression eval harness for DeepSeek Harness plugins with baseline comparison and PASS/WARN/FAIL gates.

  • YAML use-case driven evaluation
  • Baseline comparison PASS/WARN/FAIL
  • For DeepSeek Harness plugins
developerAI

Capability tags are AI-inferred or rule-extracted from the README (see badge) — vocabulary-limited, not install-verified. AI = inferred; README = literal keyword from the readme.

Installation

dsh plugin --profile web add github:BiBoyang/dsh-eval-harness#v0.6.0

security scan passed · source release v0.6.0

Install method: GitHub · sandbox install ok, web smoke failed (L5)

Verification & Compatibility

Grouped by plugin version: each card lists the dsh versions actually tested against that artifact version and their install verdicts (L5 wins). Untested combinations are omitted — absence means untested, never assumed compatible. L1–L3 are a plugin-level static archive (independent of the dsh version — the same verdict on every dsh row): they sit in each row's level grid next to L4/L5, marked "static", while their evidence is stored once per plugin inside that row's "checks & evidence" instead of being duplicated per version.

External credentials (plugin-level): none detected

Install compatibility matrix (rows = plugin versions〔latest 5〕, columns = dsh versions〔latest 5〕)
At-a-glance → evidence in the cards below
plugin \ dshdsh 0.1.7-rc.2
current
dsh 0.1.7-rc.1dsh 0.1.7-alpha.2dsh 0.1.6-alpha.2dsh 0.1.6-alpha.1
v0.6.0·····
v0.4.0✗·✗✗✗
Install compatibility (by plugin version, dsh versions within)
Legend:✓ L5 runtime verified○ ecosystem plugin◼ ecosystem app✗ failed
v0.6.0github2026-09-28

https://github.com/BiBoyang/dsh-eval-harness

No runtime install test for this version (static scan only) — compatibility unknown, never assumed.

v0.4.0github7 failed2026-09-25

https://github.com/BiBoyang/dsh-eval-harness

✗dsh 0.1.7-rc.2current2026-09-25L5 · failed
Environment & artifact
  • • Node.js: v24.14.0
  • • DSH: 0.1.7-rc.2
  • • Plugin artifact: github · https://github.com/BiBoyang/dsh-eval-harness#v0.4.0
Levels (L1–L3 plugin-level static archive · L4/L5 newest real verdicts for this combination)
L1 static
Passed
L2 static
Passed
L3 static
Passed
L4 sandbox
Passed
L5 runtime
Failed
Sandbox install failed
L5 · failed

ID: dsh-eval-harness@v0.4.0@dsh0.1.7-rc.2 · profile: l4-sandbox

Issued: 2026-09-25 · expires: 2026-10-02

View checks & evidence
L1 · Found✓ Passed
Defaulted from L3✓ Passed
defaulted to passed — L3 install spec passed (full-registry L3 sweep policy)
L2 · Structured✓ Passed
Defaulted from L3✓ Passed
defaulted to passed — L3 install spec passed (full-registry L3 sweep policy)
L3 · Install spec✓ Passed
Install command parsed✓ Passed
Source: registry verification · Method: install spec parse · Captured 2026-09-28
DSH version declared✓ Passed
Source: registry verification · Method: DSH version declaration · Captured 2026-09-28
L4 · Install tested✓ Passed
Install executed in confined sandbox✓ Passed
Source: Windows ACL sandbox · Method: sandbox install run · Captured 2026-09-25
spec github:BiBoyang/dsh-eval-harness#v0.4.0 · channel GitHub source · outcome ran · exit 0
check took: 3.8s
Install succeeded✓ Passed
Source: Windows ACL sandbox · Method: sandbox install run · Captured 2026-09-25
spec github:BiBoyang/dsh-eval-harness#v0.4.0 · channel GitHub source · outcome ran · exit 0
check took: 3.8s
Installed artifact confirmed✓ Passed
Source: Windows ACL sandbox · Method: sandbox install run · Captured 2026-09-25
spec github:BiBoyang/dsh-eval-harness#v0.4.0 · channel GitHub source · outcome ran · exit 0
check took: 3.8s
L5 · Run tested✕ Failed
Sandbox install passed✓ Passed
Source: Windows ACL sandbox · Method: sandbox install run · Captured 2026-09-25
spec github:BiBoyang/dsh-eval-harness#v0.4.0 · install mode reused L4 scratch
check took: 0ms
Web boot ready✕ Failed
plugin tree failed to load
HTTP endpoint served✕ Failed

L1–L3 are plugin-level static conclusions (baseline, identical across dsh versions): evidence is stored once per plugin.

✗dsh 0.1.7-alpha.22026-09-23L5 · failed
Environment & artifact
  • • Node.js: v24.14.0
  • • DSH: 0.1.7-alpha.2
  • • Plugin artifact: github · https://github.com/BiBoyang/dsh-eval-harness#v0.4.0
Levels (L1–L3 plugin-level static archive · L4/L5 newest real verdicts for this combination)
L1 static
Passed
L2 static
Passed
L3 static
Passed
L4 sandbox
Passed
L5 runtime
Failed
Sandbox install failed
L5 · failed

ID: dsh-eval-harness@v0.4.0@dsh0.1.7-alpha.2 · profile: l4-sandbox

Issued: 2026-09-23 · expires: 2026-09-30

View checks & evidence
L1 · Found✓ Passed
Defaulted from L3✓ Passed
defaulted to passed — L3 install spec passed (full-registry L3 sweep policy)
L2 · Structured✓ Passed
Defaulted from L3✓ Passed
defaulted to passed — L3 install spec passed (full-registry L3 sweep policy)
L3 · Install spec✓ Passed
Install command parsed✓ Passed
Source: registry verification · Method: install spec parse · Captured 2026-09-28
DSH version declared✓ Passed
Source: registry verification · Method: DSH version declaration · Captured 2026-09-28
L4 · Install tested✓ Passed
Install executed in confined sandbox✓ Passed
Source: Windows ACL sandbox · Method: sandbox install run · Captured 2026-09-23
spec github:BiBoyang/dsh-eval-harness#v0.4.0 · channel GitHub source · outcome ran · exit 0
check took: 5.2s
Install succeeded✓ Passed
Source: Windows ACL sandbox · Method: sandbox install run · Captured 2026-09-23
spec github:BiBoyang/dsh-eval-harness#v0.4.0 · channel GitHub source · outcome ran · exit 0
check took: 5.2s
Installed artifact confirmed✓ Passed
Source: Windows ACL sandbox · Method: sandbox install run · Captured 2026-09-23
spec github:BiBoyang/dsh-eval-harness#v0.4.0 · channel GitHub source · outcome ran · exit 0
check took: 5.2s
L5 · Run tested✕ Failed
Sandbox install passed✓ Passed
Source: Windows ACL sandbox · Method: sandbox install run · Captured 2026-09-23
spec github:BiBoyang/dsh-eval-harness#v0.4.0 · install mode reused L4 scratch
check took: 0ms
Web boot ready✕ Failed
plugin tree failed to load
HTTP endpoint served✕ Failed

L1–L3 are plugin-level static conclusions (baseline, identical across dsh versions): evidence is stored once per plugin.

✗dsh 0.1.6-alpha.22026-09-20L5 · failed
Environment & artifact
  • • Node.js: v24.14.0
  • • DSH: 0.1.6-alpha.2
  • • Plugin artifact: github · https://github.com/BiBoyang/dsh-eval-harness#v0.4.0
Levels (L1–L3 plugin-level static archive · L4/L5 newest real verdicts for this combination)
L1 static
Passed
L2 static
Passed
L3 static
Passed
L4 sandbox
Passed
L5 runtime
Failed
Sandbox install failed
L5 · failed

ID: dsh-eval-harness@v0.4.0@dsh0.1.6-alpha.2 · profile: l3-fullscan

Issued: 2026-09-20 · expires: 2026-09-27

View checks & evidence
L1 · Found✓ Passed
Defaulted from L3✓ Passed
defaulted to passed — L3 install spec passed (full-registry L3 sweep policy)
L2 · Structured✓ Passed
Defaulted from L3✓ Passed
defaulted to passed — L3 install spec passed (full-registry L3 sweep policy)
L3 · Install spec✓ Passed
Install command parsed✓ Passed
Source: registry verification · Method: install spec parse · Captured 2026-09-28
DSH version declared✓ Passed
Source: registry verification · Method: DSH version declaration · Captured 2026-09-28
L4 · Install tested✓ Passed
Install executed in confined sandbox✓ Passed
Source: Windows ACL sandbox · Method: sandbox install run · Captured 2026-09-19
spec github:BiBoyang/dsh-eval-harness#v0.4.0 · channel GitHub source · outcome ran · exit 0
check took: 7.2s
Install succeeded✓ Passed
Source: Windows ACL sandbox · Method: sandbox install run · Captured 2026-09-19
spec github:BiBoyang/dsh-eval-harness#v0.4.0 · channel GitHub source · outcome ran · exit 0
check took: 7.2s
Installed artifact confirmed✓ Passed
Source: Windows ACL sandbox · Method: sandbox install run · Captured 2026-09-19
spec github:BiBoyang/dsh-eval-harness#v0.4.0 · channel GitHub source · outcome ran · exit 0
check took: 7.2s
L5 · Run tested✕ Failed
Sandbox install passed✓ Passed
Source: Windows ACL sandbox · Method: sandbox install run · Captured 2026-09-19
spec github:BiBoyang/dsh-eval-harness#v0.4.0 · install mode reused L4 scratch
check took: 0ms
Web boot ready✕ Failed
plugin tree failed to load
HTTP endpoint served✕ Failed

L1–L3 are plugin-level static conclusions (baseline, identical across dsh versions): evidence is stored once per plugin.

✗dsh 0.1.6-alpha.12026-09-17L5 · failed
Environment & artifact
  • • Node.js: v24.14.0
  • • DSH: 0.1.6-alpha.1
  • • Plugin artifact: github · https://github.com/BiBoyang/dsh-eval-harness#v0.4.0
Levels (L1–L3 plugin-level static archive · L4/L5 newest real verdicts for this combination)
L1 static
Passed
L2 static
Passed
L3 static
Passed
L4 sandbox
Passed
L5 runtime
Failed
Sandbox install failed
L5 · failed

ID: dsh-eval-harness@v0.4.0@dsh0.1.6-alpha.1 · profile: l4-sandbox

Issued: 2026-09-17 · expires: 2026-09-24

View checks & evidence
L1 · Found✓ Passed
Defaulted from L3✓ Passed
defaulted to passed — L3 install spec passed (full-registry L3 sweep policy)
L2 · Structured✓ Passed
Defaulted from L3✓ Passed
defaulted to passed — L3 install spec passed (full-registry L3 sweep policy)
L3 · Install spec✓ Passed
Install command parsed✓ Passed
Source: registry verification · Method: install spec parse · Captured 2026-09-28
DSH version declared✓ Passed
Source: registry verification · Method: DSH version declaration · Captured 2026-09-28
L4 · Install tested✓ Passed
Install executed in confined sandbox✓ Passed
Source: Windows ACL sandbox · Method: sandbox install run · Captured 2026-09-17
spec github:BiBoyang/dsh-eval-harness#v0.4.0 · channel GitHub source · outcome ran · exit 0
check took: 4.3s
Install succeeded✓ Passed
Source: Windows ACL sandbox · Method: sandbox install run · Captured 2026-09-17
spec github:BiBoyang/dsh-eval-harness#v0.4.0 · channel GitHub source · outcome ran · exit 0
check took: 4.3s
Installed artifact confirmed✓ Passed
Source: Windows ACL sandbox · Method: sandbox install run · Captured 2026-09-17
spec github:BiBoyang/dsh-eval-harness#v0.4.0 · channel GitHub source · outcome ran · exit 0
check took: 4.3s
L5 · Run tested✕ Failed
Sandbox install passed✓ Passed
Source: Windows ACL sandbox · Method: sandbox install run · Captured 2026-09-17
spec github:BiBoyang/dsh-eval-harness#v0.4.0 · install mode reused L4 scratch
check took: 0ms
Web boot ready✕ Failed
plugin tree failed to load
HTTP endpoint served✕ Failed

L1–L3 are plugin-level static conclusions (baseline, identical across dsh versions): evidence is stored once per plugin.

✗dsh 0.1.5-rc.22026-09-11L5 · failed
Environment & artifact
  • • Node.js: v24.14.0
  • • DSH: 0.1.5-rc.2
  • • Plugin artifact: github · https://github.com/BiBoyang/dsh-eval-harness#v0.4.0
Levels (L1–L3 plugin-level static archive · L4/L5 newest real verdicts for this combination)
L1 static
Passed
L2 static
Passed
L3 static
Passed
L4 sandbox
Passed
L5 runtime
Failed
Sandbox install failed
L5 · failed

ID: dsh-eval-harness@v0.4.0@dsh0.1.5-rc.2 · profile: l4-sandbox

Issued: 2026-09-11 · expires: 2026-09-18

View checks & evidence
L1 · Found✓ Passed
Defaulted from L3✓ Passed
defaulted to passed — L3 install spec passed (full-registry L3 sweep policy)
L2 · Structured✓ Passed
Defaulted from L3✓ Passed
defaulted to passed — L3 install spec passed (full-registry L3 sweep policy)
L3 · Install spec✓ Passed
Install command parsed✓ Passed
Source: registry verification · Method: install spec parse · Captured 2026-09-28
DSH version declared✓ Passed
Source: registry verification · Method: DSH version declaration · Captured 2026-09-28
L4 · Install tested✓ Passed
Install executed in confined sandbox✓ Passed
Source: Windows ACL sandbox · Method: sandbox install run · Captured 2026-09-11
spec github:BiBoyang/dsh-eval-harness#v0.4.0 · channel GitHub source · outcome ran · exit 0
check took: 7.0s
Install succeeded✓ Passed
Source: Windows ACL sandbox · Method: sandbox install run · Captured 2026-09-11
spec github:BiBoyang/dsh-eval-harness#v0.4.0 · channel GitHub source · outcome ran · exit 0
check took: 7.0s
Installed artifact confirmed✓ Passed
Source: Windows ACL sandbox · Method: sandbox install run · Captured 2026-09-11
spec github:BiBoyang/dsh-eval-harness#v0.4.0 · channel GitHub source · outcome ran · exit 0
check took: 7.0s
L5 · Run tested✕ Failed
Sandbox install passed✓ Passed
Source: Windows ACL sandbox · Method: sandbox install run · Captured 2026-09-11
spec github:BiBoyang/dsh-eval-harness#v0.4.0 · install mode reused L4 scratch
check took: 0ms
Web boot ready✕ Failed
plugin tree failed to load
HTTP endpoint served✕ Failed

L1–L3 are plugin-level static conclusions (baseline, identical across dsh versions): evidence is stored once per plugin.

✗dsh 0.1.3-alpha.22026-09-09L5 · failed
Environment & artifact
  • • Node.js: v24.14.0
  • • DSH: 0.1.3-alpha.2
  • • Plugin artifact: github · https://github.com/BiBoyang/dsh-eval-harness#v0.4.0
Levels (L1–L3 plugin-level static archive · L4/L5 newest real verdicts for this combination)
L1 static
Passed
L2 static
Passed
L3 static
Passed
L4 sandbox
Passed
L5 runtime
Failed
Sandbox install failed
L5 · failed

ID: dsh-eval-harness@v0.4.0@dsh0.1.3-alpha.2 · profile: l4-sandbox

Issued: 2026-09-09 · expires: 2026-09-16

View checks & evidence
L1 · Found✓ Passed
Defaulted from L3✓ Passed
defaulted to passed — L3 install spec passed (full-registry L3 sweep policy)
L2 · Structured✓ Passed
Defaulted from L3✓ Passed
defaulted to passed — L3 install spec passed (full-registry L3 sweep policy)
L3 · Install spec✓ Passed
Install command parsed✓ Passed
Source: registry verification · Method: install spec parse · Captured 2026-09-28
DSH version declared✓ Passed
Source: registry verification · Method: DSH version declaration · Captured 2026-09-28
L4 · Install tested✓ Passed
Install executed in confined sandbox✓ Passed
Source: Windows ACL sandbox · Method: sandbox install run · Captured 2026-09-09
spec github:BiBoyang/dsh-eval-harness#v0.4.0 · channel GitHub source · outcome ran · exit 0
check took: 17.0s
Install succeeded✓ Passed
Source: Windows ACL sandbox · Method: sandbox install run · Captured 2026-09-09
spec github:BiBoyang/dsh-eval-harness#v0.4.0 · channel GitHub source · outcome ran · exit 0
check took: 17.0s
Installed artifact confirmed✓ Passed
Source: Windows ACL sandbox · Method: sandbox install run · Captured 2026-09-09
spec github:BiBoyang/dsh-eval-harness#v0.4.0 · channel GitHub source · outcome ran · exit 0
check took: 17.0s
L5 · Run tested✕ Failed
Sandbox install passed✓ Passed
Source: Windows ACL sandbox · Method: sandbox install run · Captured 2026-09-09
spec github:BiBoyang/dsh-eval-harness#v0.4.0 · install mode reused L4 scratch
check took: 0ms
Web boot ready✕ Failed
plugin tree failed to load
HTTP endpoint served✕ Failed

L1–L3 are plugin-level static conclusions (baseline, identical across dsh versions): evidence is stored once per plugin.

✗dsh 0.1.3-alpha.12026-09-07L5 · failed
Environment & artifact
  • • Node.js: v24.14.0
  • • DSH: 0.1.3-alpha.1
  • • Plugin artifact: github · https://github.com/BiBoyang/dsh-eval-harness#v0.4.0
Levels (L1–L3 plugin-level static archive · L4/L5 newest real verdicts for this combination)
L1 static
Passed
L2 static
Passed
L3 static
Passed
L4 sandbox
Passed
L5 runtime
Failed
Sandbox install failed
L5 · failed

ID: dsh-eval-harness@v0.4.0@dsh0.1.3-alpha.1 · profile: l3-fullscan

Issued: 2026-09-07 · expires: 2026-09-14

View checks & evidence
L1 · Found✓ Passed
Defaulted from L3✓ Passed
defaulted to passed — L3 install spec passed (full-registry L3 sweep policy)
L2 · Structured✓ Passed
Defaulted from L3✓ Passed
defaulted to passed — L3 install spec passed (full-registry L3 sweep policy)
L3 · Install spec✓ Passed
Install command parsed✓ Passed
Source: registry verification · Method: install spec parse · Captured 2026-09-28
DSH version declared✓ Passed
Source: registry verification · Method: DSH version declaration · Captured 2026-09-28
L4 · Install tested✓ Passed
Install executed in confined sandbox✓ Passed
Source: Windows ACL sandbox · Method: sandbox install run · Captured 2026-09-06
spec github:BiBoyang/dsh-eval-harness#v0.4.0 · channel GitHub source · outcome ran · exit 0
check took: 4.2s
Install succeeded✓ Passed
Source: Windows ACL sandbox · Method: sandbox install run · Captured 2026-09-06
spec github:BiBoyang/dsh-eval-harness#v0.4.0 · channel GitHub source · outcome ran · exit 0
check took: 4.2s
Installed artifact confirmed✓ Passed
Source: Windows ACL sandbox · Method: sandbox install run · Captured 2026-09-06
spec github:BiBoyang/dsh-eval-harness#v0.4.0 · channel GitHub source · outcome ran · exit 0
check took: 4.2s
L5 · Run tested✕ Failed
Sandbox install passed✓ Passed
Source: Windows ACL sandbox · Method: sandbox install run · Captured 2026-09-06
spec github:BiBoyang/dsh-eval-harness#v0.4.0 · install mode reused L4 scratch
check took: 0ms
Web boot ready✕ Failed
plugin tree failed to load
HTTP endpoint served✕ Failed

L1–L3 are plugin-level static conclusions (baseline, identical across dsh versions): evidence is stored once per plugin.

Only the latest 3 published versions are shown; source-snapshot (commit hash) groups, unstated groups and older versions are omitted — the full archive stays in the data layer (snapshot/unstated groups appear only when a plugin has no published version).

Author badges

Embed the official badges in your README to showcase security & install-test status:

dsh.so risk
`[![dsh.so risk](https://www.dsh.so/badge/dsh-eval-harness.svg)](https://www.dsh.so/artifact/dsh-eval-harness/)`

Security Report

Automated static scan, not manual review.

DSH.SO VETPASSED1923ab
Automated review · daily
PASSED

Vet static analysis found no suspicious behavior — verdict is clean.

0 critical·0 high·0 medium
plugin versionv0.6.0dsh manifest
scanned versionv0.6.0git2026-09-27
current versionv0.6.0same version
vet verdictclean · git

Vet is an AST static scan (npm artifact or git source).

Disclaimer: automated static analysis, not a security guarantee. Always review what you install.

Risk details

No critical or warning findings.

Activity

Last commit 2026-09-26 · activity: Active

• Repo created: 2026-08-13

• Stars: ★ 13 · Forks: 1

• Health: Active — committed within last 30 days

Source

GitHub: github.com/BiBoyang/dsh-eval-harness

Was this page helpful?