DeepSeek Harness Plugin Install Ecosystem Report — Vol.1
Volume 1 of the install track. A living report on how the dsh.so install verification ladder (L1–L5) is landing across the public plugin registry. Unlike the frozen Ecosystem Report Vol.1, every figure on this page is re-aggregated at build time from append-only sandbox test records — the report advances by itself as the daily ladder sweeps land. The always-current table view lives at /verification/.
Executive summary
The registry lists 12,921 plugins. Install verification has reached 12,767 of them on the current dsh version (98.8% coverage), and 5,543 plugins — 42.9% of the registry — have passed the full runtime gate (L5): sandboxed install, web boot, HTTP readiness, and an active plugin inventory. 2,493 failed the install test and are listed with per-plugin links; 15 carry verdicts only from older dsh versions and wait in line for retest. Below the runtime gates, the static funnel is much wider: 12,936 plugins have a parseable, version-pinned install specification (L3). Failure and queue position are kept strictly apart — untested is never reported as failing.
Headline numbers
Install funnel L1–L5
Every registry entry climbs the same ladder: three static gates first, two runtime gates last. L1 Found → L2 Structured → L3 Install spec → L4 Install tested → L5 Runtime verified. L1–L3 are version-free registry checks; L4/L5 are sandbox runtime tests whose verdicts are scoped to the dsh version they were tested with.
| Level | Passed | Failed | Unknown | Not tested | Stale | Pass rate |
|---|---|---|---|---|---|---|
| L1Found | 13,009 | 0 | 0 | 312 | — | |
| L2Structured | 12,167 | 75 | 767 | 312 | — | |
| L3Install spec | 12,936 | 0 | 291 | 94 | — | |
| L4Install tested | 12,040 | 122 | 613 | 320 | 226 | |
| L5Runtime verified | 5,543 | 2,391 | 4,173 | 1,214 | — |
Scope: the 13,321 plugins carrying verification records. L4/L5 rows count only real verdicts on the current dsh version (0.1.2-alpha.1); their pass-rate denominators exclude stale entries. L1–L3 are static checks with no version scope. Not tested never means failed.
Verdicts by dsh version
| dsh version | Passed | of which L5 verified | Failed | Unknown |
|---|---|---|---|---|
0.1.2-alpha.1current | 5,543 | 5,543 | 2,493 | 4,731 |
0.1.1-rc.2 | 7,398 | 0 | 523 | 62 |
pre-versioning | 31 | 4 | 2 | 5 |
Each dsh version keeps its verdicts independently — per plugin, the newest real verdict on that version wins (L5 over L4). Pre-versioning = records that predate version capture. A pass on an older version never counts as current.
Findings
Runtime verification has reached registry scale
Observation. 42.9% of the entire registry — 5,543 of 12,921 plugins — has passed the L5 runtime gate, and 12,767 plugins carry a runtime verdict on the current dsh version. Every verdict is derived from append-only sandbox records and is traceable per plugin through the install DATA API.
Registry plugins: 12,921 Runtime verdicts on current dsh: 12,767 (98.8% coverage) Runtime verified (L5 passed): 5,543 L4-only sandbox passes (not yet runtime-verified): 0 Failed: 2,493 · Inconclusive: 4,731 · Stale (older dsh): 15 As of: 2026-09-03
Scope. Runtime verdicts are facts about installability and loadability inside a constrained sandbox. They say nothing about code behavior beyond the smoke sequence itself: sandboxed install → web boot → HTTP readiness → plugin inventory active.
Limitation. Coverage describes how far instrumentation and queue order have reached, not ecosystem quality. An untested plugin is never a failing plugin. Figures move with every deploy; cite them together with the as-of date.
Two install channels, measurably different odds
Observation. npm-channel installs pass at 76.8% (1,084 of 1,412), while source-channel installs pass at 39.3% (4,450 of 11,328). Part of the gap tracks what the channel ladder freezes before testing: npm installs pin a published tarball; source installs build from a release tag or a commit.
npm channel: 1,084 passed / 1,412 tested (76.8%) · runtime verified 1,084 source channel: 4,450 passed / 11,328 tested (39.3%) · runtime verified 4,450 Unattributed (other): 27 As of: 2026-09-03
Scope. Channel attribution follows the channel evidence of the winning record, backtracked within the same dsh version when the winning L5 record itself carries none. Verdicts that still cannot be attributed are counted under other — they are never guessed.
Limitation. Pass rates are not a quality ranking of packaging. They mix repository health, dependency weight, and Windows sandbox specifics, and they move every day as sweeps land. Figures move with every deploy; cite them together with the as-of date.
Version-scoped verdicts are honest — and they dip on purpose
Observation. Every runtime verdict is valid only for the dsh version it was tested against. 15 plugins currently hold verdicts only from older dsh versions: retained as history, still displayed, and automatically re-queued for retest by the ladder. Headline numbers therefore dip right after each dsh release and recover as retests land.
Current dsh version: 0.1.2-alpha.1 Stale (older-dsh verdicts only): 15 dsh versions carrying verdicts: 2 Newest record environment: dsh-0.1-sandbox · win32 x64 · DSH CLI 0.1.2-alpha.1
Scope. Version scoping is deliberate. After an upgrade, older verdicts are neither silently upgraded nor silently deleted — they become stale and wait in the retest queue.
Limitation. A dip in the verified count after a dsh release is bookkeeping, not mass breakage; the ladder retest queue is the recovery path. Figures move with every deploy; cite them together with the as-of date.
Untested is not failing
Observation. 139 registry plugins have no runtime verdict yet. Most never reached the runtime ladder because no parseable install command exists (L3 not passed) or because they are queued behind newer and recently changed candidates. Failure is always explicit: 2,493 plugins are listed as failed, with per-plugin links, on this page and on the verification page.
Never runtime-tested: 139 Explicitly failed: 2,493 Inconclusive (unknown): 4,731 Install-spec (L3) passed: 12,936 of 13,321 plugins carrying records
Scope. Unknown means the run produced no usable verdict; it is never folded into pass or fail. Untested usually means no testable install spec — or a queue that has not reached the plugin yet.
Limitation. Queue order favors recently changed and never-tested candidates; a plugin’s age alone is not a quality signal. Figures move with every deploy; cite them together with the as-of date.
Methodology
- Levels. L1 Found — the repository is accessible, non-empty, and has a README. L2 Structured — manifest/package.json is valid with key fields. L3 Install spec — a parseable, version-pinned install command exists. L4 Install tested — the install command completes inside an isolated sandbox. L5 Runtime verified — the harness boots with the plugin: web ready, HTTP served, plugin inventory active.
- Verdict rule (L5 wins). On each dsh version, the newest real L5 verdict wins; versions without an L5 verdict fall back to their newest L4 verdict (sandbox install passed, not yet runtime-verified). An L5 pass is the only verdict the site counts as install verified; an L4-only pass displays as install passed.
- Version scoping. Runtime verdicts are valid only for the dsh version they were tested with. After an upgrade, older verdicts become stale — retained, displayed, re-queued — and stop counting toward current numbers.
- Honesty rules. Unknown stays unknown; untested is never reported as failed; environments are quoted from records as-is, or stated as not stated.
- Reproduce. Per plugin: GET /data/install/<id>.json (docs at /data/install/). Whole registry: /verification/. Methodology: /methodology/.
FAQ
What is the difference between L4 Install tested and L5 Runtime verified?
L4 proves the declared install command completes in a sandbox without dependency errors. L5 proves the installed plugin actually loads and runs: the harness boots, the web UI serves HTTP, and the plugin appears active in the inventory. Only an L5 pass counts as install verified; an L4-only pass shows as install passed with L5 pending.
Why did the verified count drop after a dsh release?
Runtime verdicts are version-scoped. When a new dsh version lands, verdicts made on older versions become stale — they stop counting toward current numbers but remain visible as history. The dip is bookkeeping, not mass breakage; the daily ladder retests queued plugins and the numbers recover.
How does a plugin become runtime-verified?
Nothing to submit. The ladder runs continuously: a plugin with a parseable install spec (L3) gets sandbox-installed (L4) and, on pass, smoke-tested at runtime (L5). Check any plugin via GET /data/install/<id>.json or the install badge on its detail page.
Cite this report
dsh.so — DeepSeek Harness Plugin Install Ecosystem Report, Vol.1 (living build). Built: 2026-09-03 · data as of 2026-09-03 · current dsh 0.1.2-alpha.1 Registry: 12,921 plugins · runtime tested 12,767 (98.8%) · runtime verified (L5) 5,543 Funnel: L3 passed 12,936 · L5 passed 5,543 · failed 2,493 · stale 15 https://dsh.so/reports/install-vol1/