Conformance

What the SeedSpec conformance suite covers, how to run it, and what a conformant result does and does not establish.

Running the suite

The corpus ships inside @seedspec/protocol and is bound to an exact release. With no arguments the CLI runs the bundled corpus for the protocol version it was published against.

npx @seedspec/cli conformance npx @seedspec/cli conformance --json --output conformance-report.json npx @seedspec/cli doctor --full

The report names the exact protocol release, the suite version, and the digest of the corpus that produced it. A report without that binding is not evidence of anything in particular.

What a conformant result establishes

A package structure and its semantics can be interpreted by an independent tool at the declared protocol version.
Package digests are stable and content-sensitive, so the same bytes always identify the same package.
Resolution produces deterministic output for fixed inputs, byte for byte.
Declared failures fail. Invalid identifiers, unsafe paths, unsupported versions, and broken references are rejected rather than tolerated.

What it does not establish

That the seed is complete, clear, or worth adopting.
That an agent implementing the package will produce correct software.
That the publisher is who they claim to be.
That the package is safe to execute — nothing in a package is executed by conformant tooling.
That the package will compose cleanly with any particular host.

This separation is deliberate. Conformance is strict precisely because it claims something narrow: that independent tools agree on what a package means. Whether the package is good is an authoring question, and whether an implementation is right is an evidence question. Neither is settled here.

Sources

Conformance corpus — the cases themselves.
Conformance documentation — how cases are structured and extended.
Versioned schemas — what the cases validate against.
Evaluation findings — separate work on whether specifications improve outcomes at all.