What we know about specifications
SeedSpec’s evaluation program asks what explicit intent, guided authoring, semantic structure, and packaged domain knowledge actually change—and where the evidence stops.
The clearest conclusion so far
A real specification gives an agent a better starting opportunity than leaving important behavior implicit. Making product decisions explicit reduces the consequential policy an agent must infer or invent.
SeedSpec builds on that baseline. Its authoring workflow helps experts develop a fuller specification, its semantic roles make the result easier to inspect and adapt, and its package format lets organizations distribute an exact version.
What the current work indicates
Explicit intent reduces consequential invention
In the first authorship-gradient case, the zero-shot treatment made a material custody-policy deviation, omitted a durable disagreement state, and selected an unsupported time-zone policy. Treatments with explicit author answers preserved those boundaries more accurately.
Good Markdown is a strong baseline
Ordinary Markdown produced with strong general specification guidance captured the tested material boundaries well. Multiple files, a manifest, or schema validity do not automatically produce better product intent.
Validity is not specification quality
The minimal SeedSpec treatment passed deterministic checks while omitting material cancellation and overdue behavior and part of the requested accessibility evidence. A valid package can still be incomplete or weak.
Semantic authoring makes ownership clearer
Guided and restructured treatments more clearly separated product authority, adopter configuration, acceptance evidence, implementation guidance, open choices, and decision provenance.
Specialized guidance can carry missing knowledge
In a repeated warehouse-transfer experiment, relevant domain guidance materially improved obligation coverage and decision alignment. Embedding the same knowledge directly in the prompt was nearly as strong; the finding is about making expertise available and reliably consulted.
The specialized-guidance comparison
Five authoring treatments were repeated three times each for a synthetic warehouse-transfer contract. Package-scoped guidance carried domain obligations that a general model could not reliably recover without being given the knowledge.
shared observations with specialized guidance, compared with 14 / 36 without it
observations with specialized guidance, compared with 10 / 21 without it
Embedding the same domain content directly in the instructions was nearly as strong. Packaging preserves reuse, attribution, independent versioning, and an inspectable consultation record.
Evidence table
| Finding | Evidence strength | Responsible interpretation |
|---|---|---|
| Explicit decisions reduce material agent invention. | Observed in the authorship-gradient case. | A real specification gives the agent a better starting opportunity. |
| General Markdown can be excellent. | Observed directly in the same comparison. | Structure must add authoring, reuse, provenance, or handoff value. |
| A valid SeedSpec can still have material gaps. | Observed directly in the minimal treatment. | Protocol validity and specification quality are separate results. |
| Guided semantic authoring improves explicit ownership. | Observed in guided and restructured treatments. | The authoring system is a first-class part of SeedSpec’s value. |
| Packaged domain guidance can improve coverage. | Repeated in one synthetic specialized domain. | Broader replication is still required. |
What these results do not establish
- They do not show that every SeedSpec outperforms every Markdown specification.
- They do not support a universal percentage-complete or prompt-savings claim.
- They do not establish that authorship gains always produce stronger implementations.
- They do not turn one model, case, or environment into a general performance guarantee.
Implementation congruency across matched agents, models, tools, and environments is the next important evidence boundary.
Open evaluation program
The cases, runner, schemas, methods, and operating instructions are developed in public and separately from protocol conformance so the evidence can evolve without becoming a package requirement.