Don’t trust the chart. Run the contract.
A one-command, independently operated path for reproducing the pinned mem0 2.0.12 versus mem0 + Vault governance result—with blinded provider input, five fresh stores, environment evidence, and fail-closed review.
The protocol is open. External evidence is still pending.
No third-party reproduction has been accepted yet. This page is infrastructure for verification, not a claim that independent validation already exists. Blocked environments use a separate diagnostic report and never enter the submission count.
Frozen mem0 track
Provider 2.0.12, six public synthetic cases, top-1, candidate pool four, five sequential repeats.
Fail-closed validation
Mutation, unlisted files, unsafe paths, dirty source, false attestations, and failed release gates are rejected.
Independent reproductions
Missing evidence remains visibly missing. Contract-valid is not maintainer-reviewed or endorsed.
From clean fork to public review.
Clean isolated environment
Fork a fixed revision and install the pinned mem0 requirements.
Five blinded repeats
Each provider process sees redacted input and gets a fresh local store.
Checksums + attribution
CI validates the contract; maintainers separately inspect provenance and conflicts.
Fail early with a machine-readable readiness check.
python scripts/run_external_reproduction.py \ --output-dir /tmp/vault-external-reproduction \ --model-cache-dir ../vault-repro-model-cache \ --preflight-only --json
One check covers clean source, external venv/output, pinned packages, disk, memory, model cache, and Hugging Face access. The full run then verifies immutable Hub commit and snapshot-tree digests before forcing all repeats offline. Exit 2 lists every blocker; pass means readiness only.
Preflight pass
Environment readiness—not a benchmark or third-party reproduction result.
Contract validated
The frozen synthetic retrieval-governance contract reproduced in the disclosed environment; still not universal superiority, production scale, or end-to-end QA.
Validation and endorsement stay separate.
| State | Meaning | Public claim |
|---|---|---|
| Environment blocked | Network, runtime, storage, memory, or platform constraint prevented five repeats | Diagnostic only; no reproduction claim |
| Submitted | A public bundle URL exists | No technical claim |
| Contract validated | Digests, protocol, attestations, and gates pass | Machine-valid bundle |
| Maintainer reviewed | Provenance, conflicts, and deviations inspected | Reviewed reproduction |
| Published external reproduction | Immutable record linked with attribution | Independent result under disclosed scope |
Be the first independent operator.
Run the public contract, disclose the environment, and let the artifacts—not marketing copy—carry the result.