Ship What You Can Name: The AI Release Passport

An AI service can pass its evaluation on Friday and become a different system on Monday without a conventional application commit. A provider may move a model alias. A retrieval index may be rebuilt from a changed corpus. A system prompt, safety threshold, tool schema, adapter, tokenizer, or routing rule may change independently. The interface still looks familiar, but the decision path is not the one that was approved.

That creates a concrete release question: can the team identify the exact assembly that is about to serve users, verify where its consequential parts came from, compare it with the approved assembly, and restore the previous one? If the answer depends on several people reconstructing a deployment from dashboards and memory, the release is not under control.

A software bill of materials is part of the answer, but AI needs a wider release record. This guide calls it an AI release passport: a versioned component graph bound to immutable identifiers, decision evidence, and verifiable provenance. It does not claim the release is safe; it makes a responsible decision possible.

The release decision is larger than “which model?”

The model is only one dependency in a production behavior chain. Consider a support assistant that retrieves policy passages and can open a case. Its output depends on the hosted model, model parameters, system instructions, retrieval query, embedding model, corpus snapshot, chunking code, index configuration, tool descriptions, authorization policy, application code, and the external case-management service. The observed outcome also depends on live data and user input.

The passport should therefore answer four different questions:

1. Identity: What exact components and configurations constitute this release?
2. Origin: Who supplied or produced them, from which inputs and process?
3. Decision evidence: Which evaluations, scans, reviews, limitations, and accepted risks apply to this assembly?
4. Operational control: What is deployed, what has drifted, and what known-good assembly can replace it?

Security needs transitive dependencies and integrity; legal and procurement need suppliers, licenses, data rights, and change terms; model-risk owners need intended use and evaluation limits; operations needs compatible artifacts, owners, and rollback. Use a machine-readable core with role-specific views, not one enormous PDF.

What the standards say—and where design judgment begins

Several current sources support the building blocks, but none removes the need to define the release boundary for a particular product.

NIST’s July 2024 SP 800-218A adapts secure software-development practices to generative AI and foundation models. It recommends sharing provenance for release components through mechanisms such as an SBOM or SLSA, tracking the lineage of a model and its training libraries, frameworks, and pipelines, and recording when data provenance is unknown. This is secure-development guidance, not a certification of any release.

The SPDX 3.0.1 AI Profile defines exchange concepts for AI packages, models, software, and datasets. The June 2026 CycloneDX Authoritative Guide to AI/ML-BOM shows another machine-readable approach covering identifiers, datasets, tokenizers, prompt templates, frameworks, training and testing, intended use, and limitations. Either aids interoperability; neither decides your product boundary.

The UK NCSC’s joint secure AI development guidance makes the operational scope explicit: organisations should track and protect models, data, prompts, software, documentation, logs, and assessments, use hashes or signatures where appropriate, and retain the ability to restore a known-good state. Current SLSA 1.2 adds a framework for increasingly trustworthy source and build provenance.

One distinction is especially important. The SLSA 1.2 FAQ explains that an SBOM usually gives fine-grained information about components present in an artifact, while SLSA provenance describes the build process, external parameters, and inputs at a coarser level. They overlap, but neither substitutes for the other.

Verified fact: these sources provide compatible ideas for inventory, lineage, build evidence, and secure lifecycle records. ZharfAI analysis: a deployer should bind those ideas into one release-level control object that also covers behavioral configuration, external tools, evaluation evidence, runtime drift, and rollback. That proposed passport is an implementation pattern, not a new standard.

Define the release unit before building the inventory

Begin with the thing that can actually be promoted or rolled back. “Assistant v3” is too vague; “model file” is too narrow. A useful release unit is an immutable manifest identified by a digest, such as:

product: multilingual-policy-assistant
release_id: sha256:<digest of canonical manifest>
environment_eligibility: staging, production
created_at: <timestamp>
manifest_schema: <name and version>
previous_release: sha256:<digest>

The manifest points to components rather than embedding every document or secret. Each component gets a stable identifier, version or digest, supplier or producer, owner, location, sensitivity, lifecycle status, and verification state. Relationships connect the graph. Evidence records refer back to the release digest so that an approval cannot silently migrate to a changed assembly.

Choose granularity by consequence. Identify a behavior-changing tokenizer separately; identify both a retrieval corpus and its built index because their origins and change rates differ. Cosmetic configuration can remain inside the application artifact. Write this rule down, or tools will choose the inventory boundary by accident.

Declare exclusions too. Live inputs are unknown at build time, hosted APIs may hide weight hashes, and confidential data may be undistributable. Record the supplier reference, access class, and verification limitation instead of inventing precision. Unknown provenance is a state to manage.

Use a six-layer passport

The following layers cover both self-hosted models and applications built on external model APIs.

Do not place personal data, credentials, hidden prompts, or restricted weights in a broadly shared BOM. Inventory is not disclosure. Store a content-addressed reference, classification, custodian, retention rule, and controlled evidence link. Synthetic-data governance applies the same principle when underlying records require restricted access.

Bind names to immutable evidence

A component name is useful to people; a digest is useful to a verifier. Keep both. For files, containers, indexes, prompts, policy bundles, and evaluation sets, calculate a cryptographic digest over canonical bytes. For source, use the full commit identifier, not a movable branch. For packages, capture an ecosystem identifier, exact version, source repository, and lockfile resolution. Preserve the algorithm with the digest.

Hosted services require honest fallbacks. If a provider exposes an immutable deployment or model version, record it. If it exposes only a moving alias, record the provider, endpoint, alias, observed response metadata, contractual update policy, observation time, and the fact that byte-level verification is unavailable. That limitation can trigger a stronger regression gate or make the service ineligible for a high-consequence use. Do not pretend that an API marketing name is a reproducible artifact.

Canonicalisation matters: equivalent JSON, changed line endings, or nondeterministic index timestamps produce different bytes. Define a canonical form, separate material inputs from incidental metadata, and test repeatability. Otherwise comparison and rollback become noisy.

Make relationships first-class

A flat component list cannot answer impact questions. Relationships should be typed and directional. Useful examples include:

release CONTAINS application
application LOADS model
model FINE_TUNED_FROM base_model
model TRAINED_ON dataset_snapshot
index GENERATED_FROM corpus_snapshot
index GENERATED_BY index_builder
assistant USES system_prompt
assistant CAN_INVOKE tool_schema
tool_schema ENFORCED_BY authorization_policy
evaluation REPORTS_ON release
approval AUTHORIZES release
deployment RUNS release

Add owner, environment, effective time, and evidence reference where they affect meaning. Then a newly disclosed library vulnerability can identify candidate releases; a corpus-rights issue can identify derived indexes; a changed tool schema can identify evaluations that are no longer representative.

Relationships expose shared fate: two “independent” fallbacks may share a provider, GPU region, tokenizer library, or identity service. The AI vendor dossier collects upstream evidence and change rights; the passport binds each accepted supplier component to deployed assemblies.

Attest how the passport and artifacts were produced

A BOM is a claim. If it is generated after deployment from a developer laptop, it may omit the compromised input that matters most. Generate inventory and provenance inside the controlled build and release path. The trusted control plane should resolve inputs, calculate digests, record the builder and process, emit the passport, and sign or otherwise attest to the resulting subjects. Store the attestation separately from the artifact so both can be verified.

At promotion time, a policy engine should check more than signature validity:

• the signer or builder identity is trusted for this product;
• the release digest and artifact subjects match the candidate bytes;
• required component classes are present;
• no prohibited moving identifiers remain;
• unknown provenance is below the use-case threshold or explicitly accepted;
• linked evaluations report on this exact release digest;
• approvals are current, scoped, and issued by authorised roles;
• security and license policies have terminal results;
• the previous known-good release remains deployable.

A valid signature proves control of a signing identity over a statement—not model accuracy, lawful data collection, benign dependencies, or an uncompromised builder. Those need separate evidence and trust roots. The audit-ready AI evidence trail keeps such assertions reviewable without making one log a universal source of truth.

Worked example: upgrading a multilingual RAG assistant

Suppose a team wants to replace the hosted generation model and refresh the Persian policy corpus. The candidate also changes the embedding model because the old one handles colloquial queries poorly. A conventional ticket might say “upgrade model and knowledge base.” The passport shows at least five coupled changes: generation behavior, embedding geometry, corpus content, index bytes, and evaluation applicability.

The team should not “roll back the model” while leaving an incompatible index. Rollback targets the previous release manifest, restoring model, embedding, corpus, index, prompts, policies, and application together. The runbook must distinguish reversible components from external state needing compensation or migration.

The comparison also prevents unnecessary testing. Unchanged tool and authorization digests may need only affected tests plus a protected end-to-end baseline. The passport becomes a change-impact input, not archive paperwork.

Put the passport in the promotion gate

Before a candidate reaches production, require a machine-verifiable result for each item:

1. The release has one immutable digest and a link to its previous known-good release.
2. Mandatory component classes and relationships are complete for the product’s risk tier.
3. Every consequential component has an immutable identity or a declared verification limitation.
4. Suppliers, owners, licenses, sensitivity, locations, and lifecycle status are recorded where applicable.
5. Digests resolve to available artifacts or controlled evidence records.
6. Evaluation, security, privacy, rights, and operational evidence refers to the candidate release—not just the model family.
7. Moving aliases, unknown provenance, exceptions, and expired evidence meet explicit policy.
8. Build and source attestations verify against trusted identities and expected process properties.
9. The deployed environment reports the same release digest after activation.
10. Rollback restores the whole compatible assembly and has been rehearsed within its allowed age.

The production AI readiness checklist still applies. A passport cannot supply an absent owner, fallback, incident plan, or useful evaluation; it reveals those gaps and binds existing controls to the release they cover.

Avoid inventory theatre

Several failure modes create paperwork without control:

• Generate the BOM after the fact. Discovery from production is useful for drift detection, but it is not trustworthy release provenance.
• Inventory only the base model. Adapters, tokenizers, prompts, retrieval, policies, tools, and runtime often determine the material difference.
• Accept moving tags as versions. A branch, “latest” container, model alias, or undated document cannot support exact comparison.
• Confuse presence with approval. A component can be fully identified and still fail security, rights, quality, or use-case policy.
• Sign an incomplete statement. Cryptography preserves the integrity of omissions too.
• Publish sensitive internals. Use tiered views and controlled references; do not turn transparency into secret or personal-data leakage.
• Collect fields no consumer checks. Every mandatory field should support an impact query, policy decision, operational action, or disclosure obligation.
• Ignore runtime drift. External APIs, indexes, feature flags, and policies can diverge after build; deployment must report observed identity.
• Retain forever. Passport records can contain commercially sensitive lineage and security findings. Apply access, retention, legal hold, and deletion policy.

Finer granularity improves impact analysis but increases collection and review. Start with components that change behavior, authority, security, rights, compatibility, or evaluation validity. Add a field only when its consumer can name the decision it improves.

Measure whether the passport changes outcomes

Do not report “BOM generated” as success. Measure the control loop:

• required-field completeness: populated mandatory fields divided by applicable mandatory fields;
• immutable-identity coverage: consequential components with verifiable identifiers divided by all consequential components;
• unknown-provenance exposure: weighted count of unknown-origin components by use-case consequence;
• attestation verification rate: candidate subjects passing identity, signature, and expectation checks;
• evidence-to-release precision: linked reports that actually name the current release digest;
• runtime match rate: observed deployments matching an approved manifest;
• change-impact lead time: time from a supplier incident or vulnerability to a list of affected releases and owners;
• rollback readiness: age and success rate of whole-manifest restoration exercises;
• stale-passport time: duration between runtime drift and detection or reconciliation;
• exception burn-down: open verification limitations by severity, owner, and expiry.

Keep denominators visible: 99 per cent coverage can conceal the hosted model or payment tool carrying most consequence. Pair coverage with risk weighting and stop conditions.

Start with one release gate, not an enterprise inventory project

Choose one bounded service and define its rollback unit. Encode the six component classes, relationships, identity rules, evidence requirements, and moving-alias policy. Generate the passport in the build path, compare it with production, ask “which active releases use this embedding model?”, then restore the previous manifest outside production.

The exercise reveals valuable gaps: an API without an immutable version, an irreproducible index, a report naming only a model family, an uncontrolled prompt, or a fallback sharing the same failure domain. Fix the highest-consequence gap before broadening the schema.

Revisit the passport after a new tool or supplier, risk-tier or jurisdiction change, altered data purpose, changed release unit, or new provenance format. Preserve schema versions and migrations so old passports remain interpretable.

The practical principle is simple: ship what you can name, verify what you name, and roll back the assembly—not a guessed subset of it. An AI release passport will not make a weak system good. It makes the exact system visible enough for responsible people and automated gates to decide whether it should advance.

Source Notes — reviewed August 2, 2026

• NIST SP 800-218A: Secure Software Development Practices for Generative AI and Dual-Use Foundation Models
• SPDX Specification 3.0.1: AI Profile
• OWASP CycloneDX Authoritative Guide to AI/ML-BOM, First Edition Revision 1
• UK NCSC: Guidelines for Secure AI System Development — Secure Development
• SLSA Specification 1.2
• SLSA 1.2 FAQ: relationship between SBOM and provenance