The AI Vendor Dossier: Buying Evidence, Not a Demo

An AI demo is designed to show the best path. Procurement must understand the ordinary path, the failure path, and the exit path.

Ask for the Operating System Behind the Product

Start with the proposed use case and data classification. Then examine:

• Where prompts, files, outputs, feedback, and logs are stored.
• Which subprocessors and model providers participate.
• Whether customer data is used for training or improvement.
• How access, retention, deletion, encryption, and regional hosting work.
• Which evaluations are run before and after model changes.
• How incidents, outages, abuse, and security reports are handled.

Request evidence appropriate to the risk: architecture, control descriptions, evaluation results, service history, and a live demonstration using difficult examples from your workflow.

Model the Real Cost

Price per token or seat is only the beginning. Include integration, retrieval, human review, observability, data preparation, support, overages, model migration, and the cost of a quality failure. Test how billing changes with long context, retries, tools, and peak usage.

Plan the Exit Before Entry

Identify which data, prompts, evaluations, logs, and generated assets can be exported. Clarify deletion timelines and whether the workflow can move to another model or provider without being rebuilt.

A pilot should have pass, pause, and stop criteria. Define the outcome, acceptable error, required controls, budget, and evidence needed for expansion.

The goal is not to find a vendor with no risk. It is to choose a relationship whose risks are visible, bounded, and operable.