AI Vendor Due Diligence: Questions Worth Asking First
Capability you can measure yourself in an afternoon. Everything that binds you for years, you can only be told.
AI-drafted, reviewed by Muhammad Qasim Hammad on November 1, 2026. See our AI disclosure.
Table of contents
Most AI vendor evaluations test the demo and skip the contract. The AI vendor due diligence questions that matter are the ones whose answers you cannot discover later, and almost none of them are about model quality. Quality you can measure yourself in an afternoon. Everything below, you can only be told.
Why not lead with capability?#
Because capability is the only thing you can establish independently. Run 200 of your own inputs through the thing and you will know more about its quality than any vendor questionnaire will tell you, and you will know it in a form that matches your actual traffic rather than a benchmark.
Everything else is asymmetric. What happens to your data, what the liability cap is, how deprecation is handled and what provenance the output carries are all facts held by the vendor, invisible from the outside, and expensive to discover through experience. That asymmetry is what due diligence is for.
A useful ordering rule follows from that. Spend 1 afternoon measuring quality on your own inputs and 6 emails on the things you cannot measure, rather than the reverse. The second list is shorter, duller, and decides more.
Teams get this backwards because capability is interesting and contracts are not. The result is a thorough capability assessment attached to a signature on terms nobody read, and the terms are the part that binds for the next 3 years.
What should you ask about liability?#
The cap, and how it compares to what you have promised your own customers. A cap set at some months of fees is normal and is unrelated to the size of harm an autonomous system can cause, so the gap between the two numbers is your exposure whether or not anybody has priced it.
Ask specifically what is excluded, because exclusions do more work than the cap. Output accuracy is commonly excluded entirely, which means the vendor warrants that the service runs rather than that it is right, and that distinction is the whole substance of what you are buying.
| Question | Cheap to ask now | Expensive to discover later |
|---|---|---|
| What is your liability cap? | One email | After a customer claim |
| What is excluded from warranty? | One email | When output is wrong at scale |
| Where is data processed and retained? | One email | During a compliance review |
| What provenance does output carry? | One email | On a labelling deadline |
| What is the deprecation notice period? | One email | With 30 days to migrate |
| What happens if a regulator restricts the model? | One email | On the day it happens |
The last row is new and most contracts have nothing to say about it. Regulators can now require a provider to take corrective measures including restricting a model's public availability, which is a supply event arriving on a regulatory timetable rather than a commercial one.
What should you ask about data?#
Where it is processed, how long it is retained, whether it trains anything, and how deletion works. Those four are standard and worth restating because the answers vary far more between vendors than the marketing does, and the second and fourth interact badly with your own retention obligations.
Deletion deserves a follow-up question rather than a yes. Ask what happens to data already inside logs, caches and evaluation sets, because a vendor can honestly answer that customer data is deleted while derived copies persist somewhere the deletion pipeline does not reach.
Training use has 2 separate answers and both matter: whether your inputs train the vendor's models, and whether they are retained for abuse monitoring by human reviewers. A vendor can truthfully say your data does not train anything while a person can still read it under a different policy.
Residency answers should name regions rather than describing intent. A commitment that data stays within a region is a different thing from a statement that it is usually processed there, and the difference only becomes visible when somebody asks you to prove it.
What should you ask about change?#
Deprecation notice, version pinning, and whether behaviour can change under a stable version name. The third is the one people forget, and it is the one that breaks systems quietly, because the endpoint keeps working and the outputs stop matching what your prompts were tuned against.
Version pinning is worth paying for where it is offered. A pinned version that stays available for a stated window converts an unpredictable behavioural change into a scheduled migration, which is the difference between a planned week and an unplanned one.
Ask for the active parameter count too, if you are ever considering hosting anything yourself. A model can be openly licensed and still sit far beyond what you would put on a cluster, and the ratio between total and active parameters tells you which situation you are in before you plan around it.
How many of these actually get answered?#
Fewer than you would like, and the pattern of refusals is itself information. A vendor that answers the data and deprecation questions crisply and goes vague on liability is telling you where the risk sits, and that is a more useful signal than a completed security questionnaire.
Send them as a short list rather than a document. Six questions in an email get answered by a person who knows the answers; a 40-page assessment gets routed to someone whose job is to return it completed, and the second process produces paper rather than information. What you owe users when the system is wrong is the reason the liability answers matter more than the capability ones.
Frequently asked questions
What should we ask an AI vendor before signing?
Why does the liability cap matter so much?
What is the question people forget about model changes?
Is a completed security questionnaire good enough?
What should we ask about deletion?
Written by
Muhammad Qasim Hammad is an AI agent and automation expert and the founder of Cart Gaze LLC (cartgaze.com). He builds product for the love of it: when an idea lands, a working prototype is usually running within hours, built with the same AI agents and automations he sells. He puts his own output at roughly 20× what it was before agents, and the Agentic OS behind this site is the working proof, documented in public with the tools he actually ran and what they really cost.
AI & Automation Services
Want a pipeline like this running in your business?
I'm Qasim — I design and ship AI agents and n8n automations for solo operators and small teams. Tell me what's eating your team's week, and I'll scope a fix.
Related reading
Data Privacy for AI Tools: The Decisions to Make First
Most small businesses decide what data may reach their AI tools retroactively, when a client asks. Eleven lessons on the inventory, the three data tiers, what zero retention actually covers, and why the model is rarely the leak.
Who Is Liable When an AI Agent Makes a Mistake in 2026?
An agent you deployed emailed a customer the wrong refund amount. The answer is always a person or a company, never the software, and which one is decided long before the incident.
EU AI Act Fines for Providers: Who Actually Gets Asked
The headline numbers travelled further than the detail. Almost none of that ceiling is aimed at the people quoting it nervously, and the first contact is a question.
AI Agent Audit Log Requirements: What to Record and Why
Somebody asks what your agent did on 14 March for one specific customer. Most teams find out the answer is no on the day the question is asked, not the day the log was designed.
Why AI Pilots Fail to Reach Production, and What Works
A pilot that answered 40 questions correctly in a demo is not evidence of anything yet. It is a controlled test on data somebody picked, and production asks a different question entirely.
Does a Chatbot Have to Say It Is AI? Yes, Since August
For years the answer was a matter of taste. Since 2 August 2026 it is a legal duty in the EU, the exemption is narrower than teams hope, and the deadline has already passed.





