Red flags in AI pitches

A persuasive deck can make a proposal easy to remember while leaving the delivery questions unanswered. Treat claims as prompts for evidence. A credible provider should be able to explain what the system will do on your workflow, what it will not do, and who is accountable when it fails.

Questions behind common red flags

“We use the best model.” Ask which model fits the task, what could change the choice, and whether a fallback is available.
“Accuracy is near perfect.” Request the evaluation cases, scoring rule, failure examples and the human review plan. A score without the task definition is not comparable.
“Your data is safe.” Ask for the data-flow diagram: prompts, uploaded files, retrieval sources, logs, backups, subprocessors, retention and support access.
“Our team can handle everything.” Ask who will implement, review, integrate and operate the work. Confirm those people and responsibilities in the proposal.
“We can build anything.” Ask what is explicitly out of scope, which assumptions need validation first and what result would cause the provider to recommend stopping.
“We’ll hand it over.” Ask what code, configuration, runbooks, access records and training are included, and who handles the first production incident.

Hypothetical evaluation exercise: a supplier proposes summarizing support messages. Give each shortlisted provider the same sanitized examples, including one incomplete message and one outside the intended scope. Ask them to show the output, identify what must be reviewed, describe where the input travels and explain one failure path. This is a procurement exercise, not a report of a vendor test.

Look for tradeoffs, not performance theater

Healthy signals include clear assumptions, a narrow pilot, named engineers, a reviewable output, an explicit data path, a cost model with usage assumptions and an owner for post-launch support. One missing artifact is not proof of bad faith, but repeated evasive answers or guarantees without evidence are reasons to pause before granting access or signing a broad statement of work.

Score each provider against the same workflow, technical fit, data controls, delivery team, commercial terms and transition plan. Keep a record of unanswered questions and make them conditions for the next stage. See reference-call questions and how to check case-study claims. If you are comparing live proposals, bring them to the intake desk for a structured evaluation.