The AI RFP cannot select for a future you cannot specify

The last supplier presentation ends, and the room goes quiet in a particular way.

Your room has watched three polished versions of a future that’s somehow already late.

The screen is full of weighted criteria.

The notes have become scores.

Somebody asks whether the commercial total includes implementation. Somebody else says the leading vendor has a better strategic fit.

Then the spreadsheet produces a winner with two decimal places.

You’re not foolish for wanting that number to settle the unease.

A selection meeting is full of people with work to get back to, budgets to defend, a service problem that’s been waiting too long, and an executive who wants to say the organization has an AI plan.

The score is a small mercy. It turns an argument about an unknown future into a number that can be approved before lunch.

The dangerous-AI story is available and it’s the wrong one.

It’s about a spreadsheet.

An RFP can make your uncertainty look governed without making it any smaller.

2007 — The RFP asks vendors to promise transformation. Broad enough to sound ambitious, thin enough to survive the first meeting.

2016 — The scorecard gives the promises precision. A weight, a decimal, an approved recommendation.

2024 — The AI demo wins your room. It answers a friendly question from a clean document and everybody can picture the saving.

2026 — The service is live. The source is stale, the exception is real, the queue is mixed, and the unknowns have finally become work.

The old purchasing discipline isn’t useless here.

It’s useful precisely because it was built for the less glamorous problem. How do you compare people who want to sell you something when you don’t yet know exactly what you’ll need?

But it has a limit. It can decide what’s evidenced now. It can’t select for a future your organization hasn’t managed to describe.

The sheet that knows too much

An RFP is supposed to make offers comparable.

It names an outcome boundary, a service description, assumptions, mandatory conditions, criteria, weights, score descriptors, a price schedule, a timetable, clarification rules, security questions, references, and the terms still to be negotiated.

That isn’t theatre. Without a common response template, one supplier prices ten thousand API calls, another prices a hundred thousand employee seats, and your committee calls both totals annual cost.

But comparability starts before the RFP.

An RFI exists for the moment you still don’t know whether the market can deliver, which delivery model fits, what vocabulary to use, or whether the requirement should be split.

Eight to twelve focused questions. Capacity limits.

Data locations.

Implementation lead time. Price units.

A reference architecture. One representative use case.

That’s market learning. Not a scored tender, and not a promise to award.

The distinction matters more with AI, because the unknowable part is being smuggled into the word requirements.

A vendor can show you a model reading an approved knowledge base.

It can’t settle, in a demonstration, whether your records have an owner. Whether an old policy can be removed. Whether permissions follow the employee. Whether your exception queue is where the value goes to die. Or whether an accepted draft was correct rather than merely convenient.

You’re not failing at procurement because that feels inadequate.

You’re noticing that the RFP is being asked to choose a production service when all anybody has is a picture of a feature.

In a mature market, documented research may be enough to go straight to a short RFP. In a novel one, the honest successor is an RFI followed by paid discovery, a pilot, or dialogue.

Not a grand specification pretending to know the answer.

The rule is almost embarrassingly plain. If a supplier tells you a thirty-day data migration needs a parallel-read period, test that constraint independently before you rewrite the world around that supplier’s product.

The score is a record

The second small seduction is the weight.

A zero-to-five score looks like judgment made visible. It’s only useful while the judgment stays inspectable.

Business outcome. Requirement. Criterion. Evidence request. Evaluator observation. Score. Trade-off. Award condition.

That chain is the real object. Your number is its shorthand.

Which is why a file with a hundred numerical scores and no evidence locators isn’t auditable.

It looks more rigorous than a folder of notes. It’s less rigorous, because nobody can find the exact RFP question, bid page, test output, scoring anchor or evaluator note that made the number appear.

A preference has put on a lanyard and called itself evidence.

The older rules say the same thing in dry language.

The World Bank separates pass/fail requirements from weighted rated criteria. FAR 15.305 permits numerical weights, adjectival ratings and ordinal rankings, and requires the strengths, deficiencies, significant weaknesses and risks behind an evaluation to be documented.

Neither rule makes a private buyer federal. Both make the same question hard to dodge.

Did the reason for your award exist before you knew the winner?

Take four bidders scoring between 82 and 85 out of 100 on quality.

Maybe the offers really are similar. Maybe your descriptors can’t discriminate.

Either way, don’t hand the 85 an aura of inevitability. Change a debatable major weight by ten points. Replace an uncertain score with the next plausible one. Put a realistic high and low price against the same usage assumptions.

If a twelve-scenario test produces nine different winners, your baseline winner isn’t a discovery.

It’s a prompt for more evidence, better anchors, or a conscious decision about what actually matters.

None of which makes your evaluators corrupt, timid or bad at arithmetic. It makes the uncertainty visible.

The correction is gentler and harder. You’re not choosing the perfect provider.

You’re choosing whether to keep the ability to learn after the award.

References can’t see next year

Reference calls have the same comforting quality. Someone else used the product. Someone else survived implementation.

Three calls become a little stack of borrowed confidence.

The record asks something more demanding. A reference has five fields. Comparable scope, scale, operating environment, recency, and delivery-team or product match.

A four-out-of-five reference can support a performance claim. A one-out-of-five reference isn’t a bad reference. It’s a lead for a further question.

And its adverse finding needs a disposition. Resolved with evidence. Accepted with mitigation. Escalated for negotiation. Or disqualifying.

That’s the difference between assurance and decorative diligence.

A financial report eighteen months old is history, not current assurance. A security attestation can cover the previous product. A reference from before an acquisition describes a delivery team that no longer exists.

Your shortlist is where diligence should intensify. Identity. Financial resilience. Security and privacy evidence. Delivery staffing. Subcontractors. Data exposure. And the awkward report from a customer the account executive didn’t pick for you.

You’ll recognize the feeling in that last part.

The polished demonstration has already made the decision socially expensive to reopen. The finalist is in the room. The executive is waiting. And now somebody wants to make a call that might complicate the story.

That discomfort isn’t bureaucracy asserting itself.

It’s the last space in which a promise can become a commitment. Or fail to.

The future starts after award

Vendor selection ends at the award. Enterprise AI adoption begins there.

Which is why one document can’t honestly carry both burdens.

A pilot worth the word starts with a bounded queue. Password resets and billing explanations, not “improve customer service with AI.”

It records your weekly request volume, median handling minutes, recontact rate, quality score, escalation rate and cost per completed case before the service changes the work.

It states which sources may be used, what the service must never answer, what stays human-only, and what event ends the pilot.

Without that baseline, time saved is a story told afterwards.

The vendor’s model may be excellent and your service may still not exist.

Your records have to become a data product with a named owner, defined meaning, access rules and a way to repair defects. Each source needs provenance, a version, an audience, permissions, freshness, a source of truth.

Your retrieval layer has to follow the employee’s access rule, and make it possible to remove a document or revoke an entitlement.

That isn’t a feature on the demo screen. It’s the work that makes the answer safe enough to send.

And the test has to resemble the service as it will actually run.

Held-out cases, not just development examples. Adversarial requests: another customer’s data, a prompt buried in a source document, a stale policy, a case that needs a human.

NIST says systems should be tested before deployment and regularly while operating.

An accuracy average can’t settle the matter when the unacceptable error lives in a high-consequence subset.

Then the quieter test. People.

A small trained group sees the sources behind a draft, knows when not to use it, and can say why it was edited or rejected.

Cases eligible, drafts shown, drafts sent untouched, drafts rewritten, drafts overruled, and how long each took. Those aren’t six versions of one metric.

A high acceptance rate can mean confidence. It can also mean nobody has the time to disagree.

That’s the moment your 2024 demo becomes your 2026 service.

The scorecard can’t take it for you. It can only leave an honest trace of what was known, what was assumed, and what needs watching.

What the award didn’t buy

The invoice is another pleasant lie.

Model inference gets priced per user, request, token, document, GPU hour, or committed annual minimum. Whichever unit makes the total look smallest in your meeting.

The service also consumes product-owner time, data engineering, integration, evaluation, security review, legal and procurement work, training, help desk, monitoring, incident response, redesigned work, parallel running, audit, and the cost of errors.

Those sit in different budgets. Which is one reason a vendor-led return calculation feels so clean.

The missing work didn’t vanish. It arrived on a different cost centre.

At thirty, ninety and a hundred and eighty days, you can finally compare your award case against reality.

Mobilization dates. Named-team continuity. Data access. Integration assumptions. Buyer-side effort. Implementation spend. The risks somebody named in diligence.

Variance can be supplier claim error, buyer requirement error, market change, or unavoidable uncertainty.

A sound choice can meet an unlucky shock. A weak choice can look fine until the first renewal.

The review isn’t a trial. It’s where your selection learns to tell those apart.

The RFP was never a machine for seeing the future.

It was a way to make one bounded commitment challengeable. Compare what can be evidenced. Expose the assumptions. Negotiate the promise into an obligation. Keep enough of the service visible to learn when reality shows up.

That’s less triumphant than awarding the best AI platform. It’s also more humane.

The people watching your score turn green aren’t trying to avoid thought. They’re trying to decide something in a world that handed them a demo instead of a future.

An honest award doesn’t pretend that problem is solved.

It leaves the door open long enough for the future to answer back.