Every CTO I talk to right now is being asked the same question by their board: “What’s our AI strategy?” And most of the strategies produced in response share a defect — they describe what the company would like to do, not what it can execute. The gap between those two is measurable, and measuring it takes about a week. Strategy written before the measurement is fiction with a budget.
This post is the assessment I run. It’s deliberately meeting-light: most of it is reading systems and asking individual engineers short, concrete questions. You can run it yourself with the rubric below.
What does “AI-ready” actually mean?
An organization is AI-ready when four things are true: its data is reachable and clean enough to retrieve against; at least a few engineers can build and evaluate an LLM feature end-to-end; its tooling can deploy and observe a non-deterministic system; and someone can say “no” to an AI use case with authority. Readiness is the floor under every initiative — strategy decides what to build, readiness decides whether anything survives contact with production.
Notice what’s not on the list: a vendor choice, a model choice, a platform. Those decisions are downstream of readiness and cheaper to change than any of the four floors.
Day 1–2: Can your data be retrieved against?
Pick the three data sources your most-wanted AI use case would need, and try to answer one real question from each — manually, with existing access. Time how long it takes and how many people you had to ask. If an engineer with legitimate access needs two days and three Slack threads to assemble the context a feature would need in 200 milliseconds, the feature is blocked on data plumbing, not on AI.
Score it bluntly: green if the data is reachable through an API or warehouse with documented meaning; yellow if it’s reachable but the meaning lives in someone’s head; red if access itself is the project. In my experience most companies that self-describe as “sitting on a goldmine of data” score red on two of three sources.
Day 2–3: Do you have the skills, and where are they?
You’re not counting “engineers who used ChatGPT.” You’re counting engineers who can design a retrieval pipeline, put tool boundaries around an agent, and write an eval that catches a regression. Ask five senior engineers to sketch — on a whiteboard, in ten minutes — how they’d build your most-wanted feature. You’re listening for retrieval, context structure, evaluation, and cost; the vocabulary of context engineering rather than the vocabulary of demos.
Two or three engineers who pass that bar is enough to start; zero is a finding that reshapes the strategy — your first initiative becomes building the capability, not the feature. That’s a better outcome than discovering it mid-project, and it’s a gap that closes with deliberate training (the RAG and agents course exists for exactly this profile: strong engineers, no production-LLM mileage yet).
Day 3–4: Can your process handle non-determinism?
Traditional delivery assumes the same input produces the same output; LLM features break that assumption, and process is where the break shows. The question to answer: when the AI feature misbehaves in production, what exactly would your team look at? If the answer isn’t some form of “the traces, and then we add the failure to the eval set,” you have a process gap, not a tooling gap.
This is also where I look for a specification habit. Teams that write down what a feature must and must not do before building it adapt to AI development dramatically faster — the spec becomes the eval set almost mechanically, which is the core loop of spec-driven development. Teams that ship from Slack threads discover that “the model does something weird sometimes” is not a bug report anyone can act on. If that habit is missing, building it is a prerequisite, not a nice-to-have.
Day 5: Who can say no?
Governance readiness is one question: name the person who can veto an AI use case on risk grounds and make it stick. If the answer is a committee that hasn’t met, or the CTO themselves after the project already has momentum, score it red. Everything else in a governance framework — usage policy, review gates, data boundaries — works only when the veto is real.
The corollary question is just as revealing: name a use case you’ve already said no to. Organizations that can’t name one haven’t been governing; they’ve been hoping. A written strategy inherits credibility from the no’s it can point to.
The rubric, on one page
This is the whole assessment in a form you can take into a leadership meeting. Score each floor honestly; the uncomfortable column is the useful one.
| Floor | Green | Yellow | Red |
|---|---|---|---|
| Data reachability | Reachable through an API or warehouse, and the meaning of the fields is documented | Reachable, but the meaning lives in one person’s head | Getting access is itself the project |
| Skills | Three or more engineers have shipped and evaluated an LLM feature end to end | One engineer can, with nobody behind them | Nobody can sketch retrieval, tool boundaries and evals in ten minutes |
| Process | Traces exist, and production failures become new eval cases | Logs exist; nobody reads them until something breaks | “It does something weird sometimes” is considered a bug report |
| Governance | A named person can veto a use case, and has actually done it | A committee exists on paper and has never met | No one has ever said no to an AI idea |
Strategy written before the measurement is fiction with a budget. The scores above are what turn an AI strategy from a slide into a sequence.
Turning the score into a strategy
Four floors, each scored green, yellow, or red — and now the strategy nearly writes itself. Reds are the first initiatives; there is no point funding features on top of them. Yellows shape the timeline honestly. Greens are where the visible wins should come from in the next two quarters, because a strategy with no visible wins loses the room regardless of how sound it is.
The write-up that comes out of this — scores, evidence, the sequence, owners, and the first ninety days — is one I’ve done as a facilitated engagement for companies that want an outside senior eye and a document the board will trust: that’s AI readiness & strategy. But the rubric above is the whole method. Run it before you write the strategy. A week of measurement beats a quarter of confident fiction.
