AI Readiness Diagnostic | Loxley Advisory Group

Dimension 01 Model Autonomy

What percentage of AI-generated recommendations currently proceed without human review before action is taken?

0 ptsUnder 10% — nearly every output is reviewed

1 pt10–30% — most outputs go to human review

2 pts30–60% — meaningful automation with significant review still occurring

3 ptsOver 60% — human review reserved for exceptions

How formally are the criteria for autonomous AI action versus human review defined and applied in practice?

0 ptsNot defined — review decisions are made case by case

1 ptInformally understood — criteria exist but are not written down

2 ptsDocumented but inconsistent — written criteria exist but are not consistently applied

3 ptsFormally defined and consistently applied — documented threshold criteria govern every case

Can the operators who act on AI recommendations explain the basis for a specific output to a downstream stakeholder — a customer, shipper, regulator — if challenged?

0 ptsNo — operators cannot explain why the system produced a specific recommendation

1 ptRarely — operators can give general context but not specific reasoning

2 ptsSometimes — the system surfaces some context but the audit trail is incomplete

3 ptsYes — the system provides operators with the context they need to defend outputs downstream

Dimension 02 Data Liquidity and Feedback Loops

How does the AI system receive the data it needs for inference relative to when the underlying events occur?

0 ptsBatch only — data refreshes on a fixed schedule with material latency

1 ptMostly batch — some inputs are near-real-time but core data is still batch

2 ptsPartially real time — most inputs arrive quickly but latency issues remain in some data paths

3 ptsReal time — inference data arrives within seconds of the triggering event across all key inputs

How does the organization capture whether AI outputs were correct after the fact?

0 ptsIt does not — there is no mechanism to capture outcome data

1 ptManual and periodic — a sample of outcomes is reviewed on a schedule

2 ptsPartially automated — some outcomes are captured automatically but coverage is incomplete

3 ptsAutomated — ground-truth outcome data flows back to monitoring infrastructure continuously

What governs the relationship between the teams that supply data and the AI systems that consume it?

0 ptsNothing formal — data dependencies are implicit assumptions

1 ptInformal understanding — expectations exist but are not documented or enforced

2 ptsPartial contracts — some data paths have documented SLAs but coverage is incomplete

3 ptsFormal data contracts — schema, freshness, and availability are defined and actively maintained

Dimension 03 Compute and Margin Ratio

How thoroughly have production inference costs been modeled — including peak load, redundancy, and the full inference stack beyond API call costs?

0 ptsNot at all — cost estimates come from pilot API costs only

1 ptHigh-level only — production costs have been estimated but not validated against a full stack

2 ptsPartially — full-stack costs have been modeled but not tested at production volume

3 ptsFully — full-stack inference costs at production volume are documented and validated

For initiatives designed to displace manual work, how well understood is the actual cost of the manual process being replaced?

0 ptsNot measured — the business case relies on assumed displacement value

1 ptEstimated — high-level labor cost is known but throughput per unit has not been measured

2 ptsMostly measured — throughput data exists but fully burdened unit cost has not been calculated

3 ptsFully measured — per-unit manual cost is documented from observed throughput at fully burdened labor cost

At production volume, how does the cost of automation compare to the cost of the process it displaces?

0 ptsUnknown — the comparison has not been made at production volume

1 ptUnclear — the comparison has been made but the math depends on unvalidated assumptions

2 ptsMarginally favorable — automation is cheaper but the margin is thin

3 ptsClearly favorable — automation is materially cheaper with a validated and defensible margin

Dimension 04 Governance and Lineage

How formal is the authority structure for approving AI initiatives for production deployment?

0 ptsNo authority defined — deployment decisions are made informally by whoever is driving the initiative

1 ptInformally assumed — a person is expected to hold authority but it has not been documented

2 ptsPartially formal — authority is documented but review criteria or process are not

3 ptsFormally documented — named authority, defined criteria, and documented review process in place

If a specific AI decision were challenged today by a customer, regulator, or acquirer, how quickly could your organization fully reconstruct it — data inputs, model version, and accountable owner?

0 ptsIt could not — decision logs either do not exist or are insufficient for reconstruction

1 ptDays — logs exist but reconstruction requires manual correlation across multiple systems

2 ptsHours — reconstruction is possible but requires significant engineering effort

3 ptsWithin the hour — complete provenance is logged and the process has been tested

How formally has the organization assessed the regulatory and privacy exposure of its AI initiatives?

0 ptsNot assessed — compliance and privacy have not been formally engaged on AI initiatives

1 ptInformally — awareness exists at the leadership level but no formal assessment has occurred

2 ptsPartially — compliance and privacy were consulted but a formal impact assessment was not completed

3 ptsFormally — documented risk and privacy impact assessments were completed before pilot launch

Is your AI program ready for production?