AI Readiness Checklist: 10 Gates Before Your First Pilot
An AI readiness checklist should establish whether one business workflow is ready for a controlled pilot. It needs more than a list of installed tools: identify the work, inspect its data, test its permissions, define an acceptable result and name the people who will operate it.
The useful output is a decision with evidence: proceed within a stated scope, remediate specific gaps, or stop this use case. A company can be ready for a document-answering assistant while remaining unready for an agent that changes customer records. Assess those workloads separately.
This guide provides ten practical gates for enterprise generative AI projects. Use them before purchasing a platform, commissioning a proof of concept or extending an employee pilot into production.
How to use this AI readiness checklist
Start with the business owner, a representative user, the data owner, security and the intended technical operator. One person may cover several roles, but every decision needs an accountable individual.
Copy the checklist into your project tracker. For each row, record a status of pass, gap or not applicable, an evidence link, a reviewer and a review date. Explain every not-applicable decision. A presentation saying that access is restricted is weaker evidence than a recorded test showing a restricted account being denied.
Decide which gates are mandatory for the proposed scope before testing. Do not average a missing data permission into an otherwise strong readiness score. A gate can pass for an isolated sandbox with approved sample data and still remain open for a live-data pilot.
The NIST AI RMF Playbook organizes suggested actions around Govern, Map, Measure and Manage. NIST explicitly describes the playbook as voluntary guidance, not a checklist to follow in full. The checklist below is our practical planning aid; it is not a NIST certification or a validated maturity benchmark.
Figure 1. Readiness is a decision supported by evidence. Remediation returns to the relevant checks before pilot scope expands.
The 10 AI readiness checks: evidence, owners and acceptance gates
Treat the pass conditions as starting points to adapt to the consequences of your workflow. Set any numerical quality, latency or cost thresholds with the business owner before the evaluation begins.
| Gate | Evidence to collect | Accountable owner | Pass condition |
|---|---|---|---|
| 1. Defined workflow | A process map naming users, inputs, outputs and excluded actions | Business process owner | The team agrees what the pilot may do and what a completed task means |
| 2. Current baseline | Representative task samples, handling time, error categories and existing service cost | Operations lead | The current process and proposed pilot can be compared on the same task mix |
| 3. Usable source data | Source inventory, permitted uses, content owners, freshness and known gaps | Data owner | Required sources are available, approved for use and adequate for the scoped questions |
| 4. Enforced access | Identity mapping and tests for allowed, restricted and revoked users | Security or identity owner | Retrieval and tool calls respect the tested user’s actual permissions |
| 5. Known data boundaries | A diagram of application, model, connectors, storage and logs, with retention settings | Architecture owner | Each processing destination and retention rule has been reviewed for the intended data |
| 6. Measurable quality | Versioned test cases, expected outcomes, scoring criteria and failure categories | Evaluation lead with domain reviewer | The system meets agreed criteria on ordinary and difficult cases, with failures reviewed |
| 7. Controlled actions | Allowed operations, review points, rejection behavior and an exception route | Process owner with security | Consequential actions require the specified authority; rejection prevents the protected action |
| 8. Operating ownership | Monitoring, incident contacts, support instructions, change control and recovery exercise | Service owner | An assigned team can identify, contain and recover from the tested failures |
| 9. Viable economics | Cost assumptions covering software, usage, integration, review and support | Budget owner | Expected benefits and all in-scope costs support the proposed pilot decision |
| 10. Bounded rollout | Named users, approved data scope, duration, stop conditions and decision meeting | Pilot sponsor | The team can stop the pilot and knows what evidence is required to expand it |
Keep evidence close to the decision. Record the model configuration, source snapshot and permission setup tested; a later change can invalidate an earlier pass.
Resolve the gaps that a readiness score can hide
Data readiness includes ownership and conflicting information
A document collection is not ready simply because it can be indexed. Identify which source governs when two policies disagree, how superseded material is removed and who investigates a missing answer. Check whether a user can open the cited source, not only read the generated response.
For an employee assistant, begin with a maintained collection and its real access groups. The customer-hosted knowledge assistant guide provides concrete tests for supporting passages, outdated policies and restricted users. Large historical data volumes are less useful than evidence that the selected material answers the pilot’s actual questions.
Access readiness depends on the whole action path
List each operation separately: reading a policy, searching a record, drafting a response and updating a system have different consequences. Check the destination identity as well as the chat application’s sign-in. A user interface can appear restricted while a shared connector has broader access.
OWASP’s excessive-agency guidance recommends limiting tool functionality and permissions, enforcing authorization in downstream systems and requiring approval for high-impact actions. Translate those principles into observed allow-and-deny tests. For workflows that change records, use the human-approval checklist to test rejection, changed inputs and retries.
If the proposed rollout includes a coding assistant, the Claude Code managed settings guide helps turn endpoint policy into deployment checks. Record what the administrator enforces and verify it on a managed device before expanding developer access.
Evaluation readiness means defining failure before the demo
Build the test set from the work people actually encounter. Include ordinary requests, missing information, conflicting sources, restricted content and unavailable dependencies. Preserve some cases that were not used to tune prompts. Have a domain reviewer establish the expected behavior before seeing the model’s answer.
Amazon Bedrock’s evaluation tooling supports programmatic, human and model-judge evaluation methods, including evaluation of retrieval-augmented generation. These methods can help collect evidence; your team still decides which measures establish success for the business task.
Report results by failure category. An acceptable answer that takes too long, an unsupported answer with a plausible citation and a permission failure need different remedies. For repeated or variable outputs, rerun representative cases and record that variability rather than selecting the best response.
Operational readiness starts before production
Run a small recovery exercise: remove access to a source, interrupt a dependency or roll back a configuration change. Identify who notices, who can disable the workflow and how users complete their work while it is unavailable.
The AWS Generative AI Lens includes operational excellence, reliability and cost optimization alongside security. Apply that wider perspective to the pilot: somebody must maintain prompts, source refreshes, permissions and integrations after the first successful demonstration.
Worked example: a policy assistant with a blocked access gate
Consider a hypothetical operations team proposing an assistant for internal travel-policy questions. It will answer from approved documents and cite passages; it will not approve expenses or change employee records. This is an illustrative assessment, not a customer result.
The process owner gathers routine questions, regional exceptions, missing-policy cases and queries involving restricted documents. A policy specialist writes the expected answers. IT runs the cases using employee and administrator accounts.
| Observation | Readiness decision | Required follow-up |
|---|---|---|
| Ordinary questions cite current policy and meet the agreed answer criteria | Quality gate passes for those tested cases | Retain the cases and source versions for regression checks |
| An employee account retrieves a restricted executive travel document | Access gate has a gap | Fix retrieval authorization and repeat allowed, denied and revoked-user tests |
| The assistant escalates an unanswered regional exception to the policy owner | Exception behavior passes for that case | Verify the owner can receive and resolve the handoff |
The decision is remediate before the live-data pilot. Passing answer-quality tests does not cancel the access failure. A separate sandbox using explicitly approved sample documents could continue if its own boundaries and gates are satisfied.
After remediation, the sponsor approves a limited user group and schedules a review of answer usefulness, unresolved questions and employee correction effort. The recorded authorization covers that scope; adding a new repository or enabling expense updates triggers another review.
What should AI assessment services and platforms deliver?
Ask an assessment provider for a decision package
When comparing AI assessment services, request the deliverables that let another team inspect the conclusion: a scoped workflow, baseline, evidence register, architecture, tested gaps, acceptance criteria and a remediation plan with owners. Ask which findings came from interviews, configuration inspection or executed tests.
Require assumptions and dependencies in the estimate. An assessment that identifies unavailable data should explain the next decision, not quietly price an implementation as though access already exists. Use the build-or-buy workflow guide when the question becomes who supplies and maintains the resulting components.
Evaluate an AI capability assessment platform against the evidence workflow
An AI capability assessment platform is useful if it can connect answers to evidence, assign owners, track changes and export the decision record. Test those actions during the demo. Verify who can view sensitive evidence and whether reviewers can distinguish an unsupported questionnaire response from an observed control test.
A platform’s aggregate score should not automatically clear a mandatory gate. Keep the underlying findings available and make the sponsor’s acceptance explicit. For choosing an employee workspace after requirements are clear, use the ChatGPT Enterprise vs. Claude Enterprise comparison to assess the candidates against the same pilot needs.
Leave the assessment with a decision and a next owner
Record the approved scope, unresolved gaps, remediation owners, next review and stop conditions. Revisit the affected gates when the model, source data, permissions or intended actions change. Readiness is specific to that configuration and use case.
For implementation planning, contact ASCENDING with one workflow, a representative data sample, existing systems and the evidence gaps identified here. Use that material to scope an AI assessment or a Jarvis implementation discussion, and agree deliverables before committing to a broader rollout.
References
- NIST: AI RMF Playbook
- OWASP: Excessive Agency
- AWS: Evaluate the performance of Amazon Bedrock resources
- AWS: Well-Architected Generative AI Lens
AI readiness questions before the pilot
What should an AI readiness checklist cover?
Cover the workflow, baseline, data, access, processing boundaries, evaluation, human decisions, operating ownership, economics and rollout evidence for one defined use case.
How do we know whether we are ready for an AI pilot?
Proceed when the proposed pilot's required gates have evidence and accountable owners. Remediate unresolved access, data or evaluation blockers before expanding scope.
What should AI assessment services deliver?
Request a scoped use case, evidence register, tested gaps, acceptance criteria, cost assumptions and a sequenced remediation plan with owners and a documented decision.
Can an AI capability assessment platform determine readiness automatically?
A platform can organize evidence and track reviews. Business owners must still judge usefulness, verify controls and accept the remaining risks for the specific workflow.
Is AI readiness the same as AI maturity?
Readiness asks whether a specific workflow can proceed under defined conditions. Maturity describes broader organizational capabilities; a high maturity score cannot clear a missing pilot control.
