Why Do Bank AI Projects Get Stuck in Pilot?

Banks keep funding AI proofs of concept, and few can trace a return to the P&L. McKinsey’s 2025 State of AI survey found that only about 6% of respondents attribute 5% or more of EBIT to AI and report significant value from it (McKinsey, 2025). Most stalled bank pilots break down between a working demo and an approved, integrated, owned production process.

Why do bank AI pilots fail? The short answer

Bank AI pilots stall because the institution around the model is not prepared to run it in production, and six causes recur:

• Late governance and model risk sign-off: Model risk, compliance and legal review arrive after the demo. By then the design is fixed, and no budget covers validation.
• Legacy core integration: The pilot runs on a data extract. Production has to read from and write back to mainframe cores and batch systems.
• Data access and quality: Sensitive customer data sits behind privacy controls and in silos the pilot never had to cross.
• AI bolted onto unchanged workflows: The bank inserts AI into a process built for people, so exceptions and handoffs still route to humans.
• No owner or ROI KPI: Nobody in the business owns the output, and the team measures success as model accuracy instead of cost per case or loss rates.
• Explainability and audit demands: Examiners, consumers and auditors expect reasons the bank can state and trace. Many models cannot supply them on request.

These causes help explain why bank AI pilots stall, and each one is a gap in operating context: how the work runs, who decides, which systems hold the data, and what a regulator will ask to see. That is why Skan AI’s Context Graph of Work is designed to capture all of it from observed activity. Gartner expects over 40% of agentic AI projects to be canceled by the end of 2027, and it attributes those cancellations to escalating costs, unclear business value, and inadequate risk controls (Gartner, 2025).

What the failure numbers really measure

The two most quoted failure rates measure different things:

What a COO should understand is that, while financial services moves experiments into production at roughly the cross-industry pace, generative AI pilots across industries rarely reach measurable P&L impact.

Governance and model risk arrive too late

Most bank pilots treat model risk review, the check that a model works as intended and its risks are controlled, as the final gate before launch, and that gate is where many of them stall. Model validation, third-party due diligence, and fair lending review each require artifacts that pilot teams rarely produce: documented design choices, data lineage (the record of where data came from and how it changed), testing evidence, and vendor disclosures.

A team that starts gathering this evidence after the demo ends up rebuilding the pilot to match it.

The supervisory baseline changed in April 2026, when the Federal Reserve, the Office of the Comptroller of the Currency (OCC), and the Federal Deposit Insurance Corporation (FDIC) replaced the 2011 guidance, SR 11-7 and OCC Bulletin 2011-12, with revised interagency guidance issued as SR 26-2 and OCC Bulletin 2026-13 (OCC, 2026). The revision is principles-based and keeps the core expectation that validation evaluates whether a model performs as expected, generally before the model’s first use (Federal Reserve, 2026).

Effective challenge, meaning critical review by people independent of the model’s builders, still depends on reviewers who have the expertise, the independence, and the organizational standing to force changes.

The revised guidance places generative and agentic AI models outside its scope, and it states that a bank’s broader risk management and governance practices should guide the controls for any tools it does not cover (Federal Reserve, 2026). The agencies say they plan a request for information on model risk management and banks’ use of AI, including generative and agentic AI (OCC, 2026). A generative AI pilot therefore has to define its own sign-off path, and a pilot that never budgeted for one waits for approval.

Compliance, legal and model risk should join during scoping and agree the validation and audit sign-off path before the pilot begins.

When to bring in compliance, legal, and model risk teams

Bring them in on day one, before anyone scopes the use case. Compliance, legal, and security requirements decide which data the pilot may touch, which model types suit the decision, and what evidence the production review will demand. A workable sign-off path runs through these stages:

1. Use case and materiality assessment: Model risk and the business owner agree whether the tool counts as a model, how material it is, and whether it falls inside the revised guidance’s scope.
2. Data and privacy review: Privacy, information security and legal approve the customer data the pilot may use and the places it may be processed.
3. Consumer and fair lending review: Compliance confirms how the tool will produce and document adverse action reasons, disclosures or SAR decisions.
4. Third-party due diligence: Vendor risk assesses any external model or platform under the 2023 interagency guidance on third-party relationships.
5. Independent validation: Validators with no stake in development test conceptual soundness and outcomes before first use.
6. Inventory and internal audit: The model enters the bank’s inventory, and internal audit confirms that the process followed policy.

Bolting AI onto legacy workflows vs. redesigning them

Most pilot failures come from how the bank applies the technology. Gartner’s 2026 analysis puts it directly: “AI value is an operating model challenge, not a technology challenge” (Gartner, 2026). McKinsey’s 2025 survey found that workflow redesign makes one of the strongest contributions to meaningful business impact among all the factors it tested (McKinsey, 2025).

Demo wins rarely survive contact with real workflows. A demo runs on a curated sample of clean documents in a single application, following the expected path. Production looks different. A loan file arrives incomplete. An analyst copies figures from a mainframe screen into a spreadsheet. An exception goes to a supervisor by email, and a judgment call depends on context that nobody wrote down.

Formal procedures often leave out the tacit knowledge operators use to handle these exceptions. As a result, an AI tool built from the procedure manual fails on the cases that consume the most staff time.

Redesign starts with a map of the work as it runs today, including its variants and workarounds. With that map, the bank decides which steps the model takes over, which steps a person reviews, and who holds the authority to override the output. These redesign questions belong in the pilot charter:

• Which steps does the model perform end to end, and which does it only assist?
• Where do exceptions go, and who resolves them?
• Which handoffs between applications or teams disappear, and which remain?
• What does the reviewer see when accepting or reversing a model output?
• How does a reversal feed back into the procedure the model follows?

Core banking integration and data readiness

Legacy architecture blocks pilots at the point of scaling: a pilot that reads a one-time extract avoids the mismatch between batch data (processed in scheduled bulk runs) and event-based data (generated as each transaction happens), but production has to handle it every day.

The Bank for International Settlements (BIS) describes core banking systems that “often operate on legacy mainframes with rigid batch-based data structures,” while front-office channels generate event-based data, and it traces data fragmentation to legacy systems that store data in silos (BIS, 2026). The Bank of England and the Financial Conduct Authority (FCA) also report a legacy constraint on machine learning (ML): “The greatest constraint to ML adoption and deployment is legacy systems” (Bank of England, 2022).

Banks need to assess data readiness, whether the data a pilot needs is accessible, owned, and of usable quality, before a pilot starts.

In PwC’s 2026 survey of financial services executives, 41% named fragmented or low-quality data as the biggest barrier to scaling AI, more than any other issue (PwC, 2026). Sensitive customer data adds a second constraint: privacy, information security, and legal teams decide which fields a model may see, where processing may happen, and whether a vendor may retain or train on the data, and a pilot built on masked or synthetic data (stand-in data with real values hidden or invented) has to prove itself again once it meets real customer records.

That’s why so banks should answer these questions before the pilot begins:

• Which systems of record hold each input, and do they expose APIs or only screens?
• Who owns each data field and approves its use for this purpose?
• What is the data quality across the full production population?
• Which privacy constraints apply, including data residency and vendor retention?
• How will outputs write back to the system of record?

Some of the work an AI tool must support never reaches an event log, the automatic record that an application writes of its own activity. Examples include mainframe green screens, virtual desktop infrastructure (VDI) sessions, spreadsheets, and handoffs between applications that share no data layer. Event-log approaches show only the activity that instrumented systems record, while desktop-level observation, designed so that privacy protections are built in from the start (privacy by design), adds the cross-application handoffs, manual workarounds, and exception paths that those logs miss.

Skan AI captures this activity through Skan Virtual Assistant (Skan VA), a privacy-safe desktop agent that captures how work is done across mainframes, VDI, legacy desktop and web applications, spreadsheets, and office tools. From that capture, it builds end-to-end process flows into the Context Graph of Work, a continuously updated model of how work runs. It follows privacy by design: the observation covers work and processes, not individuals and Skan PRISM redacts protected information at the pixel level (it masks sensitive data on screen) inside the bank’s environment.

No owner, no KPI means no ROI

A regulated bank cannot put an output into production if nobody owns it, because supervisors and auditors ask who approved the model, who reviews its decisions, and who answers when it errs. In 2024, then Acting Comptroller Michael Hsu warned that AI creates “greatly diffused accountability” (OCC, 2024), and interagency third-party guidance places ultimate responsibility for oversight with the board, which holds management accountable (Federal Reserve, 2023). A pilot that reaches production needs these roles filled by name:

• Business owner: Owns the P&L-linked KPI and the decision to scale or stop.
• Model owner: Answers for the model’s performance, its limitations and its use within policy.
• Independent validator: Challenges the design and outcomes and holds no stake in the result.
• Exception owner: Resolves the cases the model escalates and decides when a reversal changes the procedure.

Most pilots report model accuracy, precision, or hours saved in a test environment, while the board asks about cost per case, loss rates, false positives worked, cycle time, and capacity. McKinsey’s March 2025 survey found that, among the adoption practices it tested, tracking well-defined key performance indicators (KPIs) for generative AI solutions had the most impact on the bottom line (McKinsey, 2025).

Set the KPI, the measurable business result the pilot must move, before the pilot starts and tie it to a P&L line the business owner already reports. Then take the baseline, meaning pre-pilot performance, from current operations so the post-pilot result can be compared with real performance.

Build, buy, or partner: Which pilots survive?

Purchased and partnered tools reached deployment more often in MIT’s cross-industry study, in which MIT NANDA reports that external partnerships with learning-capable, customized tools reached deployment about 67% of the time, compared with about 33% for internally built tools (MIT NANDA, 2025). The authors caution that these figures are self-reported, that the gap may reflect organizational capability more than the sourcing choice, and that the correlation does not prove causation. The study covers custom generative AI across industries, and no source reviewed compares production success for vendor-built and internally built AI in banks.

A bought tool still carries production costs that the pilot budget rarely covers:

• Mainframe and core integration: Connectors, data mapping and write-back to systems of record that the vendor demo never touched.
• Third-party risk review: Due diligence on the vendor’s financial condition, information security, operational resilience, subcontractors and audit rights, scaled to the risk of the activity.
• Validation of the bank’s own use: Testing the product on the bank’s population and documenting any customization.
• Procurement and contracting: Data-use terms, audit rights, exit and transition plans, and service levels, all negotiated before production.

The 2023 interagency guidance states that relying solely on prior experience with a vendor is not an adequate proxy for due diligence and that using a third party does not diminish the bank’s responsibility (Federal Reserve, 2023).

The revised model risk guidance acknowledges that vendors may not provide their underlying code, data, or methodology, and still says “the principles of model risk management remain applicable” (Federal Reserve, 2026). Because the agencies proposed replacement third-party risk management guidance in September 2026 (FDIC, 2026), contracts signed now should leave room for the final text.

How does failure differs by use case?

Each core banking use case stalls at a different point, because each faces a different reviewer:

• Fraud detection: Predictive ML fraud models already run in production at large banks. Commonwealth Bank of Australia says its fraud analytics team reviews and approves new detection rules before they go live (CommBank, 2026). Fraud pilots tend to fail on monitoring. ECB supervisors expect banks to monitor model performance continuously, detect drift, and have escalation and remediation processes ready for when models behave unexpectedly (ECB, 2026).
• AML monitoring: The sticking point is proof. GAO reported that examiners wanted rules-based and intelligent systems run in parallel, to show that the new system detects the same unusual activity, because intelligent systems lack transparent rules that examiners can review (GAO, 2024). The HKMA states that banks “must be satisfied that the abnormalities identified can be explained before the alerts are cleared or closed” (HKMA, 2023). US agencies said in 2018 that pilot programs alone should not subject banks to supervisory criticism, even when the pilots fail (FinCEN, 2018).
• Credit underwriting: Pilots fail on reason codes. Regulation B requires creditors to give specific reasons for adverse action, and those reasons must relate to factors the system actually scored (Regulation B, 2026). That requirement stands even though the CFPB withdrew its complex-algorithm circular in May 2025 (Federal Register, 2025). For EU lending, the AI Act classifies credit scoring as high-risk, with obligations applying from 2 December 2027 (EUR-Lex, 2026). McKinsey found that no bank in its credit study had fully deployed gen AI for synthesizing information for credit decisions (McKinsey, 2025).
• Know Your Customer: Identity controls now face AI-generated attacks. FinCEN has observed suspected deepfake media used to get around identity verification and authentication (FinCEN, 2024). As a result, KYC pilots must show that their controls withstand manipulated documents and video, and they must clear privacy review for the customer data involved.

Generative AI and predictive ML pilots also fail at different points. The Financial Stability Board (FSB) notes that outputs from a large language model (LLM), the software behind most generative AI text tools, are typically unstructured text, which makes it harder to calculate error rates and conduct outcomes analysis (FSB, 2024). The OCC reports that banks limit generative and agentic AI to specific use cases with guardrails and human-in-the-loop accountability, meaning a person reviews and answers for the output (OCC, 2026). The main differences are as follows:

Scale, fix or stop: A diagnostic for stalled pilots

A stage gate scored at the end of every pilot tells a bank whether to scale, fix, or stop it, and most chief information officers (CIOs) still lack guidelines for making that call. Gartner’s 2026 CIO survey found that only 31% of CIOs have guidelines for deciding when an AI pilot should not move to production (Gartner, 2026), and fewer still have guidelines for abandoning low-value AI initiatives once they are in production. Score each pilot against these criteria:

1. Business KPI hit: Did the pilot move a P&L-linked metric against a baseline drawn from current operations, at a volume representative of production?
2. Risk sign-off path: Does a named path run through model risk, compliance, legal and third-party review, with each reviewer’s evidence requirements known and costed?
3. Integration cost: Has the team estimated and funded the cost of reading from and writing back to core systems, including mainframe screens and batch interfaces?
4. Data readiness: Are the production data sources owned, approved for this use, and of known quality across the full population?

Then apply the result:

• Scale: All four criteria pass. Move to production with validation complete and the business owner accountable for the KPI.
• Fix: The KPI passes, but one or more of the other criteria fails. Fund the specific gap, such as a validation workstream, a data approval or an integration estimate, and set a fixed date to score the pilot again.
• Stop: The KPI fails on representative data, or no achievable sign-off route exists for the decision the model supports. Close the pilot, record the reason, and reassign the team.

The observed record of how your bank’s work actually runs today, captured in Skan AI’s Context Graph of Work, provides the foundation you need to transform your operations and power hybrid teams of people and agents. Get a demo to see the process and operating outcomes relevant to your organization so you can score your stalled pilots against it before the next funding cycle.