Scaling AI in Financial Services: Why the hardest part of scaling AI isn't the technology

That question surfaced repeatedly last month in New York, where SoftServe and Google Cloud convened 30-plus senior financial services and insurance leaders to move the conversation from experimentation to enterprise impact. The format was peer discussion, not presentation, and we opened by asking each leader the one question their own team cannot yet answer. This piece captures the candor that followed, and the sentiments in the room. One idea ran through every conversation: scaling AI is not a technology problem. Almost everyone already has the models, the cloud, and the pilots. What separates the firms pulling ahead from those stuck in proof-of-concept is less visible and more demanding: governance, a repeatable method for proving value, and a workable relationship between people and their agents.

One executive at a global Wall Street bank captured it. Their wealth advisor desktops are making real strides with AI, while the same firm has stalled pushing AI into legacy operational and back-office workflows. The blockers were not the models. They were siloed data, unsettled governance, and manual processes never documented in the first place. Even where the technology could do more, fiduciary and regulatory concerns capped what they were willing to deploy. Strong progress in the front office, stalled effort in the back, inside one firm.

That contrast is the real story. It is rarely the model; it is the conditions around it, whether the data is reachable, whether governance and risk are settled, and whether the work is understood well enough to hand any of it to a machine. Those conditions differ by firm. The largest institutions have guardrails and are pushing on autonomy; many mid-market firms have almost none and are still deciding who may build what. So instead of prescribing, here is what we heard.

Governance is the accelerator

There is a persistent belief that governance slows AI down. Leaders told the opposite story. When teams know what they can build, what data they can use, and where the boundaries sit, they move faster. In financial services that governance is inseparable from the rules the industry already lives by: model risk management, examiner expectations, and a moving perimeter where the EU AI Act now treats credit and insurance underwriting as high-risk. Add the fiduciary duty triggered the moment AI reaches an advisor's desktop, and governance becomes a license to scale rather than a brake on it. Firms still without that foundation struggle most to win buy-in, because they cannot yet explain the risk with confidence.

Maturity sets the agenda. The largest firms want to extend existing controls to autonomous agents; earlier firms are still deciding what to put down first and who owns it. The sharpest tension is over who gets to build. Most keep agent development inside IT, which contains risk and throttles the experimentation that produces value. The opportunity is to let business and developer teams build within guardrails they help set.

The digital employee is the new hire

The digital employee, an autonomous agent working alongside people, is already in production at firms in the room. Like any new hire, it has to earn trust and be onboarded into how the team actually works. Both are still unsolved.

Trust comes first, because accountability is open. How much autonomy can an agent earn, and how do you know when it has? How do you evaluate its output against a human’s? The resistance here is legitimate. It is not fear of job loss; it is the absence of a framework for trust, and you cannot hand autonomy to something you cannot yet measure. Trust is built one scoped responsibility at a time.

Integration is the half firms underestimate. Dividing work between people and agents is easy to say; recombining it into one coherent workflow is the hard part. One stakeholder from a custody bank described running thousands of agents, most still copilots that are not integrated into core operations, held back by limited visibility into how work gets done today, a strained business-IT operating model, integration gaps, and risk constraints from legal. The firms getting this right redesign the process around the blend of human and machine, instead of dropping an agent into a workflow built for people and hoping it holds. Token economics adds urgency: as the cost of running AI becomes a visible line item, leaders must decide which tasks belong to humans, which to agents, and what each costs to run.

Method is the multiplier

The biggest obstacle is the absence of a repeatable method. Most firms have implemented something; few can measure whether it worked. Leaders named the same three blind spots:

• No clear way to identify where AI adds the most value
• No reliable way to evaluate agent output against human output
• No confident ROI narrative that survives board scrutiny

Underneath sits the least glamorous constraint: the enterprise data foundation. The Wall Street bank's back-office stall was a data and process problem before it was an AI problem. You cannot evaluate output, measure ROI, or widen autonomy on data you cannot trust or trace, or on workflows no one has documented. For some firms the limit is structural, a fifteen-year-old core or data they cannot move for regulatory reasons; method does not erase that, it gets a firm through it one governed dataset at a time. Two patterns point the way: fleets of small, purpose-built agents that are easier to govern and measure than a single do-everything agent, then orchestrated into a coherent whole; and financial discipline, where cost transparency sits alongside security as a first-class concern.

From experimentation to impact

Put it together and an operating model emerges: governance that accelerates, digital employees that earn autonomy and integrate cleanly, and a method built on trustworthy data, bounded agents orchestrated into a whole, and ROI defined up front. This is the path SoftServe walks with financial institutions, meeting each firm on its maturity curve (Assess, Architect, Activate, Scale) and grounded in governance you can stand behind. SoftServe is among the first firms certified to ISO 42001 and a global launch partner for Gemini Enterprise, with 60+ FSI accounts and 700+ Google Cloud-certified engineers. The technology is rarely the constraint; turning scattered pilots into a scaled, governed capability is the work.

Everyone in that room was wrestling with the same three questions, and where a firm sits on the curve shapes the answers. So, I will end where we began. What is the one question your team cannot fully answer yet? We are planning the next FSI Exchange and would like you in the room. If that question is on your mind, reach out and connect.