Softserve company logo
KarriereSearchContact Us

Hot Links

  • Home
  • Branchen
  • Services und Kompetenzen
  • Ressourcen
  • Presse
  • Über Uns
  • Kontakt
  • Karriere
  • Subscribe to Updates

Kontakte

  • Hauptsitz Austin

    201 W 5th Street Suite 1550 Austin, TX 78701

    +1-512-516-8880

    Toll Free:
    +1-866-687-3588

  • Privacy Notice - 
  • Terms and Conditions - 
  • Information Security - 
  • Sitemap - 
  • Search - 
  • Accessibility statement - 
  • LInkdn Link with Icon
  • Youtube Link with icon
  • Facebook Link with Icon
  • Instagram Link with icon
  • TikTok Link with icon
  • Twitter Link with icon
  • Soundcloud Link wiht icon
  • Bluesky Link with icon

© Copyright 2026 SoftServe Inc.

scaling-ai-in-legacy-it-header.png

Overcoming barriers to scaling AI in legacy enterprise environments

28 Okt 2026
Bohdan Khomych

ShareShare the article

Scaling AI in Legacy IT

  • Dinner discusses ways to overcome legacy IT challenges to scale AI deployment
  • Challenge to embed AI in existing models in ways that deliver business value at scale
  • Organisational barriers compound technology challenges
  • More flexible governance and early establishment of guardrails seen as key to success
  • Essential to have trusted and governed data sources
  • Bake in human oversight and accountability

A common reality facing large enterprises that are eager to accelerate AI adoption and move beyond experimentation, is that many remain constrained by legacy technology estates, technical debt, governance requirements, and organisational resistance to change.

This was the central theme at a recent executive dinner we hosted with Google Cloud at London’s House of Lords to examine practical ways these challenges could be overcome. While participants represented different industries and levels of AI maturity, there was broad agreement that the obstacle is no longer access to AI technology itself. The real challenge is embedding AI into existing operating models in ways that deliver measurable business value at scale.

scaling-ai-in-legacy-it-image-1.png

Legacy obstacles

A frequently aired frustration was the extent to which legacy infrastructure continues to limit AI adoption. Participants described estates characterised by ageing platforms, fragmented data environments, accumulated technical debt, and highly regulated delivery processes. In several organisations, substantial effort is still being directed towards foundational modernisation initiatives before AI can be deployed effectively.

One organisation is currently undertaking a major data platform migration programme, while others were trying to simplify environments that have become increasingly complex from years of acquisitions, changing business priorities, and overlapping technology investments.

Technology challenges were also often compounded by organisational barriers. Several attendees observed that change programmes frequently encounter resistance from stakeholders seeking to preserve existing processes, systems, or areas of responsibility. As a result, transformation initiatives were as much a cultural challenge as a technical one.

scaling-ai-in-legacy-it-image-2.png

Governance as enabler

Governance emerged as one of the most debated topics of the evening. Some participants described AI adoption within their organisations as resembling a "Wild West" environment, where employees independently experiment with AI tools, creating concerns around data security, compliance, intellectual property, and shadow IT.

At the same time, there was recognition that many traditional governance frameworks have become a barrier to innovation. Existing approval processes were largely designed for conventional software development and struggle to accommodate the pace of AI-driven experimentation.

A consistent view was that governance needs to move further left within the delivery lifecycle. Rather than introducing controls late in the process through lengthy approval gates, organisations should establish guardrails earlier so that teams can innovate safely without creating unnecessary delays.

Several also noted that organisations increasingly need different governance models for experimentation versus enterprise-wide deployment. Applying the same controls to both can make adoption prohibitively slow and discourage innovation altogether.

scaling-ai-in-legacy-it-image-3.png

Beyond central AI teams

One particularly interesting discussion focused on how AI adoption evolves within organisations. Several noted that central AI teams are effective during the early stages of adoption, helping connect business problems with technical expertise and incubating initial use cases. However, as demand grows across functions, the same centralised structure can become a bottleneck.

The consensus was that long-term success requires ownership to move closer to operational teams, supported by shared governance, education, reusable frameworks, and common measures of value.

This mirrors a broader challenge facing many organisations today. Scaling AI is less about creating more use cases and more about enabling hundreds of teams to deliver AI-enabled outcomes consistently and safely.

scaling-ai-in-legacy-it-image-4.png

Search for meaningful ROI

Perhaps the most striking observation was the gap between enthusiasm for AI and evidence of measurable business outcomes.

There was widespread belief that AI will ultimately reduce costs and improve productivity. However, when asked to identify production AI use cases that had delivered material cost reductions, participants struggled to provide examples.

In many organisations, early metrics focused on time saved. However, more mature adopters reported that this quickly became an inadequate measure of success.

The discussion shifted towards broader questions:

  • Has AI increased delivery throughput?
  • Has it accelerated time-to-market?
  • Has it reduced operational risk?
  • Has it enabled teams to achieve more without increasing headcount?

Attendees recognised that reclaimed capacity creates a management decision. Organisations can either reinvest that capacity into growth and innovation or seek direct cost savings. Most organisations represented at the table appear to be pursuing the former, with relatively few examples of explicit headcount reduction attributable to AI.

This highlights what several participants referred to as an "AI value realisation gap" where expectations remain high, but demonstrable financial outcomes remain difficult to quantify.

scaling-ai-in-legacy-it-image-5.png

Agentic AI challenges

Discussion around agentic AI highlighted that organisations are beginning to encounter a new class of governance and engineering challenges. While agent-based systems can automate increasingly complex tasks, they also create new risks around autonomy, access, accountability, and output quality.

One participant shared an example where an AI-powered writing task incorporated information from historical documents that were not intended to be referenced. While the output appeared plausible, it drew upon information outside the intended scope of the assignment.

This reinforced a common view that successful agentic implementations require:

  • Clearly bounded tasks.
  • Explicit access controls.
  • Trusted and governed data sources.
  • Deterministic validation mechanisms.
  • Human oversight and accountability.

There was broad agreement that the industry is still establishing best practices in this area.

scaling-ai-in-legacy-it-image-6.png

Data quality concerns

Several argued that trusted data is becoming the primary enabler of AI success. As business users gain access to increasingly powerful AI tools, expectations around delivery speed are changing dramatically. Traditional development cycles measured in weeks are increasingly being challenged by users who can prototype applications and workflows themselves in a matter of hours.

The emerging consensus was that central technology functions should focus less on controlling every tool and more on providing trusted datasets, shared definitions, secure data access, and reusable services. This allows business teams greater flexibility while preserving consistency and governance.

In many cases, the quality and accessibility of organisational data were seen as more important constraints than access to AI models themselves.

scaling-ai-in-legacy-it-image-7.png

AI impact on engineering

A particularly thought-provoking discussion centred on how AI is reshaping engineering roles. Some participants suggested that the differentiating skill in the future may not be technical implementation, but judgement.

As AI becomes increasingly capable of generating code, documentation, and even architectural options, greater importance will be placed on defining problems, validating outputs, making trade-off decisions, and accepting accountability for outcomes.

Several attendees challenged the assumption that junior employees are necessarily most vulnerable to AI disruption. Instead, the view emerged that roles built primarily around execution may be more exposed than roles requiring judgement, context, decision-making, and ownership.

At the same time, many organisations acknowledged a need to openly discuss workforce concerns and ensure that adoption programmes focus as much on capability development as technology implementation.

scaling-ai-in-legacy-it-image-8.png

Prototype to production gaps

Another recurring point involved the growing disconnect between what AI can demonstrate and what organisations can actually deploy.

AI-generated prototypes can now be produced remarkably quickly, creating expectations that production deployment should immediately follow. However, experienced practitioners emphasised that the majority of effort still lies in integration, testing, security, operational support, compliance, and maintainability.

Several participants highlighted that successful organisations are learning to assess not only whether an AI solution can be built, but whether it can be safely operated at enterprise scale.

As one attendee observed, many organisations continue to apply new AI technologies to existing operating models without fundamentally rethinking underlying processes. The largest opportunities may ultimately come not from automating existing workflows, but from redesigning them altogether.

scaling-ai-in-legacy-it-image-9.png

Build or buy?

The rise of agentic engineering is also beginning to reopen discussions around software purchasing strategies. Capabilities that would previously have required substantial engineering resources may now be achievable with much smaller teams supported by AI development tools.

As a result, some organisations are starting to reassess elements of their SaaS portfolios and asking whether certain capabilities should continue to be purchased, or whether they could be developed and maintained internally.

While no consensus emerged, the discussion highlighted that AI may influence technology strategy well beyond productivity gains, potentially reshaping future build-versus-buy decisions.

scaling-ai-in-legacy-it-image-10.png

Key Takeaway

The overwhelming conclusion from the evening was that AI's greatest challenges are no longer technological.

Most organisations already have access to AI platforms, models, and tools. The barriers to scaling adoption now lie in modernising legacy estates, reducing technical debt, establishing pragmatic governance, creating trusted data foundations, developing workforce capabilities, and proving measurable business value.

The most mature organisations appear to be shifting their focus away from isolated proofs of concept and towards the harder challenge of changing how the business operates. Success increasingly depends not on deploying more AI, but on building the organisational foundations required to turn AI capability into sustainable business outcomes at scale.

ShareShare the article

Authors

Don't want to miss a thing?

Subscribe to get expert insights, in-depth research, and the latest updates from our team.
SUBSCRIBE
  1. Home
  2. Insights
Bohdan Khomych

Bohdan Khomych

Associate Director of R&D Products

More from this author