The cloud is fast becoming the de facto environment for artificial intelligence (AI), with Gartner forecasting that up to half of all cloud computing resources will be dedicated to AI workloads by 2029. In particular, use of hybrid clouds — for example, to accommodate the ever-evolving regulatory landscape, reduce latency, and increase agility — is seeing an uptick in adoption rates, with 90% of organizations expected to adopt these environments by 2027.
However, IT leaders should be aware that the intersection of hybrid cloud and AI integration is a complicated one. Data flows between workloads can be challenging and, especially as the organization scales, IT teams will likely see escalating complexity that can make it difficult to realize business value.
To address these issues, we asked the CIO Experts Network — IT and business decision-makers, technologists, and influencers — this question: How are AI and hybrid cloud trends affecting application and infrastructure integration, and what are practical strategies organizations can take to reduce integration challenges? Here is their advice.
Hybrid cloud and AI: A problem of data decentralization
Data should be the focus of hybrid cloud and AI integration, says Arsalan Khan, an advisor and writer on AI. “The primary goal of application and infrastructure integration is to enable the efficient and secure sharing of organizational data,” Khan says. “When data cannot move freely, it drains resources and hampers effective decision-making.”
The core problem resides in the conflicting natures of how data is distributed in hybrid cloud and AI’s data requirements, according to Isaac Sacolick, bestselling author and president of StarCIO. “Hybrid clouds and software-as-a-service environments decentralize information, while AI benefits by having centralized, clean, and real-time data integrations,” he says.
Not only do hybrid clouds decentralize data, they also introduce significant security and compliance challenges:
“AI and hybrid cloud adoption are dramatically increasing integration complexity by creating multilayered environments where data and applications must seamlessly operate across on-premises systems, private clouds, and public cloud services. AI implementations also pose significant security and data governance challenges, as they often require access to sensitive data across multiple environments, creating expanded attack surfaces and potential regulatory compliance issues.” — Chris Selland, independent consultant, analyst and lecturer, Entrepreneurship & Innovation at Northeastern University D’Amore-McKim School of Business
Despite these challenges, hybrid cloud provides critical capabilities for AI. “On one hand, AI workloads demand fast, scalable environments and hybrid cloud offers that flexibility,” says Will Kelly, a writer who focuses on AI and the cloud. “However, that flexibility also adds layers of complexity, especially when connecting legacy systems, new AI platforms, and multi-cloud environments.”
Integration from the start: Governance and APIs
The keys to effective integration include, according to the experts, making it a priority from the beginning and involving more than just technical teams:
“I’ve found that companies do better when they keep things simple and plan ahead. This requires time and effort thinking about how the data ultimately moves between systems. Often, the more platforms you start to add, things get complicated fast. Another smart move is to get your IT and business teams working together from the start, because it’s easier to spot problems before they happen and make sure the systems support the real needs of the people using them.” — Scott Schober, president and CEO of Berkeley Varitronics Systems
“Organizations should be doubling down on their data strategies by updating data governance to include AI policies, using data fabrics to centralize information access, implementing data cleansing pipelines, and using data security posture management tools.” — Sacolick, from StarCIO
With strong policies and enforcement in place to ensure compliance and security, CIOs can further facilitate integration by making APIs a foundational part of their infrastructure.
“Organizations can reduce integration challenges by implementing API-first architectures that abstract underlying complexity and provide consistent interfaces regardless of where components are hosted. Adopting containerization with orchestration tools like Kubernetes enables workload portability while minimizing environment-specific dependencies. Investing in integration platforms that offer prebuilt connectors and low-code integration capabilities can accelerate connection-building while reducing technical debt.” — Selland, at Northeastern
Ironically, AI itself can help remove much of the complexity of integrating applications across AI workloads and hybrid cloud. As Kumar Srivastava, CTO at Turning Labs, says: “AI and hybrid cloud make it important and more complex to create a unified dataset that is ready for AI-powered processing using large language models. Organizations should leverage AI to determine unification strategies, automate this processing, and then intelligently use this data to prompt LLMs.”
Vivek Singh, senior vice president of IT and strategic planning at PALNAR, notes that AI is particularly useful “for automating data mapping, orchestration, and error detection.”
Skills and centers of excellence
Finally, CIOs shouldn’t neglect the human element of integration. Technology evolves fast, and IT employees will need support to ensure their skills remain current, said Gene de Libero, principal at the marketing technology consultancy Digital Mindshare. “Organizations should invest in skills development and create integration centers of excellence to standardize approaches,” de Libero says.
The integration challenges posed by hybrid cloud and AI deployments are daunting, but they are far from insurmountable. Success hinges on taking a proactive, holistic approach that begins with robust data governance policies and API-first architectures, while leveraging containerization and integration platforms to abstract underlying complexity. Organizations that invest in these foundational elements from the outset — rather than retrofitting integration solutions later — will find themselves better positioned to realize the transformative potential of AI across their hybrid environments.
The solution to AI-hybrid cloud integration complexity lies not just in technology, but also in the strategic alignment of people, processes, and platforms. By fostering collaboration between IT and business teams, investing in employee skill development, and leveraging AI itself to automate data mapping and orchestration tasks, enterprises can turn what initially appears to be a barrier into a competitive advantage.
Those who embrace this comprehensive approach will overcome integration challenges and build the foundation for sustained innovation as both AI and cloud technologies continue to evolve.
Visit Rocket Software, which provides a comprehensive suite of solutions that seamlessly connect hybrid clouds and on-premises systems.