Extending DevOps Pipelines Across Hybrid Stacks with Open Source Tooling

DevOps teams in AWS-native organizations naturally gravitate toward CI/CD pipelines built on native services like CodePipeline, CodeBuild, and CodeDeploy. This is applicable for Azure and GCP-first organizations as well.

While these services integrate seamlessly within their respective ecosystem, any expansion beyond the core hyperscaler immediately exposes their  limitations— multi-environment management headaches that result in vendor-lock in.

Take pipeline definitions using CodePipeline's proprietary JSON structure. It creates workflows tightly coupled to AWS services which manifests through three critical dependencies:

• Build specifications in CodeBuild's buildspec.yml format contain AWS-specific environment variables and deployment configurations that don't translate to other CI systems
• Source control and artifact storage relies on CodeCommit with AWS-specific triggers, or S3 for artifact storage using CodePipeline's specific artifact format that other platforms can't directly consume
• Deployment orchestration uses CodeDeploy's AppSpec files and AWS-specific deployment configurations that assume AWS services as the only target environment

In instances where you have to support hybrid infrastructure, the DevOps teams must maintain separate CI/CD systems for different environments, leading to duplicated effort, inconsistent deployment practices, and increased operational complexity. 

When a critical application needs to run across AWS, Azure, and on-premises Kubernetes clusters, platform-specific CI/CD tools force teams to essentially build and maintain three different deployment pipelines.

Building Portable CI/CD Architectures

The key to successful hybrid CI/CD lies in adopting open-source tooling that provides consistent workflows regardless of the underlying infrastructure, such as GitHub, GitLab, ArgoCD, Flux, Ansible, OpenTofu, and Pulumi. Open-source solutions eliminate vendor dependencies through two architectural principles: provider-agnostic deployment mechanisms, and standardized artifact management.

Deployment mechanisms must abstract away infrastructure differences while maintaining the ability to leverage platform-specific capabilities when needed. Tools like Helm and Kustomize provide Kubernetes-native deployment approaches that work consistently across any Kubernetes distribution, whether it's running on AWS EKS, Azure AKS, Google GKE, or on-premises clusters.

For broader infrastructure management, configuration management systems like Ansible, Chef, or Puppet can orchestrate deployments across diverse environments using standardized playbooks and scripts. These tools provide identical deployment experiences across cloud and on-premises infrastructure while supporting environment-specific optimizations when needed.

Infrastructure as Code for Multi-Cloud Environments

Traditional IaC approaches using tools like AWS CloudFormation create infrastructure definitions with heavy reliance on platform-specific intrinsic functions and pseudo-parameters that cannot be translated to other environments.

Provider-agnostic IaC tools like OpenTofu and Pulumi address these limitations by implementing modular approaches that isolate provider-specific details behind consistent interfaces. Instead of exposing raw cloud provider resources, these tools enable teams to create abstraction modules—for example, a database module that presents consistent inputs like engine type, version, and high availability requirements while implementing the appropriate provider-specific resources underneath.

This modular approach allows application teams to define infrastructure requirements in provider-neutral terms while enabling deployment to any supported environment. A database module might provision AWS RDS in cloud environments, Azure Database for Azure deployments, or operator-managed PostgreSQL in Kubernetes clusters, all while presenting the same interface to consuming applications.

Learn more about building extending DevOps pipelines wusing open source tooling, along with all the other common infrastructure and service setups used by cloud-native organizations in our eBook: Architecting for Openness: A Guide for Avoiding Hyerscaler Lock-in.

‍