How to set up an AWS CI/CD pipeline that your team can safely operate

Last substantive review: August 2026.

A production CI/CD pipeline is not a diagram of boxes between Git and AWS. It is the controlled path by which a reviewed change becomes the exact software running for customers. In an established codebase, the work starts with release behavior, permissions, and recovery. Choosing CodePipeline, GitHub Actions, CodeBuild, or a deployment service comes after that.

This guide is for engineering leaders who already have an AWS application and need a delivery path their team can operate. The goal is not to force a platform migration. It is to reduce the manual steps, hidden credentials, unrepeatable builds, and rollback uncertainty that make a backlog harder to clear safely.

Baseline the current delivery system before replacing it

Follow one recent production release from merge to customer traffic. Record every human handoff, script, console change, approval, credential, artifact, and verification step. Note where the process depends on one engineer’s memory. Run the current rollback or restore procedure in a safe environment. If it cannot be performed from the documentation, the first pipeline milestone is recovery, not deployment speed.

Capture a small delivery baseline. AWS describes deployment frequency, lead time for changes, change failure rate, and time to restore service as useful measures in its platform measurement guidance. Use the team’s own numbers. The baseline is there to show whether the new system makes delivery safer and easier; it is not a score to compare with an unrelated company.

Inventory the architecture at the same time:

• Source repositories, protected branches, and release tags.
• Build commands, language runtimes, lockfiles, and private package sources.
• Unit, integration, contract, migration, and end-to-end tests.
• Development, staging, production, and disaster-recovery environments.
• AWS accounts, regions, networks, compute targets, registries, and databases.
• Current IAM users, roles, access keys, secrets, and break-glass paths.
• Release alarms, dashboards, incident procedures, and on-call ownership.

This map exposes constraints a greenfield tutorial misses. A nightly batch, a stateful worker, a mobile API contract, or a database migration may matter more than which AWS service orchestrates the stages.

Choose the smallest architecture that fits the codebase

A reliable AWS pipeline needs a source trigger, a build environment, an artifact store, gates, a deployment mechanism, and operational feedback. Those pieces do not all have to come from one vendor. CodePipeline can orchestrate CodeBuild and AWS deployment targets. GitHub Actions can use short-lived AWS credentials and invoke the same infrastructure and deployment APIs. An existing Jenkins or GitLab setup may remain sensible if the team already operates it well.

Choose based on the target and the operating team:

• For Lambda, publish a version and shift an alias rather than mutating live code in place.
• For ECS, build one container image and promote its digest through environments.
• For EKS, keep cluster and workload deployment responsibilities explicit; do not hide every control inside one oversized workflow.
• For EC2, decide whether in-place delivery is acceptable or whether an immutable or blue/green pattern is worth the additional capacity.

There should be one normal path to production. Emergency access can exist, but it should be narrow, time-limited, logged, and followed by reconciliation into code. If engineers routinely bypass the pipeline because it cannot handle database changes or urgent fixes, the pipeline is incomplete.

Put infrastructure in version control without rebuilding the world

Infrastructure as code gives the release process a reviewable desired state and an audit trail. It does not require an immediate rewrite of every existing resource. Start with the resources the pipeline changes or depends on: IAM roles, artifact buckets, build projects, registries, deployment groups, alarms, and environment configuration. Import or model existing resources carefully, then expand coverage as the team understands ownership and drift.

AWS recommends storing templates in version control, reviewing and testing them, using change sets, checking drift, protecting sensitive parameters, and configuring rollback triggers in its CloudFormation best practices. Terraform or CDK can serve the same operating goal if those are already the team’s tools. The important choice is that changes are reviewed, repeatable, and tied to an owner.

Separate infrastructure with different lifecycles. A network or production database should not be replaced because an application image changed. Protect stateful resources from accidental deletion. Preview changes before applying them, and run policy checks on the plan or template. For database work, use an expand-and-contract approach: deploy backward-compatible schema additions, release code that can work with old and new forms, migrate data, and remove old structures only after the rollback window closes.

Use short-lived identity and least privilege

Do not store long-lived AWS access keys in repository secrets when the CI provider can federate to AWS. With GitHub Actions, use OpenID Connect to request a short-lived AWS role session. Scope the role’s trust policy to the expected audience and to a specific organization, repository, branch, tag, or protected environment. AWS’s IAM guidance for GitHub OIDC warns that an unscoped subject condition can let repositories outside your control assume the role.

Create separate roles for build, non-production deploy, production deploy, and infrastructure changes. Give each role only the actions and resources needed for its stage. Scope iam:PassRole to the exact workload roles a deployment service must pass. A build that compiles and pushes an image does not need permission to edit a production load balancer. A production deploy role does not need to read every secret in the account.

Keep human access separate from machine access. Production changes should identify the pipeline execution and artifact, not appear as an unexplained administrator session. Maintain an audited break-glass role for incidents and test access removal. Review policies after the pipeline is stable; temporary broad permissions used during setup have a habit of becoming permanent.

Separate environments and promote the same artifact

Use environment boundaries that match the risk. AWS recommends separating production workloads from development and test workloads in different accounts in its multi-account best practices. Smaller teams may phase this in, but production should at least have distinct identities, data, controls, and deployment permissions.

Build once, then promote the same immutable artifact through staging and production. Do not rebuild from the same branch for each environment; dependency resolution, timestamps, or base images can produce different bytes. Environment-specific values belong in configuration and secrets, not in separate application builds. Record which artifact digest and infrastructure revision each environment runs.

Staging only provides evidence if it resembles production where the risk lives. Match runtime versions, deployment mechanics, network dependencies, and schema behavior. Use synthetic or protected test data rather than copying sensitive production records by default. If a full-scale replica is too expensive, document what staging does not test and add production canaries or other controls for that gap.

Make artifacts identifiable and verifiable

A release should answer: which commit produced this artifact, which build ran, which dependencies were resolved, which checks passed, and which digest reached production? Pin build images and important tooling versions. Commit lockfiles. Fail the build when expected generated files or dependency metadata are missing. Tag artifacts with a human-readable release identifier, but deploy containers by digest.

For container workloads, enable ECR tag immutability so a release tag cannot silently point to different bytes. AWS documents the setting in ECR tag immutability guidance. Scan images as part of the release policy; ECR image scanning can use basic scanning or enhanced, continuous findings through Amazon Inspector.

For higher-assurance workloads, attach a software bill of materials or attestation and sign the deployable image. AWS supports managed and manual container image signing through ECR and AWS Signer, as described in its image signing documentation. Signature verification must be enforced at or before deployment to have operational value. A signature stored beside an image but never checked is only evidence that signing happened.

Protect pipeline artifacts in S3 with encryption and restricted bucket access. AWS’s CodePipeline security best practices recommend KMS-backed server-side encryption for S3 artifacts and warn against placing secrets in action configuration, pipeline variables, or CloudFormation defaults where they may appear in logs.

Build gates that correspond to real failure modes

Every gate costs time, so each one should catch a named risk. A useful order is:

1. Formatting, linting, type checks, and fast unit tests.
2. A reproducible build that produces the release artifact.
3. Dependency, secret, image, and infrastructure policy scans.
4. Integration and contract tests against owned dependencies.
5. Database migration compatibility checks.
6. Deployment to staging with smoke and critical-path tests.
7. A production release condition based on risk, not habit.

Do not make a manual approval compensate for missing automation. Use approval when a person has specific information to assess, such as a high-risk migration, a customer communication, a change-window requirement, or an unusual security exception. Routine low-risk changes should be able to pass on evidence the pipeline already collected.

Define how scan findings behave. Blocking every vulnerability regardless of reachability or severity teaches teams to ignore the gate. Letting every finding through makes the scan ceremonial. Set a policy for severity, exploitability, age, ownership, and documented exceptions. Expire exceptions automatically and keep the decision with the release record.

Keep secrets out of build configuration and output

Build workloads should retrieve only the secrets they need at runtime. AWS strongly discourages plaintext CodeBuild environment variables for sensitive values and supports references to Systems Manager Parameter Store or Secrets Manager in its CodeBuild environment variable reference. Grant the build role access to specific secret ARNs, not the whole secret store.

Prevent shell tracing and debug output from printing credentials. Masking in the CI interface is a useful backstop, not the primary control. Keep production application secrets away from build jobs unless a test truly requires them. Rotate exposed values through a rehearsed procedure, and make secret-access events available to the security team.

Connect deployment health to observability

A successful API response from the deployment service does not prove the application is healthy. Carry the release ID from pipeline execution into the application’s logs, metrics, traces, and error reports. Build a release view that shows error rate, latency, saturation, availability, job failures, and critical business checks for the new version.

Use health checks that exercise the dependency path customers need, without turning every temporary dependency issue into a restart loop. Distinguish deployment alarms from long-term service alerts. Deployment alarms should react quickly enough to stop exposure, with thresholds tested against normal traffic. The on-call engineer should be able to move from an alarm to the pipeline execution, commit, artifact digest, changed infrastructure, and rollback command.

Use progressive delivery and test the rollback

Choose a deployment strategy based on state and customer impact. For ECS services using CodeDeploy blue/green deployments with an Application Load Balancer, traffic can shift using canary, linear, or all-at-once patterns; with a Network Load Balancer, AWS supports all-at-once only. AWS’s ECS blue/green documentation covers test traffic, alarms, traffic shifting, and rollback settings. Give the new version a bake period long enough to expose the failures your metrics can detect.

A rollback must cover more than application code. Confirm whether the previous version can read the new schema, whether configuration changed, whether queued messages changed shape, and whether an external side effect can be reversed. For irreversible data work, stop the release or use a forward-fix plan with a tested restore path. Keep the prior artifact available and verify that its dependencies have not disappeared.

CodePipeline supports manual stage rollback and automatic rollback on failure, but there is an important boundary: the target execution must have started under the current pipeline structure version. AWS states that constraint in its stage rollback documentation. A substantial pipeline edit can therefore remove an old execution from the built-in rollback path. Keep an independent deployment record and a documented command for deploying a known-good artifact.

Run a rollback drill before calling the pipeline production-ready. Trigger a controlled failure, watch the alarms, roll traffic back, and confirm that service and data are usable. Time the exercise and fix the steps that depend on tribal knowledge.

Design the handoff as part of the build

The pipeline is not finished when a specialist can operate it. It is finished when the product team can explain it, change it, and recover it. Deliver the architecture diagram, repository map, role matrix, environment inventory, release runbook, rollback runbook, secret rotation procedure, alarm ownership, cost dashboard, and known limitations. Pair with the engineers who will own normal releases and the on-call response.

Give every component an owner. Review failed builds, slow stages, flaky tests, permission denials, exceptions, and unused infrastructure on a regular cadence. Track the original delivery baseline and look for trends. A pipeline that grows slower and noisier each month will eventually be bypassed.

If the backlog is blocked by manual AWS releases, broad IAM, brittle builds, or a rollback nobody trusts, Horizon Labs can place senior platform engineers at $150–$200 per hour to work inside the existing codebase and operating constraints. Review how that engagement differs from a broader delivery team in our staff augmentation versus managed services guide, or contact Horizon Labs with the current stack and the release failure you want removed.

Frequently asked questions

Can we add AWS CI/CD without replacing our existing stack?

Yes. Start by automating the current build, artifact, deployment, and verification path. Replace a tool only when it blocks a required control or creates an operating burden the team has agreed to own.

Do we have to use AWS CodePipeline?

No. CodePipeline is one orchestration option. GitHub Actions, GitLab, Jenkins, or another system can be appropriate if it uses short-lived AWS identity, produces a verifiable artifact, enforces the required gates, and leaves an auditable deployment record.

How should GitHub Actions authenticate to AWS?

Use OpenID Connect to assume a short-lived IAM role. Scope the trust policy to the expected GitHub organization, repository, branch, tag, or protected environment, and give the role only the permissions required for its pipeline stage.

What should trigger an automatic rollback?

Use tested signals tied to customer health, such as error rate, latency, failed critical checks, unhealthy tasks, or a deployment failure. Make thresholds specific to the service and confirm that schema and external side effects are compatible with rollback.

Should development, staging, and production use separate AWS accounts?

AWS recommends separating production from development and test workloads at the account level. The exact account layout should match the team’s security and operating capacity, but production needs a clear identity, permission, data, and blast-radius boundary.