Zero-Downtime Blue/Green & Canary Traffic Shifting with CodeDeploy
Shifting live user traffic progressively (10% -> 50% -> 100%) across ALB target groups with automated CloudWatch error-rate rollback alarms.
1. Business Problem & Context
An online banking platform experienced customer-facing downtime whenever a new release had a hidden runtime bug. Standard in-place rolling deployments updated all instances simultaneously, exposing 100% of live users to 500 Internal Server Errors before engineers could manually intervene.
2. Requirements & Constraints
- Zero In-Flight Connection Drops: Use ALB connection draining (deregistration delay).
- Progressive Canary Shifting: Route 10% of real user traffic for 10 minutes to test stability.
- Automated Instant Rollback: If HTTP 5XX errors exceed 1%, abort and revert 100% of traffic to Blue within 30 seconds.
3. Architecture Overview & Data Flow
Interactive Architecture Diagram (Use controls to zoom & pan)
4. AWS Services Used & Rationales
AWS Services Architecture Rationale
Concrete reasons why these specific services were chosen over alternatives
| Service | Category | Architectural Rationale ("Why this service?") |
|---|---|---|
| AWS CodeDeploy | DevOps | Implements Linear and Canary deployment schedules with automated rollback triggers. |
| Application Load Balancer Weighted Target Groups | Networking | Splits incoming HTTP requests by exact percentage weights without modifying DNS. |
| CloudWatch Alarms | Observability | Continuously checks HTTPCode_Target_5XX_Count and aborts deployment if threshold is breached. |
5. Key Design Trade-offs
Architecture Decision & Trade-Off Matrix
Evaluating alternative approaches under real-world constraints
In-Place Rolling Deployment
- + Zero extra infrastructure provisioned during release
- − Exposes all users to faulty code
- − Rollback requires full re-deployment cycle (15+ mins)
Canary Blue/Green with Auto-Rollback (Chosen)
✓ Chosen Design- + Exposes only 10% of users during verification
- + Instant 30s rollback if errors trigger
- + 0 connection drops
- − Requires running parallel fleet during deployment window
6. Implementation Highlights
AppSpec YAML CodeDeploy AppSpec Configuration
version: 0.0
Resources:
- TargetService:
Type: AWS::ECS::Service
Properties:
TaskDefinition: "arn:aws:ecs:us-east-1:123456789:task-definition/app:2"
LoadBalancerInfo:
ContainerName: "web"
ContainerPort: 8080
Hooks:
- BeforeInstall: "LambdaValidatePreDeployment"
- AfterAllowTestTraffic: "LambdaRunSyntheticTests"7. Results & Key Metrics
- Availability SLA: Maintained 100.00% uptime across 40 consecutive releases.
- Mean Time to Recover (MTTR): Dropped from 45 minutes to 28 seconds on failed releases.
8. Key Architectural Takeaways
Deployment Rule: Never deploy 100% of new code directly to all production servers. Always route a 10% canary slice while monitoring CloudWatch 5XX alarms for automated rollback.
