Scalability and High Availability in Azure
These are related but different goals:
- Scalability is the ability to handle more (or less) load by adding or removing capacity.
- High availability (HA) is the ability to keep serving requests when components, zones, or regions fail.
- Disaster recovery (DR) is the ability to recover after a major outage. It is defined by RTO (how long you can be down) and RPO (how much data you can lose).
Both are mostly design decisions, not features you switch on. The platform provides the building blocks, but your application has to be built to use them.
Part 1: Scalability
Scale up vs. scale out
- Scale up (vertical): a bigger SKU (more CPU/RAM). It's simple but has a ceiling and often causes downtime or restarts.
- Scale out (horizontal): more instances behind a load balancer. This is the preferred approach for web tiers and APIs because it scales almost without limit and also helps availability.
Design the app to scale out
- Be stateless. Don't keep session or user state in instance memory. Use a distributed cache (Azure Cache for Redis), a database, or signed tokens (JWTs).
- Don't depend on local disk for anything that must persist.
- Make startup fast so new instances can serve traffic quickly.
- Use asynchronous processing. Put slow work behind a queue (Service Bus, Storage Queues) and process it with background workers. This smooths spikes (load leveling) and lets each tier scale independently.
- Cache aggressively: Redis for hot data, a CDN or Front Door for static and cacheable content, and output or response caching where it's safe.
- Avoid chatty and blocking calls. Use async I/O, batch where possible, and reuse connections (
IHttpClientFactory, connection pooling).
Autoscale by compute type
| Service | How it scales |
|---|---|
| App Service | Autoscale rules (CPU, memory, requests, queue length, schedule) or automatic scaling on supported tiers |
| Azure Functions | Event-driven and automatic on Consumption, Flex Consumption, and Premium; Premium keeps pre-warmed instances |
| Container Apps | KEDA-based scaling on HTTP concurrency, queue depth, and so on, including scale to zero |
| AKS | Horizontal Pod Autoscaler or KEDA for pods, plus the cluster autoscaler (or node auto-provisioning) for nodes |
| VMs | Virtual Machine Scale Sets with autoscale rules |
Example for App Service:
az monitor autoscale create --resource-group my-rg \
--resource my-plan --resource-type Microsoft.Web/serverfarms \
--name my-autoscale --min-count 2 --max-count 10 --count 2
az monitor autoscale rule create --resource-group my-rg --autoscale-name my-autoscale \
--condition "CpuPercentage > 70 avg 5m" --scale out 2
az monitor autoscale rule create --resource-group my-rg --autoscale-name my-autoscale \
--condition "CpuPercentage < 30 avg 10m" --scale in 1
Tips: keep a minimum of two instances, scale out faster than you scale in, leave a gap between the scale-out and scale-in thresholds to avoid flapping, and scale on the metric that actually reflects your bottleneck (queue length is often better than CPU).
Scale the data tier (usually the real bottleneck)
- Azure SQL: scale vCores or DTUs, use Hyperscale for very large databases, offload reads to read replicas (read scale-out), and fix query and index problems before buying more capacity.
- Cosmos DB: choose a good partition key (the most important decision), use autoscale throughput, and watch for hot partitions and 429s.
- Storage: scales very high, but spread load across blob prefixes and watch throttling limits.
- Redis: pick the right tier and use clustering for larger datasets.
- Messaging: Service Bus Premium and Event Hubs scale with messaging units or throughput units, and Event Hubs scales with partitions.
- Partition and shard data (by tenant, region, or key) when one database can't handle the write load.
- CQRS and read models let you scale reads separately from writes.
Test it
Use Azure Load Testing (JMeter-based) to find your limits before users do, and validate that autoscale triggers when you expect.
Part 2: High Availability
Layered redundancy
Azure offers redundancy at three levels:
- Within a datacenter: multiple instances, fault domains, and update domains.
- Across availability zones: physically separate datacenters in one region, with independent power, cooling, and networking. This protects against a datacenter failure.
- Across regions: protects against a regional outage. This is the most expensive option and is only needed for stricter requirements.
Start with zone redundancy. For most workloads it gives a strong availability improvement at modest cost and complexity.
Zone redundancy by service
| Service | Option |
|---|---|
| App Service | Zone-redundant plan (supported on Premium v3 and similar tiers; requires multiple instances) |
| Functions | Zone redundancy on Premium and Flex Consumption (check current regional support) |
| AKS | Spread node pools across zones; use pod topology spread constraints and pod disruption budgets |
| Azure SQL | Zone-redundant configuration (General Purpose and Business Critical) |
| Cosmos DB | Zone redundancy per region, multi-region replication, and optional multi-region writes |
| Storage | ZRS or GZRS (zone plus geo-redundant) |
| Service Bus / Event Hubs | Zone redundancy on Premium (and Standard in supported regions) |
| Redis | Zone-redundant tiers |
| Load balancing | Zone-redundant Standard Load Balancer, Application Gateway v2 |
| VMs | Spread across zones using VM Scale Sets or availability zone placement |
Zone and region support varies by region and SKU, so check the current documentation for yours.
Multi-region architectures
Pick a model based on your RTO/RPO and budget:
| Strategy | Description | RTO | Cost |
|---|---|---|---|
| Backup and restore | Restore from backups in another region | Hours to days | Lowest |
| Pilot light | Minimal core (data replication, infrastructure defined as code) kept ready; scale up on failover | Tens of minutes to hours | Low |
| Warm standby | Scaled-down but running copy; scale up on failover | Minutes | Medium |
| Active-active | Both regions serve traffic | Near zero | Highest |
Components:
- Azure Front Door: global, Layer 7 load balancing with health probes, failover, WAF, and caching. It's the usual choice for HTTP APIs and web apps.
- Traffic Manager: DNS-based routing and failover for non-HTTP or simpler needs (DNS caching makes failover slower).
- Data replication:
- Azure SQL: auto-failover groups give a stable listener endpoint and managed cross-region failover.
- Cosmos DB: built-in multi-region replication, automatic failover, and an option for multi-region writes with conflict resolution.
- Storage: GRS, GZRS, or RA-GRS for a read-only secondary.
- Service Bus: Geo-Replication or Geo-Disaster Recovery on Premium. Note that disaster recovery replicates metadata, and (depending on feature) not necessarily all in-flight messages, so check the semantics.
- Azure Site Recovery replicates VMs, and Azure Backup protects data against deletion and corruption. Replication is not a backup, because corruption replicates too.
- Infrastructure as code (Bicep or Terraform) so you can rebuild or scale a region quickly and consistently.
Do the SLA math
Availability multiplies across dependencies in a serial chain. If your API (99.95%) depends on a database (99.99%) and a cache (99.9%), the composite is roughly 0.9995 × 0.9999 × 0.999 ≈ 99.84%, which is lower than any single component. Redundancy (zones, regions) raises the figure, and removing unnecessary hard dependencies helps as well. Use the Azure SLA documentation and the composite SLA approach to estimate it, and remember the SLA is a financial commitment from Microsoft, not a guarantee of actual uptime.
Part 3: Resilient application design
Infrastructure redundancy doesn't help if the app falls over on the first transient fault.
- Retries with exponential backoff and jitter for transient errors (Polly or
Microsoft.Extensions.Http.Resilience). - Timeouts on every outbound call. The default is often far too long.
- Circuit breakers to stop hammering a failing dependency.
- Bulkheads to isolate resource pools, so one slow dependency doesn't consume all threads or connections.
- Rate limiting and throttling to protect yourself from overload (APIM, Front Door, app-level).
- Graceful degradation: serve cached or reduced functionality when a non-critical dependency fails.
- Idempotent operations, so retries and redeliveries are safe.
- Health probes (
/health/liveand/health/ready), so load balancers and orchestrators remove unhealthy instances. - Queue-based decoupling, so a downstream outage becomes a backlog and not an error for users.
- Avoid single points of failure (a single instance, a single database, a single region-bound dependency).
builder.Services.AddHttpClient("orders")
.AddStandardResilienceHandler(); // retry, circuit breaker, timeouts, rate limiter
Part 4: Operations and verification
- Monitor with Application Insights and Azure Monitor. Alert on error rate, latency, saturation, queue depth, and dependency health.
- Define SLOs (for example, 99.9% of requests succeed under 500 ms) and track error budgets.
- Safe deployments: deployment slots, canary or blue/green releases, and automatic rollback, so your own changes don't cause the outages.
- Test failure with Azure Chaos Studio (inject zone failures, latency, and dependency faults) and run failover drills. An untested DR plan is a hope, not a plan.
- Document runbooks for failover and failback, including who decides and how data conflicts are resolved.
- Review against the Azure Well-Architected Framework (Reliability and Performance Efficiency pillars) and use Azure Advisor for recommendations.
Example: a typical highly available, scalable API
Users ─► Front Door (WAF, global LB, health probes)
│
├─► Region A (zone-redundant) ├─► Region B (warm standby / active)
│ App Service or Container Apps │ same stack, scaled down or equal
│ (autoscale, min 2+ instances) │
│ ├─ Redis (zone-redundant) │
│ ├─ Service Bus Premium (zone-redundant)│
│ └─ Azure SQL (zone-redundant) ──── auto-failover group ────► Azure SQL (secondary)
│
└─► Application Insights + alerts, Key Vault, managed identities
Practical roadmap
- Make the app stateless and add health checks.
- Run at least two instances with autoscale on a metric that matches your bottleneck.
- Enable zone redundancy for compute, database, cache, storage, and messaging.
- Add resilience patterns (timeouts, retries with backoff, circuit breakers) and decouple slow work with queues.
- Define RTO/RPO, then choose backup-restore, warm standby, or active-active accordingly.
- Add Front Door and a second region only if your requirements justify the cost and complexity.
- Load test, run chaos and failover drills, and set alerts and SLOs.
A caution: multi-region active-active is often over-built. Many workloads meet their business needs with zone redundancy, good backups, and a tested restore or warm-standby plan, at a fraction of the cost and complexity.