High Availability in the Age of AI: It Was Never Optional

2026-05-31T00:00:00Z

There's a conversation happening right now in every engineering leadership team, and it goes something like this: "We need to move fast on AI. What does the infrastructure need to look like?"

The honest answer is: the same as it always should have. Just faster, at higher stakes, and with far less tolerance for getting it wrong.

Availability isn't a new problem. But AI has changed the consequences of ignoring it.

The AI agent changes everything — and nothing

AI agents are being treated like a new category of infrastructure problem. In some ways they are. But they don't change the fundamental engineering answer.

What they change is the blast radius. When a stateless web app goes down, users get an error page. When an AI agent goes down mid-workflow — halfway through a multi-step process, having already written to one system and not another — you have broken state, wasted compute, and eroded trust.

The bar for availability just went up. The answer didn't change.

What "high availability" actually means in practice

I want to be precise here, because this phrase gets thrown around loosely.

High availability doesn't mean running two instances in the same data centre. It doesn't mean having a hot standby you've never actually tested a failover against. It doesn't mean your monitoring dashboard shows green before it shows red.

Real HA means your application can survive the complete, unplanned loss of any cloud region — and your users experience either nothing at all, or a degradation so minor they barely notice. That's what regionless infrastructure makes possible.

That requires:

Active traffic distribution, location-agnostic. Not just replication. Not just backup. Live traffic flowing into geographically separated regions simultaneously — and critically, the ability to spin up in a new region without re-engineering the architecture to do it. If deploying to a new region requires significant rework, you haven't built for resilience, you've just duplicated a fragile system.

Data that follows the compute. This is where most teams underestimate the complexity. Your application logic is often the easy part to replicate. Getting your data layer to be consistent, low-latency, and resilient regardless of where workloads are running is the hard part. Whether that's global database clusters, event-driven replication, or careful design around eventual consistency — this is the work. Don't skip it.

Tested, automated failover. Runbooks don't cut it. If your failover depends on a human reading a Confluence or Notion page at 2am, that's not HA. That's hope. The mechanism needs to be automated, regularly exercised, and your engineers need to have seen it work before the incident happens.

Health checks that catch real failure modes. Not just "is the pod running?" — but "is it actually serving traffic correctly?" Synthetic transactions, deep health endpoints, circuit breakers. AI services especially need this, because a model that's returning degraded or incoherent responses is failing even if every container reports healthy.

Why regionless — not just multi-region, not just multi-AZ

Multi-AZ, multi-region and regionless: what each one survives.

Multi-AZ gets you a long way, and if you haven't got it already, start there. But it doesn't protect you from cloud provider region-wide events. And those happen.

Anyone reading this has seen it. Region-wide degradation that takes out multiple Availability Zones simultaneously. It's not a rare edge case — across the major cloud providers there have been approximately 10–15 significant regional outages in the last 12 months alone. An infrastructure fault at the network or control plane level that cascades across everything within that blast radius. A dependency — DNS, IAM, a managed service — that's regional and fails everything that relies on it.

If your entire workload sits in a single region, that region going dark takes you with it. Doesn't matter how resilient your internal architecture is.

Multi-region gets you further. But "cross-region" as a concept still implies a fixed set of regions — typically two — that you've manually wired together. That's better than single region, but it still has a ceiling.

Regionless is different. It means the architecture itself has no opinion about where it runs. Infrastructure as code that deploys cleanly to any region. Routing that adapts to where capacity is healthy. No hardcoded region dependencies in your services. The ability to add a new region — or shift load to one — without an engineering project to make it happen.

That's the meaningful shift. You're not building resilience by picking a second location. You're building an architecture that is inherently location-agnostic. And in the AI era, where you may need to place workloads closer to users for latency, comply with data residency requirements across multiple jurisdictions, or respond to a regional cloud failure without prior planning — that flexibility is not a nice-to-have. It's a competitive and operational necessity.

For AI workloads, this matters even more. AI services often have hard dependencies on third-party APIs, model inference endpoints, or large language model providers. If any of those are pinned to a single location, you've inherited their failure modes too. A regionless architecture gives you options — fallback routing, cached responses, graceful degradation — that you simply don't have when everything is tied to a fixed location.

The cost question everyone asks

"This doubles our infrastructure cost."

It doesn't — and that's the first thing to correct when this comes up. A well-designed regionless architecture runs non-primary regions lean. You're not running multiple full production environments simultaneously. You're running one primary and right-sized capacity elsewhere that scales out when needed. With reserved capacity and intelligent routing, the overhead is typically 20–40% above single-region, not 100%.

But there's an even stronger answer for active-active deployments: you're not adding compute capacity at all. You're redistributing it. If you're already running the infrastructure to handle your current load, you can split that load 50/50 across two regions today. Same total compute, just spread across two locations. The additional cost is real but focused — network infrastructure, VPCs, inter-region connectivity, data transfer. That's a fraction of what people assume when they hear "run everything twice". The work is in the architecture. The bill is manageable.

But more importantly, the question is being asked backwards.

The question isn't "can we afford regionless infrastructure?" It's "what does an outage cost us?" One hour of downtime — lost revenue, SLA breach penalties, engineering incident response, customer trust erosion — almost always exceeds months of the incremental infrastructure cost. Put that number in the room and the conversation changes.

FinOps discipline matters here too. Reserved capacity, intelligent traffic routing biased heavily toward your primary region, right-sized standby — these are the tools that let you get the resilience benefit without overpaying for it. The architecture answer stays the same. The cost is a design problem, not a veto.

Where AI specifically raises the stakes

A few things about AI workloads that make regionless infrastructure more complex to design, and more necessary to get right:

State complexity. Traditional stateless apps are easy to replicate. AI agents often carry context — conversation history, tool call results, intermediate reasoning state. You need a strategy for where that state lives, how it's replicated, and what happens to an in-flight agent session if its primary region fails. A regionless architecture needs to handle state portability — not just compute portability.

Cost-per-failure. Every failed invocation of an AI model is compute cost you've paid for nothing. At scale, high error rates aren't just a user experience problem — they're a budget problem. Resilient infrastructure that degrades gracefully, retries intelligently, and routes around failure is also FinOps infrastructure.

Latency sensitivity. Many AI interactions are synchronous and time-sensitive. A regionless architecture should keep users on their nearest healthy region — not route them halfway around the world because the primary had a blip. The routing intelligence matters as much as the regional footprint.

Dependency chains. AI systems tend to have longer dependency chains than traditional apps — model endpoints, embedding services, vector stores, memory systems, tool integrations. Every link in that chain needs to be resilient. Regionless infrastructure at the edge doesn't help you if something deep in the chain is pinned to a single location.

This is foundational, not advanced

I want to push back on the framing that regionless infrastructure is an "advanced" engineering concern — something you graduate to once you've matured. It isn't. It's foundational infrastructure for anything that matters.

The reason it gets treated as optional is because it's invisible when it's working and deemed expensive to build. Availability doesn't appear on a feature list. It doesn't show up in a sprint demo. Engineers don't get celebrated for building the thing that never fails.

But it is, without question, the most commercially important resilience investment a platform team can make. It protects revenue. It protects reputation. It protects the teams who depend on the platform to deliver.

In the AI era, where an agent might be acting on behalf of a user, automating a business process, or making something happen in the real world — the stakes of availability failure are higher than they've ever been. An AI system that's sometimes available is not a system people will trust. And trust, once lost, is far harder to rebuild than the infrastructure that would have prevented the outage in the first place.

What to do if you're not there yet

If you're reading this and you know your architecture isn't regionless yet, here's where to focus:

1. Map your failure modes honestly

Which services are single-region? Which are multi-AZ only? Which have hardcoded regional dependencies baked in? Prioritise by commercial impact — start with the workloads where downtime costs the most.

2. Fix the data layer first

Compute is the easy part. Database replication, event streaming, consistency models across regions — this is where the real design work lives and it takes the longest. Start it before anything else.

3. Build for location-agnostic deployment

Infrastructure as code, no hardcoded region references, routing that treats region as a variable not a constant. Every time you build something new, ask: could this deploy to a different region without rework? If no, fix it before it scales.

4. Automate your failover before you need it

Build the mechanism, test it in non-production, run a game day. The first time failover runs should not be a real incident.

5. Get observability everywhere you run

Centralised logging, distributed tracing, synthetic monitoring from outside each region. You can't manage what you can't see — and you can't route around failure you haven't detected.

6. Make the case in commercial language

Resilience work is hard to fund because the benefit is invisible until something breaks. Flip the framing: what is the cost of the outage you're preventing? That's the investment case.

The bottom line

Regionless high availability infrastructure is not a nice-to-have for organisations running AI systems. It's the foundation. Everything else — the agent capabilities, the intelligent features, the automation, the personalisation — all of it depends on the infrastructure being there when it's needed, wherever it's needed.

AI has raised the stakes, sharpened the blast radius, and made the cost of unavailability more visible than it's ever been. The answer is the same as it's always been — just bigger in ambition.

Build it once. Build it right. Build it so it doesn't care where it runs — because your users won't care where it fails.

System online

2026-09-25T00:00:00Z

After 25+ years building and running infrastructure, I've accumulated a lot of hard-won opinions and very few of them are written down. This site fixes that.

What to expect

Posts will fall into a few recurring areas:

Platform engineering and DevEx — building internal platforms engineers actually choose to use.
Cloud governance and FinOps — guardrails that scale, and spend that maps to value.
Resilience — regionless HA/DR, failure domains, and recovery you have genuinely tested.
AI platforms — MCP, governed gateways and treating AI as a platform capability rather than a pile of experiments.

How I'll write

Practical over theoretical. Every post should leave you with something you can apply on Monday: a decision framework, a checklist, or a mistake you don't have to make yourself.

The best platform is the one nobody has to think about.

$ zion --status
all systems nominal

More soon.

Ultimate Zion