Scourge · No culprit · October 20 2025

The Day us-east-1 Fell Silent

Chronicle

During the night of 19–20 October 2025, at 23:48 Pacific time, a latent defect in DynamoDB's automated DNS management left the DNS record for the service's regional endpoint in AWS's us-east-1 region empty. The failures cascaded through EC2, Network Load Balancer and many other AWS services for about 15 hours. Snapchat, Fortnite, banks, Alexa and thousands of other services were disrupted worldwide. No attacker: just an empty record in the wrong place.

Lesson learned

It was a reminder of how much of the Internet depends on a single region of a single cloud provider, and that automation systems can harbour defects capable of stopping everything built on top of them.

Prayer of reparation

O us-east-1, firstborn region of every cloud, always resolve our names, and keep our architectures from putting all their eggs in one data centre. Amen.

How it is remembered

Remembered by designing services to survive the loss of a region, actually rehearsing failover to another and checking that the emergency plan does not depend on the very region that went down.