The Amazon S3 Outage Is What Happens When One Site Hosts Too Much of the Internet

Corporate consolidation in tech has implications for competition—but it also affects the resilience of the internet itself.
Image may contain Graphics Art Outdoors Text and Nature
Getty Images

If you've been having trouble using some of your favorite apps today, you're not alone. Users have reported trouble with sites and apps like Medium, Slack, and Trello.

The problems seem to stem from trouble with Amazon's cloud storage service S3, which Amazon confirmed is experiencing "high error rates," particularly on the East Coast. Several other Amazon services appear to be having problems as well, but countless sites rely on S3 to host images and other files. Even Amazon's site itself relies on S3, leading to some baffling updates from the company.

The outages bring to mind the attack on an internet company called Dyn last October that brought much of the web to its knees. Technologically, the S3 outage doesn't bear much resemblance to the Dyn incident, but the effect is similar: So many sites and apps are down that it feels almost like the internet itself is malfunctioning. That flies right in the face of the promise of the internet.

Although it's not true that the internet was designed to withstand a nuclear attack, the fact that there's no central authority in charge of the network makes it, in theory, resilient to attacks on a single company or computer. But as the Amazon outage and the attack on Dyn prove, the internet is actually pretty brittle.

The "winner takes all" dynamic of the tech industry concentrates more and more power into fewer and fewer companies. That consolidation has implications for competition but also affects the resilience of the internet itself. So many people rely on Gmail that when the service goes down, it's as if email itself has gone offline, even though countless other email providers exist. Facebook is practically synonymous with the internet for many people all over the world.

Dyn isn't as well known as Amazon or Facebook. But its internet address service plays a central role in helping web browsers translate an address like "facebook.com" into a string of numbers that a computer can use to actually download a web page. When that service broke down under a DDOS attack, users were stuck unable to access much of the internet.

Amazon plays its own outsized role. Amazon won't say exactly how big its cloud is, but in 2012 one analyst estimated that Amazon hosted around 1 percent of the entire web. It has only grown since then, leading to speculation the cloud could some day overtake Amazon's retail business. The S3 storage service alone hosts about 1.6 times more data than its major competitors combined, according to the analyst firm Gartner. Even many sites not fully hosted by Amazon take advantage of its Cloudfront service, which speeds up websites by distributing their data across the country. According to the firm Datanyze, Cloudfront is by far the most widely used service of its kind. Meanwhile, Google and Microsoft—two other giants—have emerged as Amazon's major cloud competitors.

Amazon's cloud itself relies on the decentralization of the internet. It has servers all over the world, though customers generally pick which regions to host their data. Even within a region, Amazon has multiple data centers in case one goes offline. But Amazon occasionally runs into problems that knock out services for an entire region. When that happens in the eastern region, its most popular, it tends to take down large swaths of the net, which may be what happened today.

Some of the web's founding thinkers have been working alongside younger hackers to try to make the internet more truly a decentralized utopia. In the meantime the tech giants have only gotten bigger, pulling ever-more of the internet into their orbit. Until the web breaks free from that gravitational pull, expect more internet outages.

Update at 5:30pm ET on 2/28/2017: Amazon says the problem was resolved at around 5:10pm ET today. The problem lasted for at least four hours.