Why Dropbox decided to drop AWS and build its own infrastructure and network
There is always a tension inside companies about whether to build or to buy, whatever the need. A few years ago Dropbox decided it was going to move the majority of its infrastructure requirements from AWS into its own data centers. As you can imagine, it took a monumental effort, but the company believed that the advantages of controlling its own destiny would be worth all of the challenges they faced to get there.
For starters, a company like Dropbox is dealing with a huge number of customers storing an enormous amount of data. The latest numbers are 500 million users and 200,000 business customers. When they made the transition, they had to move an epic 500 petabytes — that’s five followed by 17 zeros — that had been sitting on AWS servers. (They still use AWS for some workloads.)
The first step was building the infrastructure to replace it. We’re talking about a company that had 1500 employees, with just around a dozen on the infrastructure team. This was not a huge operation, yet what they were trying to do was build something themselves at web scale that only a small number of companies with much larger teams had tried to this point.
That included building and equipping three US data centers. It also meant building the network backbone, the infrastructure that facilitated the connections between the US data centers and other facilities they had located throughout the world. When you open Dropbox and request a file, you want your file to download pretty much instantaneously without latency, and it was up to the team to ensure that happened, while trying to navigate between the old system and the new one.
“What’s neat about the backbone is that it’s similar to something you might find at Google or Facebo...