The Mapbox platform is powered by Amazon Web Services — specifically, most of our computing workloads run on AWS EC2. We’re in the process of wrapping up a company-wide migration (led by the Platform team) onto Docker, hosted on AWS ECS (EC2 container service). Here’s what we’ve achieved by containerizing any and all of our computing needs:
- Lower AWS bills. By switching to ECS we cut our EC2 bill in half - here’s how:
- Better resource utilization: Docker allows us to improve the resource utilization on our EC2s by using them to run more than a single service. For example, on an instance type where we used to run a single occurrence of our Maps API, we can now run both our Maps API and Geocoding API. 2 for the price of 1!
- Spot by default: All of our clusters (EC2s where containers are hosted) are powered by Spot fleets, which are groups of different-instance-type Spot EC2s optimized for diversity. Diversity = stability on the spot market, which means lower cost and lower risk of instance turnover. Docker allows us to take advantage of that diversity, because containers don’t care about the size of the machine they run on, as long as there’s room. Containers are also wicked fast to start up and shut down, which allows us to dodge spot price outs gracefully.
- Centralized EC2 cost management: Using ECS will centralize EC2 cost optimization work in the hands of the Platform team. For most teams, running ECS services with accurate reservations on our clusters will mean their work is EC2 cost-optimized by default. Centralizing control of our EC2 bill will make it much easier to manage.
Better security and credentials management.
We run logging, metrics, and security infrastructure on every EC2. With ECS, the oversight of these systems is consolidated to the team that manages ECS cluster provisioning, the Platform team. This means that if our logging credentials are ever compromised, we only have to rotate credentials in a single place and everyone benefits.
Consistency across teams and services.
The number of different things you need to know to work on a given service will shrink drastically with Docker, ECS, and the frameworks and tools we’ve built for using them. More consistency across services means we can more effectively work and share knowledge and resources across teams. This process streamlines our workflows as a company, leading to better collaboration and higher throughput.
Separating private infrastructure from otherwise sharable code.
We have to consider logging, metrics, and security infrastructure when we run EC2s, and we need to keep these parts of our systems private. If Mapbox-specific EC2 bootstrapping is no longer part of the application’s code base, it is suddenly much easier to open-source that application. Once we switched, we were able to move Watchbot, the library we use to power our data processing stacks, from private to public. Check out watchbot on GitHub if you’re interested!
Less environment confusion.
Understanding the EC2 environment can be confusing and obscure if you don’t know where to look. Using Docker asks the application developer to be more explicit about their required runtime environment and less dependent on convention or other systems. Since we can run Docker locally, we can run our code in an environment that is that much closer to production. This can help spot issues that currently can take several iterations of stage, fail, debug, repeat.
Do you love listicles about AWS infrastructure and scale? The Platform team is hiring!