AWS assessment, cost optimization and a new foundation in Terraform

The challenge

The client, an international data platform, had grown substantially on AWS. The environment counted more than a hundred Lambda functions, dozens of database tables, more than a hundred queues, around ten servers and several networks and APIs.

With that growth, the overview was gone. What does each part cost? How secure is the environment? Which components will soon go out of support, and are the backups in order? The team wanted a complete and honest picture, and infrastructure that is reproducible and set up securely.

Our solution

We tackled it in two phases.

First, we reviewed the entire AWS environment, using our own scripts that map all resources, network and message flows, security settings and costs. That resulted in four reports:

• an infrastructure assessment with an overview of all resources and the security findings, prioritized with concrete actions;
• a cost analysis per service, with the saving and the effort for each measure: Savings Plans and Reserved Instances, storage moved to gp3, fewer NAT gateways through VPC endpoints and lifecycle rules for S3;
• a snapshot analysis of the backups, including the saving of an archive storage tier;
• a timeline of every component going out of support, such as Lambda runtimes, TLS versions and instance types, with the impact and the recommended action for each deadline.

Then we built a new foundation as code with Terraform and Terragrunt. Development, acceptance and production each got their own AWS account, with a separate account for security and a central account for container images. Access runs through AWS IAM Identity Center. Networks, encryption keys, the shared load balancer, the ECS cluster and the databases are all defined in code. The public API runs as a container on ECS Fargate, the frontend through S3 and CloudFront, with their own subdomains and domain names for the platform's customers.

The result

The team got a complete, documented picture of its AWS environment and a clear order of work: first the critical security issues, then the deadlines of components going out of support, then the savings.

The cost analysis showed a realistic savings potential of 12 to 20% of the monthly AWS bill, most of it with little effort. By moving older snapshots to an archive storage tier, snapshot costs can drop by two thirds to three quarters.

The new infrastructure as code makes every environment reproducible, strictly separates development, acceptance and production, and builds security in by default: encryption with customer-managed keys, SSL required for databases and no public snapshots. Zodi Innovations continues to support the platform as its DevOps partner.