Tips to avoid surprises in your AWS bill

When the cloud became a thing, not that I was around when it happened. One of the biggest selling points was cost.

“Wait a minute, so I don’t need to pre-buy all of the servers that I think I might need and employ a team of sysadmins to keep the dark data center in the basement running?” - Every cloud admin everywhere

It goes without saying that life “post-cloud” is orders of magnitude better than life “pre-cloud”, but the matter of cost is still a sticky subject.

The reality is that the cloud is great, but if mismanaged it can become a very expensive endeavour.

Who is this for?

Before jumping into the deep end I wanted to acknowledge that each company is its own universe and each project has a different set of requirements and stages of development. Obviously, the tips that would help a mammoth enterprise to save the big bucks might differ from what somebody who is just firing up a Free Tier AWS account to test some code might need.

Having said that, these tips are more directed to Startups and mid-sized companies without a huge number of specialized cloud engineers on the team.

I don’t want to leave anybody empty handed though, so:

A tip for hobbyists:

Just firing up a test Free Tier account? Always be mindful that not all services have a free tier offering and you might want to set up the AWS Free Tier usage alerts. which can be done in the Billing preferences section of your AWS account.

A tip for enterprise organizations:

I would check out AWS Control Tower Account factory for Terraform, which makes it easy to set up and manage a secure, multi-account AWS environment. With the configuration heavy lifting done on your side, you can set up clear guardrails and boundaries so users in your organization only provision infrastructure that is approved or allowed.

Even though there are hundreds of AWS services we could talk about, these are the main ones from which you might benefit from optimizing.

Network

Elastic IPs

Any Elastic IP not actively attached to an EC2 instance is incurring cost, either use it or release it back into the Elastic IP pool.

Note that you always want to keep the number of unnattached IP addresses down to 0.

You can use a tool like Komiser to detect any unattached Elastic IPs, as seen below:

Amazon API Gateway

API Gateway is a great managed service but when demand is high and traffic increases it can run the risk of you not getting the best bang for your buck. Since the price is pretty high (for Serverless standards) at $3.50 per million requests. Consequently, you may want to consider swapping the service for an application load balancer since it has an easy Lambda integration.

Be sure to use the right type of API Gateway, HTTP API is much cheaper than REST API. As seen below:

Storage

Amazon EBS volumes

One of the easiest ways to minimize your AWS bill is to reduce EBS costs, below is the pricing of EBS volumes:

Unattached volumes

Most times when you delete an EC2 instance the associated EBS volume will be deleted too, but this is not always the case, by using a tool like Komiser we can see the number of volumes that we have in our account, and if any are unused we can see them in the storage dashboard. And promptly move to remove them.

Komiser can also be used to detect unattached EBS volumes, check out the video below:

Snapshots

At the time of writing, EBS snapshots are charged $0.05 per GB-month of data stored. Other factors that affect snapshot cost include:

If you’ve stored snapshots of your Amazon EBS volumes and no longer need them, you should  archive or delete them as well. Deleting a volume does not automatically delete the associated snapshots. Which incur costs.

Amazon S3

S3 is flexible managed service offering a number of storage classes apart from the default Standard class that can provide a considerable amount of savings if non-instant object retrieval is an option. As a baseline below we have the standard class cost breakdown per month.

If you have a large number of files that you don’t access that often, you might want to consider moving them out of Standard class. By transitioning them into Standard-IA or Onezone-IA which is has the downside of being less reliable, with an AWS commitment of 99.5% availability. Down from 99.99% you get with the standard class.

You may also want to consider using S3 Intelligent tiering, a service that for a small fee will analyize your object access and retrieval patterns, making automatic storage class decisions to help you use S3 as efficiently as possible.

Compute:

Amazon EC2

Your organisation might allow for EC2 instances to be spun up directly but, many times, we are using managed services that provision EC2 instances for us, such as Amazon RDS and Amazon EKS among others. It is quite easy to over provision resources and not realise that we can reduce the amount resources needed to maintain the same performance. Apply some of these tips if applicable to your organisation:

• Use Spot Instances when possible (you can afford instance interruption by designing a fault-tolerant application)
• Use Spot Fleet
• Identify and Delete Orphaned Snapshots
• Leverage GPU Instances (when running ML workloads)
• Use AWS Compute Optimiser to get performance and resource provisioning recommendations by using machine learning to analyse historical utilisation metrics.

AWS Elastic Beanstalk

Elastic Beanstalk is designed to ensure that all the resources that you need are running, which means that it automatically relaunches any services that you stop. To avoid this, you must terminate your Elastic Beanstalk environment before you terminate the resources that Elastic Beanstalk has created.

Remember the load balancing resources associated with the Elastic Beanstalk environment incur cost too.

Amazon EKS

Kubernetes clusters are a perfect example of a resource that can be very expensive if mismanaged, they are great candidates for big potential savings.

Be sure to:

• Add and adjust the right resource requests and limits to your deployments.
• Use a Cluster Autoscaler to automatically adjust the size of your cluster.
• Cluster sharing, try to reduce the number of clusters you manage and operate.
• Use Spot or Reserved Instances for fault-tolerant and instance type flexible applications.

AWS Lambda

A well configured Serverless stack is probably the most cost-effective way to build on the cloud (in most cases).

But keep these tips in mind:

• Optimise the memory settings for functions that use provisioned concurrency and adjust memory allocation for a function by using a tool like this to avoid guesswork.
• To reduce the number of Lambda invocations and hence reduce the cost, add caching with API Gateway
• Build small Lambda Functions: make the function small and purpose built. The Lambda runtime has to get your function’s code from S3 or a Docker image registry on every cold start. Downloading one gigabyte takes Lambda much more time than downloading one megabyte. This is the waiting time you pay for.
• Lambda will send all logging data to CloudWatch Logs. This service is beneficial, but it isn’t free. If you’re logging excessively, the CloudWatch costs can end up eating your Lambda savings

Services launched by other services:

Be careful with CloudFormation templates that you or someone from your organization run and then forget about. You might delete a resource manually and leave others behind since they were deployed from a stack.

Many third-party tools use CloudFormation under the hood too, such as eksctl and AWS CDK so be sure to track the tools that were used to provision resources, and when it’s time to decommission those resources, be sure to use the tool that provisioned them in the first place.

Conclusion

All in all, in order to avoid any surprises in our monthly AWS bill we need two things.

We need to be aware of the pricing models we have agreed to and should be notified if the pricing models change. A great tool to be sure you don’t miss any pricing model changes is this one.

Most importantly though, we need visibility into our cloud accounts, that’s why we recommend using tools like Komiser or AWS Cost Explorer in which you can have a clear and direct view into what resources you have to your name, and with that data, you can make better-informed decisions regarding your infrastructure. Continuous monitoring and alerting is the key when it comes to saving costs!

If we had to depend on the AWS console to track all of the billable services in our account, we are bound to be unpleasantly surprised at the end of the month when the bill hits our inbox.

‍

Regardless if you are a Developer, DevOps or Cloud engineer, dealing with the cloud can be tough especially on your own. If you are using Tailwarden or Komiser and want to share your thoughts doubts and insights with other cloud practitioners feel free to join our Tailwarden Discord server. Where you will find tips, community calls and much more.