Cloud Resource Tagging, optional?

Anybody who has ever clicked a few buttons to provision an S3 bucket or an EC2 instance in the AWS console has probably glossed over the tagging stage. Why wouldn’t you? AWS itself says it’s optional and what are tags even good for really?

It makes sense to skip the step when setting up a quick POC in a demo account or if you are following a tutorial for educational purposes. But when you find yourself collaborating with others and working in your professional accounts, tagging is essential.

Tags are the instrument we have to be able to track origin, ownership, and purpose. You can also use them to organize and filter resources. The specifics of the tagging conventions are left to each organization to decide but the reason for existing is the same.

Ideally, tags should allow us to quickly answer questions such as:

• Who signed off on the provisioning of the resource?
• Which budget does the resource fall into?
• Which team uses the resource?
• Which team spends the most on cloud resources?
• When was the last time a team requested a service?
• Which resources are used in Production? What about Staging?
• What is the infrastructure cost for a certain project?

If your cloud resources are correctly tagged finding answers to these questions is as easy as opening AWS Resource groups and searching by tag. Or better still, connect a single or multiple cloud accounts to Tailwarden and head over to the inventory page where you will can easily filter based on region, service, account, and crucially by tag.

Tagging and naming conventions

There is never a situation in which it’s better to have resources that are more difficult to locate. If there is a security breach, a cost spike, or unusual activity taking place inside your cloud accounts, you won't be able to identify resources quickly and accurately. As an organization and systems scale, this becomes an almost impossible task unless a tagging and naming convention exists and has been applied cohesively across the board.

It’s really down to the particular organization to define the conventions that make sense for them. They must decide which relevant business information should be represented and the appropriate operational information.

If you expect individual contributors to abide by these conventions in an imperative way, you have a recipe for disaster. Since, as the organization scales and an increasing number of resources are provisioned it’s probabilistically more likely that mistakes will happen. Conventions will be forgotten or distorted and work to correct non-compliant resources might be required. That’s why further below we will work on tagging conventions into our IaC (Infrastructure as code) templates and automate their creation. Therefore, avoiding these error-prone mouse-clicking processes.

The challenges of effective tagging enforcement

In an ideal world, the resources we provision in our cloud account would have concise, clear, and useful tags associated with them. Everybody would remember to correct tagging convention, nobody would get lazy and as the business grows and adapts so would the tagging strategy. But few people need convincing that expecting individual contributors to add tags consistently through the console in an imperative way is risk prone and unreliable.

Most organizations have already partially, if not fully, transitioned their infrastructure provisioning efforts to an IaC approach. And those that haven’t are surely aware that they are falling behind.

If your organization is already working in a declarative way using tools like Terraform or Pulumi for instance, then making sure you don’t miss any tags is just a matter of adding a few lines of code to your IaC template.

But let’s take it a step further, we can leverage tagging policies available to us through AWS Organizations. By applying tagging restrictions to organizational units (OUs) we can enforce restrictions on provisioning resources in our accounts unless they have the requisite tags that align with companies tagging conventions.

Afterwards, we will see an example of how we can use default-tags in a Terraform template to make sure we are tagging compliant and avoid ever forgetting about tags.

Let’s explore the diagram below.

Diagram

We have an AWS Organization containing two organizational units (OU) to which we attach the tagging policies we would like to see enforced.

The tagging policies represent the organization's tagging conventions and when applied to the organizational unit we can make sure that only resources that have the series of accepted tags will be eligible for provisioning.

Tagging policy

Below you can see the policy manifest for the tagging policy that applies to OU-1. Find out more about AWS tagging policies here.

The policy enforces the rule that if you want to provision a resource it must contain three tags and each tag has three acceptable values:

• Environment: Sandbox,Staging,Development

• Project: Frontend, Backend, DevOps

• CostCenter: Team_1, Team_2, Team_3

Now that the policy is in place we’ll have to make sure that the resources that are provisioned in the accounts inside OU-1 are compliant. Let’s add the tags to our IaC template to avoid and non-compliant provisioning errors.

Default tags with Terraform

Depending on where your team falls on your organization’s OU map, you will have to apply a certain tagging convention. Configure the default_tags to the aws provider block to ensure every resource that is provisioned using the module is compliant. You can still add the resource-specific tags at the resource level.

In the following example, we are provisioning a VPC with two default tags and two resource-specific tags. Remember that tags are case-sensitive, so you are going to want to make sure that they match your tagging conventions.

The code above adds the Environment and Project tags by default to all resources provisioned through the Terraform template.

Detecting non-compliant tags.

Tagging policies are great at enforcing tags when provisioning resources. Unfortunately, these policies won’t notify us if we have any nonconforming tags already present in our account. The best way to report resources that have noncompliant tags is by using the required-tags rule in AWS Config.

In the example below we can see the AWS Config rule we step up, specifying the tags we want to make our resources compliant for.

Since there are resources that were created prior to the application of the tagging policies we see that we have over 25 noncompliant resources. Now we have a list of resources to remediate and now we can address each element accordingly

Let's expand the required_tags rule to access the noncompliant resources.

To remediate each resource we will click on the resource name link and then access the service console by clicking "manage resource"

By adding the required tags to the resource and waiting around 1-2 minutes for the AWS Config dashboard to read the new tags your resources will be compliant again and you won't be seeing any scary red signs in the Config console.

The pay-off

Depending on the size of your environment, trying to improve tagging and apply a unified tagging convention can be a non-trivial task. The resources might seem never-ending it probably won’t be a very fun task. A former coworker would describe these types of efforts as trying to change the tires of a moving car.

One thing is for sure though, when the painstaking work of applying tags across your existing resources and also ensuring the IaC templates are in place to make it as easy as possible to provision tagging-compliant resources in the future, it will have been all work it.

Coupling these well-tagged resources with a tool like Tailwarden we will easily find questions to the questions we touched on above. By heading to the cost explorer feature on the main dashboard page, for example, we will be able to filter by tag and have a cost breakdown that might shed some valuable light on where your organization is dedicating resources and budgets and can inform your future actions.

Conclusion

Tags at the end of the day are user-defined resource metadata. If you take the time to define which tags make sense in your organization, then you will be one step closer to having the most meaningful custom view into your cloud infrastructure. It’s only when you fully see what you have that you can make the most impactful decisions.

‍

Regardless if you are a Developer, DevOps, or Cloud engineer. Dealing with the cloud can be tough at times, especially on your own. If you are using Tailwarden or Komiser and want to share your thoughts doubts and insights with other cloud practitioners feel free to join our Tailwarden Discord server. Where you will find tips, community calls, and much more.

‍