Discovering Idle Multi-Cloud Resources with Tailwarden

Introduction

As a young kid, my brothers and I often spent our weekends assisting our father, a skillful carpenter, with his side projects in the shed at the back of our garden. These activities frequently required trips to the local hardware shop to purchase materials. When we reminisce about those times, one of our most vivid memories is how every time our dad came across a skip (a large open bin usually found around building sites with used building materials and random detritus), our dad couldn't resist the magnetic pull and felt obliged to inspect its contents. While I wouldn't consider him a hoarder, recycling just comes naturally to him, and any kind of waste feels wrong. The saying “One person's trash is another person's treasure” could easily be my dad’s catchphrase.

One person’s skip is another person’s treasure chest

This very quote proves to be quite useful as we delve into the topic of this article because we shouldn’t consider our idle and unused cloud resources as anything but trash.

To make the comparison even more vivid, let’s apply the saying directly to a cloud context to uncover what’s at stake.

By drawing this analogy, we can see the significance of optimizing and efficiently managing cloud resources to avoid unnecessary costs and wastefulness.

Managing cloud environments is becoming increasingly complex with their rapid growth and the rising trend of multi-cloud adoption. Keeping up with the ever-expanding inventory of cloud resources while optimizing and ensuring everything stays up-to-date presents significant challenges. As your organization expands into new regions and welcomes new cloud engineers to the team, the likelihood of resources going astray, unused, and undetected. And by a consequence, unnecessary expenses negatively impact your infrastructure budget.

Finding idle resources can feel like looking for needles in more than one haystack, how can we easily detect resources when they are no longer needed? How can we cut the fat in our inventory long before it impacts your bottom line? Having the right tool for the job is the first step. It becomes crucial to utilize tools specifically designed to handle the intricacies of modern multi-cloud environments. These tools should be capable of actively retrieving all cloud resources and effortlessly compiling a comprehensive resource inventory offering easy mechanisms to uncover, spot, and alert on resource anomalies.

Enter Tailwarden, the cloud-agnostic resource management tool that can assist you in achieving your cloud resource management objectives.

The Hidden Cost of Idle Resources

In the world of cloud engineering, it's tough to battle an enemy you don't even know exists. Many cloud engineers handling massive cloud environments struggle to wrap their heads around the hefty and frankly, sometimes intentionally confusing cloud bills. Those sneaky idle resources go under the radar, secretly stacking up costs, and get mistaken for regular monthly expenses.

It's entirely understandable how these resources can easily slip by unnoticed. Even for those well-versed in navigating a cloud provider's console, finding what you need can prove to be a daunting task, even within a single account. Once multiple cloud accounts are added to the mix, the constant juggling of logins, windows, and user credentials only compounds the complexity. This cumbersome practice of merely searching for a cloud resource becomes even more arduous when attempting to locate resources that may not even be on your radar.

Relying solely on traditional cloud console clicking or scouring through lengthy Cloudwatch logs to discover, manage, and optimize cloud resources would undoubtedly lead us down a path of frustration and futility. Fortunately, this is precisely where Tailwarden enters the scene, by fetching the cloud resources of any integrated cloud account (currently supporting AWS, Azure, GCP, Civo, DigitalOcean, and OVH), Tailwarden saves you the hassle of having to chase down resources. Once a cloud account is correctly added it automatically fetches and builds your cloud resource inventory without any manual intervention. Imagine the efficiency and peace of mind this brings as your cloud infrastructure dynamically expands.

Now, let's explore Tailwarden more in-depth and discover how you can harness its full potential to elevate your cloud resource management to new heights.

Unleashing Tailwarden's Resource Discovery Features

The Resource Inventory Page

Throughout our discussion, we have alluded on multiple occasions to the resource inventory page—an extensive list gathering all the cloud resources provisioned across your multiple cloud accounts. Without the aid of filtering or slicing capabilities, this list would undoubtedly provide be an overwhelming amount of information hard to digest. However, luckily for us, Tailwarden offers a plethora of tools for filtering, sorting, and logically grouping our resources, transforming the inventory into an easily manageable single window pane in your multi-cloud environment.

Leveraging Strong Filtering

There are many metrics you might find useful to filter by. Tailwarden offers a wide array of options such as Cloud Service, Regions, Cloud Provider, Creation date, Resource name, and many more.Hopefully, as cloud engineers, you are leveraging tags and assigning them to your cloud resources. Since Tailwarden filtering can also leverage this type of metadata. By using filters you can quickly zero in on areas of interest.

Let’s imagine the following situation, all DynamoDB tables that are in use in your organization are tagged in a manner aligned with your internal tagging convention. Meaning if any untagged DynamoDB tables are found, they should be further inspected to know if they are still in use or if they are in need of tagging. Finding these resources now is extremely easy.

Filter by Creation Date

When trying to locate idle, unused, or forgotten resources the metric of time can be very useful. By leveraging the creation date filter you can sort through resources based on when they were provisioned. The example below filters all EBS volumes that were provisioned before a certain date and a particular region.

In the example, 3 EBS volumes were found to be provisioned before 01/02/2023 in the eu-central-1 region. Resources that have been provisioned for longer are great candidates for more in-depth inspection.

Daily Activity Digest and Budget Alerts

For deeper insights into the inner workings of your cloud environments, you have the option to enable the Daily Activity Digest feature. This nifty functionality delivers a concise Slack message to your chosen channel, providing you with a comprehensive rundown of all the resource changes and activities recorded in your account. With this timely and informative notification at your disposal, you can stay informed about the latest happenings in your cloud setup, helping you stay on top of any critical updates or potential issues that may arise.

Also, Tailwarden empowers you to set up cloud account budget alerts with ease. You can effortlessly define a dollar threshold and designate a Slack channel to receive these notifications. Should any idle resources accumulate and potentially lead to exceeding your budget, this mechanism acts as your early warning system, promptly alerting you to take immediate action. By proactively monitoring your cloud spending, you can stay in control and swiftly respond to any budgetary concerns before they get out of hand.

💡 The Daily Activity Digest and Budget Alerts are applied at the cloud account level

The Power of Cost Explorer

The most effective tool for tracking cost trends on Tailwarden is the cost explorer widget, conveniently located at the bottom of the homepage dashboard. Within this feature, you have the flexibility to toggle between Service, Region, Account, and Provider views. While it may not always be easy to pinpoint the precise offending resources, the cost explorer offers valuable insights with its daily or monthly breakdown of the most expensive resources and their comparisons to previous months. For instance, if a particular service consistently incurs the same cost month after month, it could signify that the service is provisioned but remains unused, thereby alerting you to potential inefficiencies that need attention.

Best Practices for Ongoing Resource Management

Cloud environments are constantly evolving and dynamic. To maintain continuous alignment and efficiency throughout the lifecycle of your environments, it is crucial to establish well-defined processes. We recommend three essential steps to ensure cloud cost optimization and management effectiveness, enabling you to pay only for the resources you truly need and use them efficiently.

Regularly perform idle resource checks using Tailwarden

Ensure that you and your SRE teams adopt the practice of regularly sifting through the Resource Inventory page to identify any anomalies. To streamline this process, save the filter searches demonstrated above as custom views, allowing you to create virtual nets that can catch any resource infractors. By default, Tailwarden offers the "Expensive Resources" and "Untagged Resources" custom views designed precisely for this purpose. Apart from depending on custom views to catch idle resources, embrace the practice of filtering and sorting your inventory and a regular cadence. Who knows, unknowingly those needles buried deep in the haystack might be costing you a pretty penny.

Implementing proactive tagging and resource naming conventions

It’s crucial to operate in an environment that adheres to a clear, concise, and widely adopted tagging convention. While the default filtering metadata, such as Cloud service and Creation date, provides some filtering capabilities, it can be generic. However, when you combine the power of default tags with your custom tags, the possibilities for filtering and resource management become truly boundless. The combination of default and custom tags significantly enhances the effectiveness of your management capabilities due to the fact that you can filter based on the logical groupings that make up the internal structure of your cloud environment. Therefore, it is crucial to follow recommended best practices (the article focuses on AWS tags but can be applied to other cloud providers) when implementing tagging strategies. When strict tagging conventions are followed, incorrectly tagged or untagged resources stick out like a sore thumb. But how can we make sure the tagging convention is applied consistently? The right team culture.

Educate your teams on the importance of resource optimization

As teams and organizations expand, a lack of well-defined conventions and agreed-upon procedures often leads to individual contributors requesting or creating resources that linger unnecessarily long, even after their purpose has been fulfilled. To tackle these resource inefficiencies head-on, it becomes imperative to unite your team around a shared culture that emphasizes the importance of minimizing waste in cloud resource usage.

By fostering a collective understanding of the need to optimize resource utilization and implement appropriate tools and processes, you can ensure that everyone is onboard and actively engaged in the responsible management of cloud assets. This proactive approach not only leads to significant cost savings but also results in improved performance and efficiency in your cloud infrastructure.

Conclusion

Avoiding unnecessary expenses from idle and unused resources should be a top priority, it’s the low-hanging fruit for any cloud analytics-based cost-reduction effort. Imagine if you could save money for your organization by simply pressing a delete button, if this was a possibility, you would take it every time right? Traditionally finding these cost savings opportunities wasn’t that easy. But now, Tailwarden eliminates the need for arduous manual searches for lost cloud resources in distant regions.

Building, securing, and scaling applications in the cloud is fun, but searching for pesky resources costing you money is not. Tailwarden streamlines this process, allowing you to focus on what truly matters.