“30% of Cloud Spending is Potentially Wasted” – this statement is particularly relevant for platform engineers, as most of our core tools run on the cloud. It challenges us to reflect on our current work and consider how we can make better use of the resources we have.
Recently, the concept of FinOps has gained traction as an approach to improving cloud efficiency and cost-effectiveness. To explore its potential, we’ve started experimenting with the FinOps Framework in small groups, aiming to learn and test how it can help us reduce costs and increase visibility into our resource usage.
Today, I’d like to share the “Start Small, Iterate Often” approach by introducing the FinOps Phases from the FinOps Framework. These phases form a continuous cycle aimed at maximizing cloud efficiency, consisting of three key steps:
Inform – Create transparency, allowing all stakeholders to clearly see cloud usage and costs.
Optimize – Develop strategies to reduce costs by eliminating unnecessary resources and improving performance.
Operate – Continuously manage cloud usage to ensure resources are used appropriately and cost-effectively.
Inform: Eyes Don’t Lie
Making cloud costs visible and ensuring that everyone on the team is aware of daily expenses is one of our core goals. These insights must be accurate and as close to real-time as possible, allowing for quick adjustments to reduce waste and maximize resource efficiency.
When working with AWS, we rely on the Cost and Usage Report (CUR) as our primary tool for extracting cost data from each account. This data is stored in S3, and we use Athena to query and analyze it based on different dimensions or views, helping us gain a deeper understanding of our cloud usage.
Of course, we love colorful, customizable visualizations, which is why we chose Grafana as our primary tool for displaying cost data. It makes it easy for everyone to access and understand the information, while also allowing us to set up alerts to respond quickly to critical events.
Optimize: What’s Next?
For this task, we use AWS Compute Optimizer as our primary data source to assess the performance of EC2 instances and EBS volumes. The data from Compute Optimizer is exported to S3, and then queried using Athena to be visualized in Grafana, just like our cost data. This approach allows us to quickly identify underutilized resources (not optimized) and take immediate action, such as:
Resizing instances with underutilized resources to reduce costs.
Migrating from AMD to ARM (Graviton) to save costs and improve performance.
This approach significantly improves our resource management, reducing waste and enhancing overall cloud efficiency.
Operate: Deleting Doesn’t Mean Forgetting
To maintain ongoing cloud efficiency, we hold regular cost review meetings every month. The main goals are to summarize the past month’s expenses, list action items for improvement, and gather feedback from team members to refine our processes and reduce costs further.
In addition to these reviews, we use tools like Cloud Custodian to automate resource management. This powerful tool allows us to set automated cleanup policies, reducing repetitive tasks and preventing resource sprawl, ensuring our cloud environment remains efficient and cost-effective.
This approach not only cuts down on manual work but also helps us stay in control of our cloud expenses, keeping our infrastructure lean and optimized.
Gale: Congratulations!
- From September to October: -4.40%
- From October to November: -9.01%
- From November to December: -1.65%
Through our small but consistent improvements, we’ve managed to significantly reduce cloud costs while maintaining the performance of our various services without impacting the user experience. Additionally, this approach has allowed us to take advantage of newer, more efficient instance types from our cloud provider, which offer better performance at lower costs. This progress is a direct result of leveraging the FinOps Framework for resource management, giving us greater visibility and control over our cloud usage.
We hope this approach will be helpful for others looking to cut costs and improve the efficiency of their cloud environments.
Finally, if your organization is looking for DevOps solutions to automate processes, reduce business costs, and accelerate product and service delivery, SCB TechX is here to support your growth with innovative solutions. Let’s take your organization to the next level. Contact us at https://bit.ly/3RTZ54B.
#SCBTechX #xPlatform #DevOpsaaS #DevOpsSolutions #SoftwareCatalog #SDLC #SystemIntegrator #SI #FinOps
Reference
- https://www.forbes.com/sites/joemckendrick/2020/04/29/one-third-of-cloud-spending-wasted-but-still-accelerates/
- https://www.finops.org/framework/phases/
- https://docs.aws.amazon.com/cur/latest/userguide/what-is-cur.html
- https://aws.amazon.com/compute-optimizer/
- https://cloudcustodian.io/
- https://grafana.com/
