Guide to Cloud Financial Management: How to foster a culture of FinOps

Lauren_vdv · 03-27-2023 07:42 AM

Businesses are always looking for ways to maximize their cloud investment and realize more business value. However, legacy processes, methodologies, and technologies can make managing and predicting cloud spend a challenge.

In this article, we’ll cover common cloud financial management challenges and how to foster a culture of FinOps that accelerates business value. We’ll also address frequently asked questions and provide a roundup of helpful resources.

If you have any questions that aren’t addressed in this article, please leave a comment below and someone from the Community or Google Cloud team will be happy to help.

Event recording
Common cloud financial management challenges
What is FinOps?
Building blocks of Cloud FinOps
1. Accountability & Enablement
2. Measurement & Realization
3. Cost Optimization
4.Planning & Forecasting
5. Tools & Accelerators
7 FinOps principles to drive lasting cultural change
Cloud FinOps resource roundup
Cloud FinOps FAQS (and answers!)

Event recording

The Google Cloud Community came together in February for a live learning and Q&A session on how to foster a culture of FinOps, led by the Google Cloud Technical Account Management team. Check out the recording here and use the time stamps in the YouTube description to jump to the topics you care about most.

The session recording is also available in Portuguese and Spanish and we're hosting an additional live session for those in the JAPAC region on April 26th at 11AM SGT (GMT +8).

Common cloud financial management challenges

It’s natural for companies that are starting to migrate to the cloud to rely on financial processes used to manage their data center environments, but as shown in the image below, many find that traditional IT financial controls have limited ability to manage and predict cloud spend effectively.

To address these challenges, companies are establishing cloud FinOps capabilities to not only implement effective cost visibility and control, but also to accelerate the realization of value from cloud investments.

But what do we mean by FinOps? And how do you put it into practice?

What is FinOps?

Google Cloud defines FinOps as:

“An operational framework and cultural shift that brings technology, finance, and business together to drive financial accountability and accelerate business value realization through cloud transformation.”

Cloud FinOps is a shared responsibility across the organization, including IT, DevOps, Engineering, Finance, and the Business to help ensure cloud services are consumed in the most cost effective manner to drive sustainable business outcomes.

Sounds great - but how does this work in reality?

Building blocks of Cloud FinOps

The FinOps journey can be seen as a flywheel of five pillars (Accountability & Enablement, Measurement & Realization, Cost Optimization, Planning & Forecasting, and Tools & Accelerators), across three phases in the FinOps lifecycle (Inform, Optimize, and Operate, as proposed by the FinOps Foundation).

Now let’s dive into each of the Cloud FinOps pillars in more detail.

1. Accountability & Enablement

The Accountability & Enablement pillar of FinOps is focused on providing visibility and transparency into cloud spending so that teams can make informed decisions and take ownership of their cloud usage.

The main goal here is to establish clear accountability for cloud spending across the organization, ensuring everyone has a shared clear understanding of their role in managing costs.

To kick this off, focus on identifying an executive sponsor and building a governance committee, which will be responsible for reviewing and approving budgets for business cases and emerging tech - which is especially key for digital natives.

It’s also important to form a central FinOps team of cross functional resources who are responsible for setting up controls, such as deciding and approving Committed Use Discount (CUD) purchases and renewals, creating a central policy document, automating policies, etc.

Additionally, having a service catalog will enable auto provisioning of cloud infrastructure and self service, which will ensure consistency and less wastage. We’ve seen some successful examples like Banco BV where the FinOps team is key for a successful journey, fostering decision-making empowerment, knowledge enablement, and data democratization.

The enablement aspect of this pillar focuses on providing teams with the tools, resources, and training they need to effectively manage costs. This includes implementing cost optimization best practices, cloud financial training, and education on shared terminology, etc.

2. Measurement & Realization

This Measurement & Realization pillar focuses on developing a defined set of business value metrics and KPIs to measure success of transformation.

Oftentimes, we see customers start with a set of cost optimization metrics (e.g. actual vs budget per service, % underutilization of services, spend per application, etc.) and eventually shift to unit economics / business value metrics across several factors, including cost efficiency, resiliency, velocity, innovation, and sustainability.

Learn more in the blog, Decoding Cloud FinOps to accelerate digital transformation.

Foundational to the Measurement & Realization pillar is beginning with accurate data, which is based on the labeling and tagging data architecture behind your company’s use of cloud resources. Start with metrics that are easily available and measurable (e.g. cost efficiency metrics like % of commitment coverage, % of idle instances, average CPU and memory utilization, % spot coverage, etc.).

Since business goals and strategic imperatives will likely change over time, it is important to review and update your metrics as needed whenever goals and priorities change.

Effective measures not only create awareness and enable agile processes, but also support a culture that celebrates success and rewards teams for achieving new milestones.

3. Cost Optimization

Cloud cost optimization is not just about cutting costs—it's about knowing where to spend your money to maximize the business value. Cost optimization is an iterative and continuous process that provides a consistent methodology to visualize and manage cloud consumption in a most cost effective manner.

Success in cloud cost optimization can result not only in significant reductions of cloud spend, but also in re-architecting for improved application performance to manage user demand growth within the same cost envelope.

Cost Optimization strategies address three factors:

Cost visibility: Identify what you spend, how services are billed, and more importantly - the ability to showcase why you spent a specific amount to achieve a business outcome. This is foundational to fully understanding and organizing cloud costs.
Resource optimization: Focus on eliminating cloud waste (idle resources), autoscale where necessary, and rightsize the compute and storage tier to only use what you need. Often this function is decentralized, since engineering has the most context on where and how to optimize the resource used.
Rate optimization: Uses historical data to make pricing model adjustments by buying commitment based discounts, and works to manage the pricing aspects of services it’s using in the cloud. This is typically centrally owned by the FinOps team, as they have visibility into all cloud historical usage, trends, and patterns.

We’ve seen many successful examples like Optiva, MD Insider, and Raycatch where customers optimize their total cost of ownership by leveraging Google-managed services and serverless offerings to offload operational overhead, thus allowing them to focus on growing their business. See the top 10 ways in the cloud cost optimization matrix below.

4. Planning & Forecasting

Effective planning and forecasting in the cloud requires organizations to better understand their cost drivers, allocate spend, and benchmark efficiency.

A few key practices in the Planning & Forecasting pillar include:

Monthly budget reviews: Engage application teams in budget discussions and ensure there’s an established process to determine and fund a budget.
Forecast metrics: Track how many times there are budget overruns by application and/or business area, and also track the frequency and accuracy of forecast changes (this should get better over time as teams iterate)
Set budgets: Review budgets monthly and monitor weekly. Identify variances, update forecasts as needed, and communicate the updates to all teams, including the potential impact.

5. Tools & Accelerators

The Tools & Accelerators pillar focuses on selecting and using the right tools and resources to fully benefit from FinOps practices.

In earlier stages, companies may have limited ability to report detailed analysis of cloud spend. As practices mature and improve, labeling and tagging of resources proves valuable to understanding costs for specific projects and teams. These capabilities become even more powerful through automated monitoring of resources through a dashboard that offers insights on spend and value.

Google Cloud provides a suite of Cost Management tools for monitoring, controlling, and optimizing your Google Cloud costs. You can set budgets to closely monitor your costs and alert stakeholders when you’re about to hit your budget threshold, enforce organizational policies with granular permissions to control who can spend or view costs, create custom dashboards or use intuitive reports to see at-a-glance views of your current cost trends and forecasts, and much more.

7 FinOps principles to drive lasting cultural change

Regardless of the organization size and their experience in cloud, we have seen these guiding principles deeply rooted for successful execution of cloud FinOps:

Embrace the variable nature of Cloud: Embrace the pay-as-you-go nature of cloud and benefit by serving your customers dynamically during their constantly changing needs.
Make financial decisions from a business-value perspective: FinOps is successful when decisions are made from a business value perspective.
Collaborate across boundaries and departments: Partner with IT Finance, Application Development, and Infrastructure teams for cloud financial management.
Accountability: Hold teams responsible for value realization reporting and cost optimization with a set of clearly defined KPIs and metrics.
Blamelessness: Treat mistakes as blameless opportunities to learn and improve existing processes.
Democratization of data (cost transparency): Provide access to real-time billing/cost data at all layers in the organization.
Agility: Drive agile processes for budgeting and planning within IT by promoting iterative budget allocation, IT spending, and forecasting.

Cloud FinOps resource roundup

Published whitepapers

Google Cloud Blogs

Podcasts/YouTube

Documentation

Learning and training

[Google Cloud Skills Boost] Understanding your Google Cloud Costs
[Google Cloud Skills Boost] Optimizing your Google Cloud Costs
FinOps Foundation learning and certification options
Google Cloud Community Learning & Certification hub

Cloud FinOps FAQS (and answers!)

1. How to motivate cloud operations teams to implement cost optimization recommendations generated by cloud cost management tools?

Make sure that the recommendations are actionable and realistic. The recommendations should be something that the team can actually implement, and they should be realistic in terms of the amount of effort required.
Highlight the benefits of implementing the recommendations. This could include saving money, improving performance, or reducing risk.
Make sure that the team has the necessary resources to implement the recommendations. This includes both the technical resources and the financial resources.
Provide training and support to the team. The team needs to be able to understand the recommendations and how to implement them. They also need to be able to get help if they run into any problems.
Set a good example. If you are the manager or leader of the team, it is important to set a good example by implementing the recommendations yourself. This will show the team that you are serious about cost optimization and that you are willing to put in the effort to make it happen.
Reward the team for implementing the recommendations. This could include giving them bonuses, giving them time off, or simply thanking them for their hard work.
Make cost optimization a part of the team's performance review. This will show the team that cost optimization is important and that they will be evaluated on their ability to implement cost optimization recommendations.

2. What are the top 3 areas (specific product configs) that are the lowest hanging fruits to reduce spend?

Compute: Rightsize, reduce idle resources, purchase commitments, use spot, and consider containerization with auto scaling
Storage: Choose the right location and tier, pick the right storage class and only store data you need and purge the rest. Archive data that you don’t need to access, but needs to be stored for compliance reasons
Networking: Choose the right network tier, architect to reduce egress costs and minimize data that needs to be sent over the network

3. Is there any single source where I could receive all the pricing updates of Google Cloud products?
Yes - the Google Cloud pricing page. This page provides a comprehensive overview of the pricing for all Google Cloud products and services. It is updated regularly to reflect the latest pricing changes.

4. What are your tips on establishing buy-in from engineers who think they already "do finops"?
Here are some tips for establishing buy-in from engineers who think they already "do FinOps":

Understand their perspective. Take the time to understand why the engineers are resistant to FinOps. What are their concerns? What do they think FinOps is? What do they think it isn't?
Address their concerns. Once you understand their perspective, you can address their concerns head-on. What are the benefits of FinOps? How will it help them? How will it help the company?
Show them the data. Data is a powerful tool for convincing people. Show the engineers the data on how FinOps can save money, optimize resources, and improve efficiency. Use tagging/ labeling and cost attribution
Get them involved. The best way to get buy-in is to get people involved. Include the engineers in the FinOps process. Let them see how it works and how it can benefit them.
Make it easy for them. FinOps should be easy to adopt and implement. Make sure the engineers have the resources and support they need to make it happen.
Celebrate successes. Once the engineers see the benefits of FinOps, celebrate the successes! This will help to motivate them and keep them engaged.

FinOps is a relatively new field, and there is still a lot of confusion about what it is and what it entails. As a result, some engineers may be resistant to adopting FinOps because they think they already "do FinOps." However, FinOps is more than just managing cloud costs; it is a holistic approach to managing the financial aspects of the cloud. By understanding the engineers' perspective, addressing their concerns, and showing them the data, you can convince them of the benefits of FinOps and get their buy-in.

5. What are the best practices for Cloud forecasting and forecasting BigQuery usage?
Depending on your organization, consider the following three primary ways to forecast cloud costs:

Trend based forecasting: Uses historic trends to forecast future spend, considering seasonality. Does not capture new events.
Driver based forecasting: Uses KPIs to forecast the effect on business results, such as active accounts, number of bookings, ad impressions, etc. Cannot forecast workloads that are not in the cloud.
Rolling forecasting: To predict next month, quarter, and year. It allows companies to adjust their plans based on any shifts in the business, including economic changes like COVID, for example.

Learn more about these cloud forecasting options and best practices from the FinOps Foundation here.

For forecasting BigQuery usage:

Use the BigQuery slot estimator to estimate the right number of slots to purchase for your workload based on historical performance.
Create Dashboards from Information schema to monitor slot utilization and BigQuery reservations. The INFORMATION_SCHEMA metadata tables contain relevant, granular information about jobs, reservations, capacity commitments, and assignments. Using the data from these tables, users can create custom dashboards to report on the metrics they’re interested in in ways that inform their decision making. Get a jump start with these report templates for daily slot utilization (measured in slot days compared to active commitments), hourly slot utilization, reservation utilization, and job execution (with regards to slot utilization). Find more details here.

6. Is there any internal prioritization to add GA support for Labels on more components (all billable SKUs ideally) to enable better cost analysis? Any plans to enable easier access to slice/dice of costs within GCP platform itself (similar to AWS Cost Explorer) to avoid building out Looker/DataStudio?
You can find the Google Cloud services that support labels here, which is a pretty exhaustive list. Currently, we have the following tools under cost management to help with visibility, and all of them support labels:

Built-in reports
Cost breakdown
Cost table
BigQuery Billing Export
Custom Dashboards

7. A common problem I run into on GCP is that engineers/devs teams create assets and then “forget” or seldom use them. Is there a way to generate a report to see which assets are not being used so we can avoid paying unnecessarily?

For VMs, you want to ensure that they’re actively being used, and also have the right level of resource allocation. Active Assist is a collection of tools and resources in Google Cloud that simplify the identification of unused/idle resources and save you money. We have three primary types of recommenders that you can use:

Idle VM recommender: Will highlight VMs that haven’t been used recently in the last 14 days for existing machines. You can either shut it down (to restart late) to stop accruing costs or completely delete from the project.
VM Machine type recommender: Helps with rightsizing VMs by monitoring CPU/memory usage over the past 8 days to suggest a better machine type fit/pricing fit. This helps reduce cost without impacting performance.
Unattended project recommender: Identifies unused projects over the last 30 days. Also helps identify projects that do not have an active owner so you can reclaim those projects.

An additional recommender to help optimize your cloud costs is the committed use discount (CUD) recommender, which provides recommended commitments to further optimize your Google Cloud costs.

Watch this video for a live tutorial on how to reduce costs with Active Assist.

8. During the 90 day/300-in-credit trial, how can I see a breakdown of what I am using per day? I seem to be burning through the credits quite rapidly but can't seem to find what's doing it. Apparently normally you would be able to see this information (the services that are costing you money) in the 'Billing' section of the cloud console but during the free trial it doesn't seem to be there (as you're just using the free credits).
You can go to your billing report and see your resource usage grouped by project. On the right-hand side panel, you can see something called promotions. If you uncheck promotions, you will see how much your actual cost is without the free trial credits being applied.
Also you'll get billing statements at the beginning of each month telling you how much of the credit was spent on what, so you can get an idea of the charges. Any usage against the free tier isn’t charged with the credits. At the end of your trial period, if you upgrade, you can still keep your remaining credits.

Note: The data can sometimes take between 24 and 48 hours to be updated.

9. What are best practices to limit data egress charges on files in cloud storage? Is it to enable requester pay on the bucket or limit availability to a particular region... considering both have their disadvantages
Making a Google Cloud Storage (GCS) bucket public or sharing with a group is an easy way to make your data more available to people. However, if someone copies your data out of the GCS region, network egress charges can be incurred. The Requester Pays feature can be enabled on the bucket to protect you as the data provider. Unfortunately, the cost is passed onto end users, which they are unaware of, unless you communicate this with them.

It is possible to avoid network egress charges through the use of Google Cloud's VPC Service Controls. By creating a service perimeter around the Cloud project that contains your bucket, you can prevent data being copied outside of the region (preventing egress charges) and also help block access from specific IP ranges.

10. How to track Capex and OpEx expenses on projects?
Capital expenditure (CapEx) is the cost a business incurs to acquire assets that will provide benefits beyond the current year. CapEx is also referred to as PP&E, which stands for Property, Plant, and Equipment.

For example, in Google Cloud, CapEx can be used when you own dedicated hardware like sole-tenant nodes, Google Cloud VMware Engine, or bare metal hosts (physical hardware dedicated exclusively to the customer’s use and control).

When it comes to financial reporting, while some capital expenditures are fully expensed the same year you make them, they usually go into the balance sheet as assets, not expenses. Only a percentage of it goes on the profit and loss statement (as a depreciation expense on an ongoing basis).

In Google Cloud, you can create a specific project for your CapEx assets and they will be assigned a unique CapEx id. These assets require payment upfront and an approval process, which can mean longer procurement time. These are deductive over the lifetime of the asset and amortized over the lifetime of the intangible asset.

Operating expenses (OpEx) refer to the money a company spends to run day-to-day operations.

Examples are cloud services based on subscription (SaaS, Paas, IaaS, etc.). Operating spend is usage-based and variable, and hence, fluctuates as consumption increases or decreases. Cloud providers own the asset that they lease you and they own updating and maintaining the asset. It's pay-as-you-go pricing and recorded in P&L as expenses. They should be deducted in full in the same year they are incurred.

11. How soon can you start thinking about FinOps? Is there any historical data needed before thinking about this?
We recommend getting started with FinOps practices as early as possible, even before/during the point where you’re planning a cloud migration. It’s always easier to implement FinOps when you’re at an earlier stage without a lot of resources in the cloud yet. You can set the right policies from the beginning without needing to re-architect or start over later on. See the complete live response at 30:37.

12. Does FinOps support multicloud/hybrid cloud?
FinOps does support multicloud and hybrid cloud deployments. In terms of tooling that can help you with cloud financial management and FinOps in a multicloud or hybrid environment, Looker is a solution that many customers use. With Looker, you can:

Create insights and automate processes with data regardless of where it is
Power data experiences from Google Cloud and/or on-premises systems—and with data stored in BigQuery and other databases.
Develop a flexible multicloud strategy by measuring cloud usage and costs.

We recommend using native cloud tooling where possible, and augmenting with 3rd-party tooling as needed. 3rd party solutions for multi-cloud and hybrid cloud cost management include:

CloudHealth by VMware: Provides detailed insights into cloud usage and spending across multiple clouds and accounts
Cloudability by Apptio: Offers features such as cost tracking, budget management, and customized reporting
RightScale Optima by Flexera: Offers features such as cost analytics, optimization recommendations, and budget tracking

See more information in the Tools and Accelerators part of the presentation at 21:41.

13. Where do you recommend positioning the Cloud FinOps team? Infra, Finance, Procurement?
We see a variety of alignments between the FinOps team and the rest of the organization. Initially, a FinOps team often forms without the organization taking any deliberate action at all – usually forming as a small group of finance or technical professionals who are asked to generate reports on cloud spend. Without becoming aware at the time, these individuals often become the first FinOps practitioners in the organization. Once the need for a formal FinOps team is identified, there are several common organizational-structure options that companies can use to implement an official team. See pages 5-8 for more details and recommendations.

14. What background or experience is needed for the Practitioner and Professional certifications?
As indicated on the FinOps Foundation website, the FinOps Certified Professional course and exam is geared towards experienced FinOps practitioners who want to move to the next level, further their careers, and their FinOps knowledge. Before registering for the FinOps Certified Professional course, you must meet the following prerequisites:

FinOps Certified Practitioner
6+ months of FinOps work experience

The FinOps Certified Practitioner course and exam allows individuals in a large variety of cloud, finance, and technology roles to validate their FinOps knowledge and enhance their professional credibility. There are no required prerequisites to take the FinOps Certified Practitioner exam, but the FinOps Foundation recommends the following:

Understand the basics of how cloud computing works, know the key services on your cloud providers, including their common use cases, and have a basic understanding of billing and pricing models.
Be able to describe the basic value proposition of running in cloud and understand the core concept of using a pay-as-you-go consumption model.
Have a base level of knowledge of at least one of the three main public cloud providers (AWS, Azure, Google Cloud).

15. How to get cloud cost analysis data with cost APIs and cloud sdk with python?
We have a python client library for the Cloud Billing API. Details can be found in our public documentation here: Python Client for Cloud Billing Budget API.

16. How can we tag dataflows and see how much each one costs in the billing reports?
Dataflow provides user specified labels (available in billing exports), which can be used for cost attribution. Once you have the billing export in BigQuery, you can run your custom queries to get the cost for each of your Dataflow jobs, including how many CPUs, memory, hours, and other units you’ve consumed on a per job basis.

17. How can I make sure I’m not accidentally getting billed when I’m trying the Google Cloud free trial?
You will not be charged unless you explicitly enable billing by upgrading your Cloud Billing account to a paid account. With that said, you can always avoid surprises on your bill by creating Cloud Billing budgets and alerts to monitor all your Google Cloud charges in one place.

18. Is there a good practice or tool that allows grouping the consumption of APIs within an organization that has n projects each consuming n APIs according to their purpose?
Cloud Billing Reports lets you view your Google Cloud usage costs at a glance and discover and analyze trends. The Reports page displays a chart that plots usage costs for all projects linked to a Cloud Billing account. To help you view the cost trends that are important to you, you can select a data range, specify a time range, configure the chart filters, and group by project, service, SKU, or location.

Thank you to our contributing authors!

Upasana Pati (@upasanapati) Technical Account Manager, Google Cloud
Ícaro Ribeiro (@icaroribeiro) Technical Account Manager, Google Cloud
Fabian Navarrete (@fabiancrisnz) Technical Account Manager, Google Cloud
Vicente Allende (@Vicho13) Technical Account Manager, Google Cloud
Pedro Bomente (@pebomente) Technical Account Manager, Google Cloud
Kuntal Mitra (@kuntalmitra),Technical Account Manager, Google Cloud