"Network Internet Data Transfer Out from Americas to Americas" gradual cost increase

[jackson-oomdo]

My bill has been increasing by around $200 a month over the past 5 months or so. The main culprit seems to be attributed to "Network Internet Data Transfer Out from Americas to Americas" (the blue bars in the screenshot below).

I was hoping someone could tell me how I can dig into this further and figure out why this is. I know this is based on data transfer so it will vary but the gradual increase resulting in my bill being around $600 more than it was 5 months ago, and this month projected to be higher than it was last month yet again, is a concern.

How can I figure out exactly why there has been such an increase so I can hopefully find a solution to bring it back down?

I, unfortunately, only consider myself a web developer but in my small company I've inherited everything tech related because there is nobody else to. So this stuff is pretty confusing to me. I'm thinking VPC flow logs might be the answer? But I also see that logging also comes with a cost. Is there anyway to figure this out without having to pay for it?

Any insight or direction would be greatly appreciated.

[Roderick]

Hi there @jackson-oomdo, Thanks for bringing your questions to the Community! I definitely can empathize with you -- understanding your GCP statement and drill down to how your usage might be creating unexpected bills in frustrating. Hopefully one of our experts will hop onto the thread and help demystify what's going on. 

 Have you checked out Cloud Billing to help you track and understand your Google Cloud spending, pay your bill, and optimize your costs? There you'll also find video explainers, training and tutorials and resources that will walk you through each line of your bill and goes over topics that help you understand, control, and optimize your Google Cloud costs.

Let us know if we can help connect you to more resources, including a conversation with Billing Support. 
 

[jackson-oomdo]

I've narrowed down the majority of the price increase to "Network Internet Data Transfer Out from Americas to Americas". That is what the blue bars are labeled in the screenshot.

But just knowing that "Network Internet Data Transfer Out from Americas to Americas" cost keeps increasing doesn't help me. I can't drill into that any further in the cost breakdown. I need to be able to see the specifics of what's going on on my servers that have caused this to keep increasing and then I can figure out a solution.

Does that make sense? Are you able to ask one of your experts to read my post?

[nickb123]

Hi jackson-oomdo

Did you find a solution to identifying what was causing the price increase?
I have the same issue where I have isolated a significant increase on our bill to the SKU  - 'Network Internet Data Transfer Out from Belgium to Apac(Excluding Korea and Indonesia)'.  However I cannot link this to a process/app on our cluster.

Thank you and regards 

[jackson-oomdo]

Nope. As you can see, no help has been provided here lol.

Unrelated to this post - my company was contacted by a Jose Solano Vindas - (PII Removed by Staff) - offering to go over anything GCP related for free. I spoke to him, he wasn't able to help with this issue but he set up a meeting with our company's account manager for next Tuesday and hopefully she will be able to help me with this. This meeting is also for free. I will post what I learn from that meeting but maybe you can contact Jose and get your own meeting set up.

[ArtemBondar]

I also see exactly the same issue and the same concern.

In my billing, the "Network Internet Data Transfer Out from Americas to Americas" spiked after I have deployed a new application, and I have not idea how to debug this to understand what exactly is billing into this categoy.

I hope my comment will bring the attention to this tipic from Google Support

[jackson-oomdo]

I had a free call with some members from Google. They gave me ideas of what it could be but couldn't help me debug it. They eventually told me to open a ticket with billing support which I did 7 days ago on Tuesday March 12th. I haven't heard back from them.

[ArtemBondar]

Please keep posting updates here if you get a resolution about your ticket!

[ArtemBondar]

@jackson-oomdo I was doing some experiments and research, and here is what I found, hoping to be helpful.
So the traffic calculated to "Network Internet Data Transfer Out from Americas to Americas" bucket is all outbound traffic from GCP to consumers within Americas. The inbound traffic to GCP is free. The price for outbound traffic is $0.12 per GB of data.
So all data, that are transered to your counsumers, for web applications, APIs and Services goes to this bucket.
So you see this number is growing because of two reasons:
1. You have more consumers that use your applications
2. Your application generates more outbound traffic because of application changes.

What you need to do, is analyze the dataflow, and what can be improved. In my case, I have reduced the size of static data assets, such as images for my web application. So for each consumer it will transer less data. And at scale it all adds up.

I have found this approach to query raw data from Google to analize per deployed service: https://stackoverflow.com/questions/75880810/how-to-calculate-the-monthly-costs-of-an-individual-clo...
I didn't try it, because for me picture is clear. But if you need, you can try to find out which of your apps consumes excessive traffic and can be optimized.

[ArtemBondar]

Okay, I have found one more interesting thing.

By default, google project is configured to "Premium Tier" networking, which is more expensive.
You can reduce networking cost by switching to "Standard Tier"

It's under: Main Menu -> Network Service Tiers -> Change Tier.
After tier change need to redeploy all services for this to take effect.

Standard tier is $0.08/GB vs $0.12/GB for premium tier. Also on Standard tier, first 200GB per month is free.

[jackson-oomdo]

Thanks for the information! I will take a look into these things.

Still haven't heard back from Google.

[Teshy]

You can assign or change the static IP associated with a resource to get the change to standard to take effect. It is under VPC networks > IP addresses > 
External IP Addresses

[jackson-oomdo]

This is the response I received from Google:

I observed that you have created several VMs and two VPC networks.
Upon checking the Metric Explorer for VM Instance Sent Bytes, it was noted that high bytes were transmitted between September 2023 and February 2024 for Compute Engine instances.

Root Cause Identification:

The high charges are attributed to Internet Egress traffic, which occurs when a compute engine instance or application hosted on an instance sends requests outside to the internet.
Unfortunately, it is not possible from our end to pinpoint the exact cause of why the VM instances or applications started sending high traffic during specific periods. This can typically be identified by examining application logs, as traffic is initiated from within the VMs.

Egress Firewall Rules:

I observed that your project does not have any egress-denying firewall rules. It's important to note that every VPC network has implied egress firewall rules that allow traffic to any destination unless explicitly blocked.
To prevent unwanted egress traffic to the internet, we recommend implementing egress deny firewall rules. You can refer to the documentation [2] for guidance on setting up these rules.
Best Practices for Protection:

I will also share documentation [3] outlining best practices for protection, which includes strategies to secure your network and manage traffic effectively.
By implementing egress deny firewall rules and following best practices, you can better control and manage your network traffic, reducing unexpected charges and enhancing security.

{2} https://cloud.google.com/firewall/docs/firewalls#egress_cases
{3} https://cloud.google.com/files/GCPDDoSprotection-04122016.pdf

[invm]

For us, the Network Internet Data Transfer Out from Frankfurt to EMEA has increased from 1500 GiB to 2500 GiB per month, and so have the costs. Google cannot give us an answer to this. Strangely, I read your posts about the same phenomenon. We neither have more access to our apps nor do we deliver more data.

We couldn't get any help from the free billing support either. We were referred to the contact form for the sales team, but unfortunately no one ever contacted us.

We were previously contacted by the sales team about another issue. This was about increasing the cloud build triggers from 300 to ~350. They promised that it would be done, but nothing happened.