Unexpected Charges for Network Internet Data Transfer Out from Americas to South America

[Vijay4]

Hi everyone,

Over the past three days, we've been seeing charges under the SKU "Network Internet Data Transfer Out from Americas to South America" in our GCP billing reports.
We've checked our VM instances, storage buckets, and other services but couldn't trace which specific resource is responsible for this outbound data. Even after enabling grouping by project, SKU, and labels in the billing dashboard, it’s still unclear where the traffic is originating from.
We’ve raised a support request but haven’t received any response yet.
Has anyone experienced a similar issue or can suggest how to:

• Identify which resource or instance is generating this data transfer?

• Trace or log the exact traffic responsible for cross-region data egress?

Any help or insight would be greatly appreciated!

Thanks in advance.

[nmagcalengjr]

Hi @Vijay4,

Welcome to Google Cloud Community!

VPC Flow Logs provide valuable insights into network throughput and performance, allowing you to identify sources within your VPC network. By analyzing these logs, you can optimize network traffic costs and use them for network forensics.

Here's a breakdown of how to trace this:

1. Enable VPC Flow Logs:

VPC Flow Logs provide detailed insights into both ingress and egress traffic within your VPC, helping you identify which instances, services, or IP addresses are generating traffic to South America.

• In the GCP Console, go to the VPC network section, select the subnet associated with your resources, and enable Flow Logs for it. Set the log format to JSON for detailed insights, and then view the logs through Logs Explorer.

2. Check the Firewall Rules:

Check for any firewall rules that might be causing routing outbound traffic across regions. Misconfigured firewall settings could cause traffic to be directed even if it's not originating from your resources.

• To check, go to the Firewall rules page in the GCP Console, review rules that allow outbound traffic to external IPs or regions, and make sure no unnecessary traffic is allowed across regions.

3. Audit External Services and APIs:

External integrations or third-party services, such as Google Maps APIs or Cloud Pub/Sub, may unintentionally cause data to flow out of your region. Check if any of these services are generating cross-region traffic.

• To identify any unusual calls causing data egress, check your API usage under APIs & Services > Dashboard.

4. Use Google Cloud Monitoring (Cloud Monitoring and Cloud Logging):

You can set up custom dashboards and alerts in Google Cloud Monitoring (formerly Stackdriver) to track network usage in real time.

• You can set up custom metric filters in Cloud Logging to monitor network traffic related to your resources and configure alerts for specific thresholds, such as data transfer to South America, to detect any unexpected traffic spikes.

5. Check Inter-Region Traffic or Peering:

If you have VPC peering or VPN connections established, cross-region traffic might be routed through those connections. Review your VPC peering or interconnect configurations to identify any potential causes of the charges.

6. Export Cloud Billing data to BigQuery:

Export your billing data to BigQuery for more detailed insights, and run SQL queries to pinpoint the specific services generating the charges. This will help identify any resources responsible for the data transfer to South America.

• To export billing data, set up billing export in your GCP Billing account to send data to BigQuery, then query the dataset to filter and aggregate traffic by regions or specific resources.

If you need further assistance, you can reach out to Google Cloud Support at any time.

Was this helpful? If so, please accept this answer as “Solution”. If you need additional assistance, reply here within 2 business days and I’ll be happy to help.

[Vijay4]

Hi @nmagcalengjr,

Thanks for the reponse.

I'll review the points you mentioned.

Thanks & Regards

[drikpanchang]

Hi...we are facing similar issues. During last 30 days, we see Network Data Cost going as high as 50% due to data transfer to "North America". We hardly have any user base in North America and historically Network Data Cost to North America was almost zero.

[drikpanchang]

Vijay4, have you found any solution for huge Network Data cost from "Mumbai to South America". We are facing huge data cost due to this issue since last 30 days.

[Vijay4]

Hi @drikpanchang

Yes, we recently encountered a similar issue involving unexpected charges under the "Network Internet Data Transfer Out from Americas to South America" SKU.

Here’s what we did to identify the root cause:

1. Enabled BigQuery Billing Export
   We set up BigQuery export for detailed billing data. While it takes up to 24 hours to populate, it enabled us to run queries grouped by project, SKU, and instance ID. This was crucial in pinpointing the VM responsible for the outbound traffic.

2. Enabled Cloud Monitoring (Operations Suite)
   We configured network-related metrics (like sent_bytes_count) across all VM instances and created dashboards to track traffic patterns. The visual spike in outbound traffic helped us quickly identify the affected instance.

3. Checked VPC Flow Logs (if enabled)
   If VPC Flow Logs are active, they can help trace the exact source and destination of network traffic, providing deeper insight into unexpected egress.

4. Filtered BigQuery Data by resource.labels.instance_id or location
   Filtering billing data by instance ID and location and grouping by project name and resource allowed us to isolate the issue effectively.

In our case, the source was a test server that had started generating high egress traffic. We immediately shut down and destroyed the server to prevent further charges.

If your issue involves a production server, we recommend enabling Google Cloud Armor and configuring tighter egress firewall rules to mitigate such risks in the future.

Thanks!