GCP Batch noExternalIPAddres =True

[gradientopt]

Hi Team,

This post https://www.googlecloudcommunity.com/gc/Infrastructure-Compute-Storage/GCP-Batch-In-use-IP-addresses... mentioned that one could use GCP Batch service without attaching an IP address to each machine when running task by setting noExternalIPAddres =True. However, the additional network configuration seems complicated and I am wondering what would be the cost and speed/job scheduling efficiency comparison of  noExternalIPAddres =False (default, attach IP address) vs   noExternalIPAddres =True (do not attach ip address)?  We may run 40 thousand tasks in parallel and just want to make sure that would not cause things to be any extra expensive (cost of Batch API using ip address) or slow (hard to allocate so many ip addresses). Thanks!

[bolianyin · Accepted solution]

To clarify

>> the additional network configuration seems complicated

Do you mean setting noExternalIPAddres =True or something else?

Setting noExternalIPAddres =True should only provides benefits (easier to allocate VMs, etc) unless you do need external IPs. 

[gradientopt]

Thanks for the reply! 

1. Yes I meant noExternalIPAddres =True. We can not just set this parameter and expect it to work right? We also need to do steps here https://cloud.google.com/batch/docs/job-without-external-access

How much benefit in run time and cost could this provide? We are debating whether or not we should set this given the network configuration is quite complicated…

2. Is external ip required to make outgoing http download request?

[gradientopt]

Also, our Batch task uses docker that mounts gcs storage, if we disable external ip address, can we still access gcs? Thanks!

[wenyhu]

Hi @gradientopt,

Yes, in general, the external ip is required to make outgoing http download request.

However, if you only need to call Google APIs, you can consider setup internal network with Private Google Access. Since GCS is a Google service, you can still access to GCS if you setup network with private google access properly.

The other option to use internal only ip with outbound connections is to setup Cloud NAT with non-external ip.

You can find more detail in https://cloud.google.com/batch/docs/job-without-external-access.

Hope this helps!

- Wenyan

[gradientopt]

Thanks for the reply!  Sorry for keep asking these but I am a newbie to computer networking :-(. My confusion is the following. What else do I need to do if I set "noExternalIpAddress": true? The tutorial you pointed out says that I just need to specify the network and subnetwork. Can I just use the default vpc network for my project (I did not do any modification on default vpc after GCP initializes for me)? Or do I have to set up another new vpc network that has Private Google Access?

  "allocationPolicy": {
    "network": {
      "networkInterfaces": [
        {
          "network": "projects/HOST_PROJECT_ID/global/networks/NETWORK",
          "subnetwork": "projects/HOST_PROJECT_ID/regions/REGION/subnetworks/SUBNET",
          "noExternalIpAddress": true
        }
      ]
    }
  },

[gradientopt]

It seems that the only additional thing I need to do is to toggle Private Google Access to be on for my default vpc? The default is set to be off so I am wondering will turning it on affect my other more permanent VMs (such as my static dev server that I ssh to ) on this VPC that actually has an external ip address?

[wenyhu]

Hi @gradientopt, the default network and subnets are auto-generated when you enable a project with services. Although it should work, I would recommend you to create a new VPC for your `no external ip` cases instead of modifying the default VPC. Thanks!

[taltman]

Hi there, I hope it's OK to ask a related question here. Please let me know if I should post it at the top level.

I have a project that requires running hundreds of jobs, each of which has to download ~40 GB on average. I was hitting my IPv4 limit, so I tried using Cloud Public NAT. That turned out to be prohibitively expensive. 

I see that I have a very large IPv6 quota for my region. I tried to follow the Google documentation to set up my VPC to use IPv4 for internal, and IPv6 for external addresses. But it seems that batch doesn't honor this, or doesn't allow you to configure your batch instances to request a "dual stack" setup. Is there any way to to utilize IPv6 external addresses to avoid the small IPv4 quota, but still allow downloading content from the public Internet?

[wenyhu]

Hi @taltman,

Batch hasn't supported IPV6 yet. For your internal IPV4 addresses, could you try to adjust primary ipv4 range to help on the quota limitation? E.g. if the range is 10.0.0.0/24, you have 256 ip. If you change it to 10.0.0.0/9, it would be 8,388,608 ip.

Hope this helps!

[taltman]

Hi @wenyhu, I wish I could increase my quota in my target region, but I am capped at 70 external IPv4 addresses, and a quota adjustment request was denied. Is there some other way to increase the number of external IPv4 addresses that I can use? Or is there some way to set up a bastion host that will proxy the external requests?

[wenyhu]

I meant that you could increase your IPv4 address range without increasing your quota. In that way, without increasing your quota limit, you should still be able to get enough addresses.