CPU quota vs. All CPUs

[rholowczak]

Folks:

I started up a simple three node cluster in Google Dataproc. The Manager node and two worker nodes are all machine type: n2-standard-4  (gcloud command line is shown below). It has been running for about 15 minutes now.

When I look at my Quotas and System Limits page, I see that in the line for "CPUs (All Regions)" it shows I am using 12 CPUs.  However, on the line "N2 CPUs" it shows only 8 CPUs.  

Why is there a discrepancy between these two numbers?

This is what the Quotas/Utilization look like: https://imgur.com/a/AqGJoMt

I have no other VMs running at this time.

Thanks

Rich H.

gcloud dataproc clusters create cluster-9e64 --enable-component-gateway --region us-central1 --master-machine-type n2-standard-4 --master-boot-disk-type pd-balanced --master-boot-disk-size 100 --num-workers 2 --worker-machine-type n2-standard-4 --worker-boot-disk-type pd-balanced --worker-boot-disk-size 100 --image-version 2.2-debian12 --optional-components JUPYTER ---project xyz-123

[reinc]

Hi @rholowczak,

Welcome to Google Cloud Community! 

The discrepancy between your "CPUs (All Regions)" (12) and "N2 CPUs" (8) usage is likely due to the internal workings of the Dataproc cluster and how Google Cloud reports resource usage.

Your cluster consists of three n2-standard-4 machines:

• Master node: 4 CPUs
• Worker node 1: 4 CPUs
• Worker node 2: 4 CPUs

This totals 12 CPUs. The "CPUs (All Regions)" quota reflects the total CPU usage across all your resources in the project, regardless of machine type.

Here are the reasons for the discrepancy (besides reporting delays):

• Each node's operating system requires CPU resources for its own processes, independent of the Dataproc jobs running on the cluster.
• Dataproc's internal operations consume CPU resources beyond those directly allocated to the virtual machines (VMs) in the cluster. These background services and daemons are essential for the cluster's functionality but are not included in the VM's CPU count.

What to expect:

• The numbers should eventually reconcile after the cluster has fully initialized and settled into a steady state, the "N2 CPUs" usage should likely match or nearly match the 12 CPUs total, considering the system overhead explained above. Give it some more time (perhaps 30-60 minutes).

If the CPU usage discrepancy remains large after a significant wait, investigate further by closely examining the Dataproc cluster logs and resource usage.

To ensure accurate usage reports of your CPUs, please feel free to reach out to our support team.

I hope the above information is helpful.

[rholowczak]

Hi. After one hour the numbers remained the same. I shut down the cluster. "CPUs (All Regions)" went to 0.  N2 CPUs usage went down to 4 and stayed there for the rest of the day.  In the morning, both were at 0.

Something is obviously wrong with how the quota systems gets notified about actual resources. Probably for most people it does not matter if the numbers "almost match".  But for some reason, we cannot increase our quotas and so it is critical that there is accurate accounting of resources.