Node CPU resources are different from VM CPU resources it is tied to

[omarcevi]

Hello i have a node tied to an "a2_ultra_4g" vm instance according to documentation the instance should provide 48 cpu cores and 4 gpu. however my node can only utilize 24 of these cores.

i have checked the quota limits on the machines but theres nothing indicating that i have hit a limit in terms of cpu cores.

any idea where to start solving this?

[Manish_B]

Hello @omarcevi,

Thank you for contacting Google Cloud Community.

Please note that the GKE nodes run system components that let the node function as a part of your cluster. These components use node resources, such as CPU and memory. You might notice a difference between your node's total resources, which are based on the size of the underlying Compute Engine virtual machine (VM), and the resources that are available for your GKE workloads to request. This difference is because GKE reserves a pre-defined quantity of resources for system functionality and node reliability. 

There are many different system services that require resources on the node in order for the cluster to behave correctly. The fraction of a CPU that you are unable to allocate is reserved for those services. If you look closely at a single Node, you can divide the available resources in:

1. Resources needed to run the operating system and system daemons such as SSH, systemd, etc.
2. Resources necessary to run Kubernetes agents such as the Kubelet, the container runtime, node problem detector, etc.
3. Resources available to Pods.
4. Resources reserved to the eviction threshold.
5. Resources reserved for the System pods in the “kube-system” namespace

To inspect the allocatable resources on an existing node, run the following command:

kubectl get node NODE_NAME \
    -o=yaml | grep -A 7 -B 7 capacity

Replace NODE_NAME with the name of the node.

If the above provided information is not helpful and you are having the Cloud Customer Care package, reach out to Google Cloud Support as it requires more specific debugging and analysis as there is a big difference in the actual and provisioned resources. 

Thanks & Regards,
Manish Bavireddy.