A100 ram limitations

[louisryan]

Can someone please help me understand why the best GPUs google offer (A100) have a fixed CPU RAM of 85GB (only 2x that of the GRAM) and all the other poorer GPU options can go over 300GB. It's terribly frustrating for large dataset training pipelines. Especially when mmdetection libraries don't work well on mutiple GPUs and would rather just use the 1

[josegutierrez]

The NVIDIA A100 GPU must use the a2 Machine series, the lower CPU that a `a2 Machine` is 85GB and the highest a2 machine memory is 1360GB.

Also, You can attach up to 257 TB of local storage to these machine types in this series for applications that require higher storage performance.

You can also use Optional local SSD support: you can get up to 3 TB of Local SSD with `a2 machine` types. This can be used as fast scratch disks or for feeding data into the A100 GPUs while preventing I/O bottlenecks.

You can see further details about the `a2 Machine` series here.