How to determin which GCP VM do I need for ML

[holguinmora]

Hi to all

Im trying to run a procedure looking to reduce the number of features for a model.

The first try was with google Colab pro+ but it keep crashing and nver run the entire process, then I got a VM n1-highmem-8 that has: 

GPUs1 x NVIDIA Tesla V100  +  n1-highmem-8 (vCPUs: 8, RAM: 52GB)

and still not getting the process done.

The question is how to determin which type of machine should I use? Can I get any metric from the cell that is runing in colab and be able to determin the Type of VM that I need?

[grobledo · Accepted solution]

There are a few things to take in consideration:

1. Have you installed all the necessary drivers for the GPU? Here is a complete guide that you can follow.
2. I do not see any Python wrapper for CUDA in your code. The way you specify when to use the GPU for specific tasks is through this wrapper, it seems to me that you are using the CPU instead and that is why the task keeps crashing. Now, converting your code to a CUDA version is not a trivial task, and it involves a deeper knowledge on how a GPU works. If you are in a hurry, you could try the Py2CUDA github project, but I would strongly recommend taking a look at the Getting Started Blogs.  

[holguinmora]

I sincerely appreciate your response, you cannot imagine how important and valuable your help has been