How to decide the number of node hours before training a new (Gemini Pro Vision) model?

[Joppe]

I want to train a new object detection model with Gemini Pro Vision, and need to decide the number of node hours, keeping costs in mind. I have 28000 labelled images, with roughly 4 labels per image on average.

How do I decide what a good number of node hours is for this job? When will an increase in node hours give deminishing returns?

Thank you!