Regarding the cost calculation of Fine tuning for Gemini 1.0 Pro

[MaxChen1126]

Currently, in Google's documentation, Fine Tuning for Gemini is in preview status, so testing Fine Tuning is free of charge. However, I would like to understand in advance how much it would cost for Gemini PEFT when it becomes generally available (GA) in the future. My current dataset has about 100 entries, and performing Fine Tuning for 3 epochs takes approximately 1 hour and 15 minutes. Is the cost for the machines involved in the training, which are the same as the current PaLM Fine Tuning, also based on a2-ultragpu-8g and 8 NVIDIA_A100_80GB accelerators?

[McMaco · Accepted solution]

Hi MaxChen1126,

You're right, currently testing Fine Tuning for Gemini is free as it's in preview. While Google hasn't released official pricing details for Gemini PEFT yet, here's what we can infer based on existing services and your questions:

Inference on Cost:

Based on your information:

• Machine type: You mentioned a2-ultragpu-8g and 8 NVIDIA_A100_80GB accelerators, which aligns with PaLM Fine Tuning's setup. These are likely expensive machine types.
• Training duration: 1 hour 15 minutes might not be a significant cost factor if the cost per minute is high for these powerful machines.
• Dataset size: A dataset of 100 entries is relatively small, so storage costs might be minimal.

Here's how you can estimate potential costs:

1. Check PaLM Fine Tuning Costs: While not a perfect comparison, look at the pricing for PaLM Fine Tuning on AI Platform Training (https://cloud.google.com/vertex-ai/pricing). This might give you a rough idea of the cost per minute for similar machine types (a2-ultragpu-8g).
2. Consider Training Duration: Multiply the estimated cost per minute by the total training duration (1 hour 15 minutes) to get a ballpark figure for the training job itself.

Recommendations:

• Once Gemini PEFT becomes available, explore the official pricing details on the Google Cloud website.
• Consider using Google Cloud's pricing calculator (https://cloud.google.com/products/calculator) to estimate costs based on different machine types and training durations.
• If possible, experiment with smaller datasets and shorter training times during the free trial period to get a more accurate sense of the resource usage for your specific case.

By following these steps, you can get a better understanding of the potential costs associated with Gemini PEFT for your use case based on your dataset size and training requirements.

Hope this helps. Thanks.