Fine tuned model in Vertex AI studio and use Python to use the model

[GG-Araujo]

Good morning! I hope everyone that is reading this message has a great day!

Basically, I deployed a fine-tuned model in Vertex AI Studio, and I am using my credentials in Python to use the model that I created. 

However, when I submit a request to my fine-tuned model, for example "give me the recipe of a chocolate cake", sometimes I get normal and good answers, but there are other situations where the model retrieves empty or incomplete answers. Does anyone know how to fix this, please?

Kind Regards.

[cassandramae · Accepted solution]

Hi @GG-Araujo,

Welcome to Google Cloud Community!

It’s possible that there is something causing the model to fail intermittently. With this, you may check the possible cause and solutions below that may help:

• Parameter Tuning:
  • Experiment your model’s temperature:
    • Start with a low temperature to make the model more deterministic and predictable. Increase it gradually (e.g., 0.2, 0.3, 0.4) until you find a balance between consistency and creativity. You may check this documentation for the temperature range and default value of each model.
    • In your Vertex AI Studio deployment settings or your Python API call, set the temperature parameter explicitly. If it's not set, it's using the default (often 1.0), which is usually too high for consistent responses.
• Data Analysis:
  • Review Your Fine-tuning Data: Examine your training data for inconsistencies or biases.
  • Augment Your Data: Add more examples of chocolate cake recipes to your fine-tuning dataset, especially examples that cover a wider range of ingredients, styles, and levels of detail. Consider using data augmentation techniques (e.g., paraphrasing existing recipes, adding variations in ingredient quantities) to increase the diversity of your dataset.
• Prompt Refinement:
  • Consider being more specific in your prompts

Was this helpful? If so, please accept this answer as “Solution”. If you need additional assistance, reply here within 2 business days and I’ll be happy to help.