This website uses Cookies. Click Accept to agree to our website's cookie use as described in our Privacy Policy. Click Preferences to customize your cookie settings.
Accept
Reject
Preferences
Log in to ask a question
Get hands-on experience with 20+ free Google Cloud products and $300 in free credit for new customers.
Log in to ask a question

Evaluation of fine tuned model in Vertex AI

Posted on 10-15-2023 11:49 PM
madhan_007
Bronze 2
Bronze 2
Reply posted on --/--/---- --:-- AM

I have Fine Tuned text-bison@001 model and at the time of tuning I have given the evaluation dataset to get the Evaluation details for the tuned model. I couldn't find those evaluation details anywhere. Can you tell me where can I find those test results or point me out in the direction please?

0 2 773
0 Likes
2 REPLIES 2

Posted on --/--/---- --:-- AM
hsbedi
Staff
Reply posted on --/--/---- --:-- AM

Hello!

Model Evaluation is currently in preview and supports base and tuned versions of the text-bison. Below metrics are supported as of today.

Task Metric
Classification Micro-F1, Macro-F1, Per class F1
Summarization ROUGE-L
Question answering Exact Match
Text generation BLEU, ROUGE-L

Please refer to the documentation for details. Thanks!

Posted on --/--/---- --:-- AM
madhan_007
Bronze 2
Bronze 2
Reply posted on --/--/---- --:-- AM

Thanks, But it is the custom Evaluation which we do after the model is tuned and is available in model registry. I'm looking for the evaluation metrics which we should get after tuning the model as we are providing the evaluation dataset at the time of tuning. I'm asking where could I find that?

0 Likes