BigQuery ML Evaluation Metrics

[Amyhooo]

Hi,

Wondering for BigQuery ML model evaluation, is it possible to pass custom metrics for model evaluation or model comparison purpose while using BigQuery. I am planning to do a binary classification and have some self defined metrics that I want to use to compare and pick the best model.

Thanks!

[Aris_O]

Thank you for contacting Google Cloud Platform. I understand that you are getting lot of error for your application. My name is Shobhit and I am glad to assist you with this case.

I would like to have some more information so that I can investigate further for your issue.

1.) Can you provide me the configuration files (app.yaml, queue.yaml and snippet of the code which causing the error)
2.) What is your application doing ?
3.) Did the same code was working fine before or not ? If yes then did you made any changes in your code ?
4.) Are you using deferred library for your taskqueue ?

Once I have above information in hand. I will be in a better position to help you further. Let me know if you have any other question or concern. I will be more than happy to help you.

[Aris_O]

Hi @Amyhooo 

Welcome to Google Cloud Community,

You can use the ML.EVALUATE function to evaluate model metrics.

For information about model evaluation in BigQuery ML, see Model evaluation overview.

For information about supported model types of each SQL statement and function, and all supported SQL statements and functions for each model type, read End-to-end user journey for each model.

Thanks!

[Amyhooo]

Hi Aris, Thanks for your reply! I was reading through the documents and saw that in the ML.EVALUATE, for classification it outputs: "Reports the precision, the recall, the accuracy, the F1 score, the log los and the roc auc"

Just wondering for example we have some self defined metrics (formula) to evaluate classification, are we able to set that somewhere and ML.EVALUATE can utilize that for evaluating models?