BigQueryML Explainability Apparently Not Working

[shanif]

I'm using BigQueryML to train an XGBoost model on some of my data. When I create the model, I set the ENABLE_GLOBAL_EXPLAIN flag to TRUE, the model then trains properly and I can evaluate it. However there is no Interpretability tab on the model's page, and when I try to query the model with the ML.GLOBAL_EXPLAIN command, I get an error that says:

Invalid table-valued function ML.GLOBAL_EXPLAIN In function GLOBAL_EXPLAIN, the input model was not explained when it was created. at [4:3]

Is this a bug or am I doing something wrong?

Here's my create model code:

CREATE OR REPLACE MODEL `apteo-gcp.2539775073233929517.gb_prod_recos5`
OPTIONS(
    INPUT_LABEL_COLS=['next_product_id'], -- label of future products purchased
    MODEL_TYPE='BOOSTED_TREE_CLASSIFIER', -- gradient boosting using xgboost
    CLASS_WEIGHTS=[STRUCT('gid://shopify/Product/6995522453704', 265),STRUCT('gid://shopify/Product/6995522355400', 265),STRUCT('gid://shopify/Product/6995522715848', 265),STRUCT('gid://shopify/Product/4448707969120', 100),STRUCT('gid://shopify/Product/6970658717896', 100),STRUCT('gid://shopify/Product/5688760467623', 100),STRUCT('gid://shopify/Product/4448711213152', 100),STRUCT('gid://shopify/Product/5430667804839', 100),STRUCT('gid://shopify/Product/5651893158055', 100),STRUCT('gid://shopify/Product/6995568918728', 125),STRUCT('gid://shopify/Product/4405707243616', 100),STRUCT('NO ORDER', 0.05)], 
    LEARN_RATE=0.15,
    L2_REG=2.0,
    L1_REG=1.5,
    EARLY_STOP=TRUE,
    MAX_ITERATIONS=75,
    MIN_REL_PROGRESS=0.0001,
    ENABLE_GLOBAL_EXPLAIN=TRUE,
    DATA_SPLIT_METHOD="RANDOM"
)

AS

...

[dikaur]

You are using the right syntax for creating the model  as per the examples mentioned in documentation[1].

=> Some possibilities for the error are :

- Models with the same id were re-training again with the script during the evaluation.

- Evaluation data was not provided (might have been explicitly disabled)

You can  open up details about the model in the UI in BigQuery, under Evaluation and verify what details it shows. Also, you can try to rerun the model again (creating, training, evaluating) and verify if it works for you.

[1] https://cloud.google.com/bigquery-ml/docs/reference/standard-sql/bigqueryml-syntax-create#create_mod...

[shanif]

Hi @dikaur 

Thanks so much for your response!

I did try to train new models with new names, and as you can tell from the code above, we do provide the "RANDOM" method for data evaluation. In addition, on the model's page, there is information about the model's evaluation, so I believe it is receiving an evaluation dataset. I've also tried several other models with several different datasets and unfortunately they are all seeing the same error.

This was working about 2 months ago so it's possible that something changed recently.

Are there any other possible explanations for something that I'm doing wrong?

Thanks!