Vertex AI Datasets & batch predictions

[arny]

Hi all,

I just started to play with Vartex AI. I am working with "Tabular" - "Forecasting" and currently struggling with few things and i hope you can help me in order i can continue. I tried to organize my questions to three categories:

1) Datasets for training: 

     a) "series identifier" define to which time series data are belonging ... lets assume that i have two series identifier - one is called "A" and one is called "B". Does this means that AI treats them as completely separate and noncorrelated - this means any data that belongs to series A don't have any correlation to B, right? This give me possibility to train different dataset with one shot right? Otherwise i would need to make (in my case) two trainings - one for A and one for B.

2) Training new model --> Model details

a) Is possible to predict more then one target column? 

b) lets assume that my dataset data granularity is 1 day. Can i use data granularity of "5min" for Forecast configuration or can this setup decrease quality of my forecast? Should it be more correct to use already at beginning lets say dataset granularity of 5 minute and afterwards it could be more flexible when setting data granularity for forecast configuration without influencing forecast quality?

c) If I set Forecast horizon of 7 and context window 30, does this means that this setting limit my forecast to maximum 7 time steps and requesting always exactly 30 time steps of historical data as input when forecasting on existing trained model?

3) Batch predictions

a) Batch Source file

Lets assume that i have data with 15 columns from which one is "serial identifier" , one is "time step" - actually date and one of those columns is target column. Rest of 12 columns are used as influencer and used to train my model. I know that i need to have same structure for batch source file - i read that i can use same file as i used for training, but i just need to add in my case a 2x7 new rows with adding 7 dates and serial identification (in my case 7x A and 7xB) and target column need to be empty. But what should i do with data of rest 12 columns? Do i need manually to enter data for those new rows (2x7) of those 12 columns which values should be for future? But what if i don't have those data? Does this means i cannot do prediction?

I tried to make prediction without those future data and i got following message:

"There are rows with non-empty target values after this row. The time series has been excluded from predictions."

I hope you can help me with above questions.

Tnx in advance!

Regards,

Arny

[josegutierrez]

A)
Yes, that's correct you can train different datasets with the same shot (only if there is a specific column that differentiates them). Follow this document for best practices.

B)
No, for datasets that train AutoML models, one column must be the target, and there must be at least one feature available to train the model.

When you train a forecasting model, you specify the data granularity, or the time interval between the training data rows. It can be hourly, daily, weekly, monthly, or yearly. In addition, it can be every 1, 5, 10, 15, or 30 minutes.  Vertex AI treats the interim day as missing data, which can degrade model performance.

C)
For this error what you could do is to copy the table into another and set the value to NULL to the other columns or set the data values manually.

[om_shah]

Hi josegutierrez,

Can you please elaborate on your "C" answer. I am bit stuck at this error "There are rows with non-empty target values after this row. The time series has been excluded from predictions.". Also can you please include an example too!! 

Thank You,

Om

[lambdalove]

I'm also stuck on the same error. The target column is empty for every row in my forecast horizon, so I'm really at a loss. I've posted my own question about this here:

AutoML forecast model batch predictions: "rows with non-empty target values" error

@om_shah Not sure if this helps you, but in response to your asking for an example, you might want to check out this guide if you haven't already. It includes sample data for requesting batch predictions.
(Again, not sure how much this helps, because in my case I'm seeing the error message even though my target column is, in fact, empty where I expect the model to make predictions.)

[om_shah]

Hi lambdalove,

Yes!! I have checked your post and the guide. I have repeated same steps from the guide but no luck. Waiting for the answer.

[om_shah]

Hi @josegutierrez, @arny ,

Waiting for your reply. I am having an urgency to complete this. 

Thank You, 

Om