Recommendation Engine takes too long to prepare

[Hong5986]

Hi, I am trying to create a recommendation engine using Vertex AI Agent Builder. I followed the "Get started with generic recommendations" tutorial and set up my generic recommendation engine. It had already taken more than 1 week to prepare but it is still not done. What could be affecting it? What am I possibly be doing wrong?

[ruthseki]

Hi @Hong5986,

Welcome to Google Cloud Community!

Review the tutorial thoroughly to ensure you've followed all the steps accurately. Remember that this feature is still in its preview phase, indicating that our Engineering team is continuously working to improve and expand its capabilities. It's possible you may encounter some limitations as we refine the feature.

Furthermore, while it's common for large datasets to increase processing time, a week-long preparation seems unusual. Here's a breakdown of possible issues and how to address them:

1. Data Size and Complexity:

If you're using a significantly larger dataset, that's the most likely cause of the long preparation time. You may test with a smaller subset of your data. It might be representative enough to catch errors, and it'll prepare much faster. Alternatively, randomly sample your data while keeping its distribution to reduce the workload without losing the essence.

In addition, the intricacy of your data can also slow things down. Temporarily remove or simplify features to see if it speeds things up.

2. Resource Allocation:

To optimize your Agent Builder's performance, ensure you're allocating sufficient Google Cloud resources, such as CPU, memory, and GPU. While increasing resources can speed up the process, be mindful of associated costs. Additionally, the geographical region you choose for training can impact performance. Select a region with good resource availability and, if possible, one that's close to where your data is stored for faster data transfer.

3. Check for Errors and Logs:

Check for error messages or warnings in the Agent Builder user interface. Additionally, review the logs in Cloud Logging for insights. These logs can often highlight bottlenecks or errors that may be contributing to the slow preparation time.

4. Connectivity and Other Cloud Services

Verify a stable internet connection and investigate any potential network issues that might be hindering the preparation process. Furthermore, if your data resides in other Google Cloud services such as Cloud Storage or BigQuery, confirm that these services are properly configured and have the appropriate permissions to access your data.

I hope the above information is helpful.