Multilingual Gen AI

[j_witty]

Hello,

I am working on building a multilingual agent that could include 10 or more language options.  My agent is very gen ai heavy and uses an English-only advanced website indexed datastore as its backend.  I have been building out translation functionality within the prompt itself leveraging Gemini.  Within my 'en' agent everything is working as expected, I can communicate with it in almost any language.  The problems arise when I actually switch the version of my agent to a different default language, 'es' for example.  What I am experiencing is that the datastore is now being hit in Spanish and my rule for translating the $rewritten-query to English before querying the datastore is ignored. 

I am looking for any thoughts on workarounds.  Switching to a webhook based architecture would be an option but increases the complexity and overhead of the solution.  Another option could be to actually translate the datastore itself and use multiple flows based on the language to hit the correct DS.  Problem here is that there would be 100,000+ webpages in my datastore constantly changing.

The ideal solution here would be to figure out how to have control over the $rewritten-query prior to hitting the datastore.

[dawnberdan]

Hi @j_witty,

Welcome to Google Cloud Community!

Making sure the $rewritten-query is properly translated into English before querying the datastore is a bit challenging. Here are some potential approaches and considerations to address this:

1. Pre-query Translation Control

• When the agent's language switches to Spanish, the query ($rewritten-query) in Spanish is being passed directly to the datastore without being translated to English first.
• The key here is to ensure that before hitting the datastore, you can consistently translate the user's query into English. The challenge is making sure that the translation step is effectively inserted in the process flow without having to change the backend architecture.

2. Language Detection and Query Translation Based on Rules

• Before the query reaches the $rewritten-query phase, implement a language-detection step to identify the user's input language. If the language is not English, enforce an online translation step that sends the query through Gemini for translation to English. For example, you might use Cloud Natural Language API or Cloud Translation.
• By managing this process as part of the natural workflow, you could ensure that the query is always translated to English regardless of the default language of the agent.

3. Embedding a Translation Layer

• If you cannot directly control the $rewritten-query process, consider embedding a translation layer between the user input and the query generation. For example:

1. User inputs query in any language.
2. Input is translated to English before being processed into $rewritten-query.
3. Translated query is then passed to the datastore.

• This essentially acts as a filter that ensures English-only queries are submitted.

4. Multi-Language Datastore Strategy

• If preprocessing or intercepting queries proves unfeasible, another option could involve creating a parallel indexing and caching mechanism for common translations of the datastore content. While translating 100,000+ web pages sounds daunting, using dynamic translation on-demand could minimize the need for upfront translation.
• For instance, dynamically translate and cache the most frequently queried content in the top languages your agent supports. This hybrid solution allows for multilingual datastore queries while keeping the effort to update translations manageable.

5. Webhook-Based Query Control

• While you noted that a webhook-based architecture adds complexity, it would give you greater control over the query process. By routing all queries through a webhook, you could manage preprocessing, translation, and customization with precision.
• This approach might align better with a scalable system if you are planning for long-term expansion across multiple languages.

The ideal solution will depend on your scalability requirements, how often the datastore is updated, and the level of complexity you're prepared to handle. You may also find this documentation helpful; it offers insights on building multilingual agents, handling language-specific data, and using AI-generated content for different languages.

Was this helpful? If so, please accept this answer as “Solution”. If you need additional assistance, reply here within 2 business days and I’ll be happy to help.