Get hands-on experience with 20+ free Google Cloud products and $300 in free credit for new customers.

Using Vertex Agent Builder to build a chatbot to answer questions about my website

So I have been trying for almost a week now to create a chatbot using Google Vertex Ai Builder (ex "Search & Conversation" feature of Vertex).

I have created a Data Store linking to my website, so that the bot would read my website and respond to any questions from the users with information from the website. The process of creating, indexing and linking the data store to the app have all been successful.

Whenever I use the "Test Agent" function of Dialogflow, however, the bot gives me either no answer at all or "I'm not sure I understand. Can you rephrase your question?".

Any idea of what might be going on and how to fix it?

Thanks!

Screenshot 2024-09-19 at 18.59.33.pngScreenshot 2024-09-19 at 18.59.42.pngScreenshot 2024-09-19 at 19.03.20.png

 

0 18 2,930
18 REPLIES 18

@fabiodpc Can you share the entire console message?
Sometimes could be the filter when the query contains something bad or forbidden.

Sure thing, Mizar, bellow follows my latest test report:

What are the products or services of V(company name)?
I'm sorry, I don't understand. Can you rephrase your question?
 
What does V(company name) do?
I'm sorry, I don't understand. Can you rephrase your question?
 
Tell me something aboutV(company name).
I'm sorry, I don't understand. Can you rephrase your question?

Hi @fabiodpc,

Welcome to Google Cloud Community!

I understand that you have successfully created, indexed and linked the data store to the app but there's a problem when you use the “Test Agent" function it does not give you the answer intended to the question.

Here are possible key considerations that might help you find the solution to the issue:

  • Check the Data Store Indexing While you mention indexing was successful,make sure Vertex AI Builder can understand the data store and extract meaningful information from it.If you suspect indexing problems, try re-indexing your data store.
  •  Examine Data Structure - The website’s content structure can impact how well the chatbot understands it. Ensure that your website uses clear and well organized content. It will help the chatbot identify key information and relationships within your website’s data. 
  • Check Data Store Relevance - The chatbot needs enough relevant information in the Data Store to answer the user's questions. Your chatbot might struggle to find the answer if the website doesn’t contain the specific information the user was asking about.
  • Check Agent Configuration - The chatbot needs to be trained on the Data Store to learn how to answer questions based on its content. Make sure you have completed the training process. Review the agent’s setting in Dialogflow. If Vertex AI Builder can't determine the user's intent from the question, it won't be able to provide a relevant answer.
  • Test and Debug - Test starting with simple and straightforward questions that are directly related to the content on your website. This helps you isolate whether the issue is within chatbot's understanding of the data or with the way you're phrasing your questions. Also try to use a variety of questions to test the chatbot with different types of questions, including open-ended questions, specific questions that require multiple steps to answer.

You can refer to Vertex AI Agent Builder and Dialogflow documentation for more detailed information. 

I hope the above information is helpful.

Hello MJane, thank you for your suggestions, but none of them worked... The testing agent in DialogFlow continues to reply "I'm sorry, I don't understand. Can you rephrase your question?".


Hi fabiodpc - I had that problem. I solved it by clicking on the gray "Start Page" button on the Dialog page. Once clicked, various options drop down - one of which says "Edit data stores." Click on that and a menu opens up on the right called "Data stores." Choose the data store to display for your "Website." Save all that and wait a few minutes and try asking again. I noticed my agent was very literal - so I asked something verbatim from my website and it worked. Hope that will help you. Good luck.

Thank you for your idea Hubie.
I actually have already done that , but it did not work... I am starting to believe that there may be something odd with the way our website is constructed. I managed to create other chatbots, based on some of our companies documents for instance, but the one based on the website doesn't work. It seems the agent simply cannot read the website.

Does that make sense for you guys? Do you know if there is a type of website that Vertex simply cannot read?

Screenshot 2024-09-27 at 17.27.15.png

Sorry I couldn't help you more. I think it's true what you say re certain triggers preventing access that the agent may encounter as it reads the targeted website. My agent won't read password protected webpages of our association site. It seems only to read what the public sees on the screen, nothing more. My workaround is to also attach the agent to an unstructured documents data store and stick documents in there that it doesn't otherwise read verbatim from the webpages. Extra work but it does help prevent the agent from pulling out hidden private information - so I do appreciate that. Good luck and take care.

It sounds like Hubie is sharing their experience with using the Vertex
Agent Builder to create a chatbot that interacts with their website. They
mention that the agent is limited to reading publicly accessible
information on webpages and cannot access password-protected content. As a
solution, Hubie has connected the agent to an unstructured document store
to ensure the chatbot can access necessary data that isn't readable
directly from the website. This helps maintain privacy and security by
preventing the agent from retrieving hidden or private information.

Is this an AI generate summary of Hubie's message? 🤔

Thanks for sharing, Hubie. I did not manage to fix the issue, but you actually game me a neat idea... I could upload a folder with all of my website's content and direct Vertex to read that instead of the website instead.
It's a workaround, but perhaps it may work.

Thanks for the ideas, everyone!

I still didn't manage to fix this issue, but it is becoming more clear to me that the issue is with my website. I'm going to talk to the website manager to see if he has an idea of what can be preventing Vertex of reading the website.

@fabiodpc you can disable the data stores, For the issue you had, can you share the console message? The json response, there you can see if your getting blocked and thats why your bot is not responding properly

Sure thing, bellow follows the JSON response.

 

{

  "advancedSettings": {

    "loggingSettings": {

      "disableLogging": true

    }

  },

  "currentFlow": {

    "displayName": "Default Start Flow",

    "name": "XXX – information deleted by user"

  },

  "currentPage": {

    "displayName": "Start Page",

    "name": " XXX – information deleted by user "

  },

  "dataStoreConnectionSignals": {

    "rewriterModelCallSignals": {

      "model": "text-bison-002",

      "modelOutput": " The question is asking for the services provided by the company.\nAction: Search\nAction Input: What services does VCompany provide?",

      "renderedPrompt": "Your name is AI Assistant, and you are a helpful and polite AI Assistant at VCompany. Your task is to assist humans on the company website.\n\nYou are consistently efficient with your use of tools.\n\nYou have access to the following tools:\nSearch: This tool returns search results related to VCompany. Use it with the question you must answer and only reformulate the question if it is really necessary.\nno_action: This tool should only be used if no Search is necessary. As its input parameter, it takes the original Question UNCHANGED, with only spelling and punctuation correction.\n\nUse the following format:\nThought: you should always think about what to do\nAction: the action to take, should be one of [Search, no_action]\nAction Input: the input to the action in English\n\nExample:\n\nConversation History:\nQuestion: wassup\nThought: No search is necessary.\nAction: no_action\nAction Input: What's up?\n\nConversation History:\nQuestion: Do you sell product or service?\nThought: I should search to see if VCompany offers product or service.\nAction: Search\nAction Input: Does VCompany offer product or service?\n\nConversation History:\nQuestion: What is product or service and how does it work?\nThought: I should search for information about product or service and how it works.\nAction: Search\nAction Input: What is product or service and how does it work?\n\nConversation History:\nQuestion: How do I perform a task?\nThought: I should search for instructions on how to perform a task.\nAction: Search\nAction Input: How do I perform a task?\n\nConversation History:\nHuman: I want to learn how to code.\nAI: Great! Coding is a valuable skill that can open up many doors for you. What programming languages are you interested in learning?\nQuestion: I'm not sure. What are some popular languages?\nThought: Based on previous history, languages refers to programming languages.\nAction: Search\nAction Input: What are some popular programming languages?\n\nConversation History:\nHuman: I'm looking for a new hobby.\nAI: There are many great hobbies to choose from. What are you interested in?\nQuestion: I like to write and I'm interested in history.\nThought: Based on previous history, writing and history are interests to consider when suggesting a new hobby.\nAction: Search\nAction Input: I'm looking for a new hobby that involves writing and history.\n\nConversation History:\nHuman: I want to automate my living room.\nAI: ... (suggests options)\nHuman: I'm interested in smart lighting and a smart thermostat.\nAI: ... (provides details)\nQuestion: Can you suggest something compatible with my existing smart home hub?\nThought: Based on previous history, something refers to previously mentioned living room automation, smart lighting and smart thermostats.\nAction: Search\nAction Input: Smart lighting and thermostats for a living room compatible with my existing smart home hub.\n\nConversation History:\nHuman: What's the best way to lose weight?\nAI: Exercise and a healthy diet are essential for weight loss.\nHuman: I'm too busy to exercise.\nAI: There are many home workouts you can do without equipment.\nHuman: I don't have much time.\nAI: Some workouts can be done in as little as 15 minutes.\nQuestion: What's an easy one I can do?\nThought: Based on the context, the user is asking about an easy home workout that can be done in 15 minutes or less.\nAction: Search\nAction Input: Easy 15 minutes home workout.\n\nBegin! Let's work this out in a step by step way to be sure we have the right answer.\n\nConversation History:\nHuman: Hello can you tell me the services provided by the company?\nAI: I'm sorry, I'm not sure what you mean. Can you rephrase your question?\nQuestion: The company that you are reading the website of, can you tell me witch services they provide?\nThought: "

    },

    "rewrittenQuery": "What services does VCompany provide?",

    "safetySignals": {

      "decision": "ACCEPTED_BY_SAFETY_CHECK"

    }

  },

  "diagnosticInfo": {

    "DataStore Execution Sequence": {

      "steps": [

        {

          "info": "pageSize: 20, ucsFilter: unset, boostSpec: unset, searchResultMode: DOCUMENTS, maxExtractiveAnswerCount: 5, maxExtractiveSegmentCount: 5, returnExtractiveSegmentScore: true, numPreviousSegments: 0, numNextSegments: 0, enabledFeatures: [RETURN_ANSWER_SNIPPET RETURN_SEGMENT_CONTENT RETURN_SNIPPET_CONFIDENCE_SCORE RETURN_SNIPPET_CONTEXT]",

          "status": {

            "code": "OK"

          },

          "name": "Call Search [log search request parameters]"

        },

        {

          "status": {

            "code": "OK"

          },

          "name": "Responsible AI (with original query)"

        },

        {

          "status": {

            "code": "NOT_FOUND"

          },

          "responses": [],

          "info": "Information deleted by the user",

          "name": "Call Search with original query [website]"

        },

        {

          "status": {

            "code": "OK"

          },

          "responses": [],

          "name": "Convert UCS results for original query"

        },

        {

          "status": {

            "code": "OK"

          },

          "responses": [

            {

              "text": " The question is asking for the services provided by the company.\nAction: Search\nAction Input: What services does VCompany provide?"

            }

          ],

          "name": "Query rewrite"

        },

        {

          "name": "Call Search [log search request parameters]",

          "status": {

            "code": "OK"

          },

          "info": "pageSize: 20, ucsFilter: unset, boostSpec: unset, searchResultMode: DOCUMENTS, maxExtractiveAnswerCount: 5, maxExtractiveSegmentCount: 5, returnExtractiveSegmentScore: true, numPreviousSegments: 0, numNextSegments: 0, enabledFeatures: [RETURN_ANSWER_SNIPPET RETURN_SEGMENT_CONTENT RETURN_SNIPPET_CONFIDENCE_SCORE RETURN_SNIPPET_CONTEXT]"

        },

        {

          "name": "Call Search with rewritten query [website]",

          "responses": [],

          "status": {

            "code": "NOT_FOUND"

          },

          "info": "Information deleted by the user"

        },

        {

          "name": "Convert UCS results for rewritten query",

          "responses": [],

          "status": {

            "code": "OK"

          }

        }

      ],

      "additionalInfo": {

        "ucs_project_number": "Information deleted by the user",

        "rewritten_query": "What services does VCompany provide?",

        "tracking_id": " Information deleted by the user",",

        "user_query": "The company that you are reading the website of, can you tell me witch services they provide?",

        "agent_project_number": "Information deleted by the user”,"

      },

      "executionResult": {

        "response_type": "NO_RESULT",

        "unstructured_citation": false,

        "banned_phrase": "",

        "response_reason": "EMPTY_SEARCH_RESULT",

        "ucs_fallback": false,

        "banned_phrase_check_type": "BANNED_PHRASE_CHECK_TYPE_UNSPECIFIED",

        "faq_citation": false,

        "language": "en",

        "latency": 0,

        "website_citation": false

      }

    },

    "Response Id": "Information deleted by the user”,

    "Session Id": "Information deleted by the user”,",

    "Execution Sequence": [

      {

        "Step 1": {

          "InitialState": {

            "Event": "sys.no-match-default",

            "SessionParameters": {

              "$request.generative.response": "I'm sorry, I'm not sure what you mean. Can you rephrase your question?"

            },

            "FlowState": {

              "FlowId": "00000000-0000-0000-0000-000000000000",

              "PageState": {

                "PageId": "START_PAGE",

                "Status": "ENTERING_PAGE",

                "Name": "Start Page"

              },

              "Name": "Default Start Flow",

              "Version": 0

            }

          },

          "Type": "INITIAL_STATE"

        }

      },

      {

        "Step 2": {

          "StateMachine": {

            "FlowState": {

              "FlowId": "00000000-0000-0000-0000-000000000000",

              "Version": 0,

              "PageState": {

                "Status": "ENTERING_PAGE",

                "PageId": "START_PAGE",

                "Name": "Start Page"

              },

              "Name": "Default Start Flow"

            },

            "TriggeredEvent": "sys.no-match-default",

            "TriggeredEventHandlerId": "Information deleted by the user”,"

          },

          "Type": "STATE_MACHINE",

          "FunctionExecution": {

            "Responses": [

              {

                "responseType": "HANDLER_PROMPT",

                "text": {

                  "containAttemptedDataStoreInteractions": true,

                  "containGenerativeFallback": true,

                  "text": [

                    "I'm sorry, I'm not sure what you mean. Can you rephrase your question?"

                  ],

                  "containAiGeneratedContent": true

                },

                "source": "VIRTUAL_AGENT"

              }

            ]

          }

        }

      }

    ],

    "Triggered Transition Names": [

      "Information deleted by the user”,"

    ]

  },

  "generativeInfo": {

    "actionTracingInfo": {

      "actions": [

        {

          "userUtterance": {

            "text": "The company that you are reading the website of, can you tell me witch services they provide?"

          }

        },

        {

          "agentUtterance": {

            "text": "I'm sorry, I'm not sure what you mean. Can you rephrase your question?"

          }

        }

      ],

      "conversationState": "OUTPUT_STATE_PENDING",

      "name": "Information deleted by the user”,

    }

  },

  "intentDetectionConfidence": 0.3,

  "languageCode": "en",

  "match": {

    "confidence": 0.3,

    "event": "sys.no-match-default",

    "matchType": "NO_MATCH"

  },

  "parameters": {

    "$request.generative.response": "I'm sorry, I'm not sure what you mean. Can you rephrase your question?"

  },

  "responseMessages": [

    {

      "responseType": "HANDLER_PROMPT",

      "source": "VIRTUAL_AGENT",

      "text": {

        "containAiGeneratedContent": true,

        "containAttemptedDataStoreInteractions": true,

        "containGenerativeFallback": true,

        "redactedText": [

          "I'm sorry, I'm not sure what you mean. Can you rephrase your question?"

        ],

        "text": [

          "I'm sorry, I'm not sure what you mean. Can you rephrase your question?"

        ]

      }

    }

  ],

  "text": "The company that you are reading the website of, can you tell me witch services they provide?"

}

@fabiodpc 

        "response_reason": "EMPTY_SEARCH_RESULT",

        "ucs_fallback": false,

        "banned_phrase_check_type": "BANNED_PHRASE_CHECK_TYPE_UNSPECIFIED",

As you see seems that your data store doesnt know what to do, and thats why your not getting the response. So add more data and train again

But what do you mean "add more data"? 
It's a website, with a lot of content... Btw, the website is www.valetec.com.br, maybe that helps.

kindly check your data ingestion.

Was this issue resolved? 

I'm considering using the Vertx AI Agent builder for my website's chatbot and the knowledge base would come directly from my company's webpages. I've been researching this builder but it seems buggy, and hard to manage for someone like me with NO developer skills. I like the solutions it offers and that I can continue to be supported by the Google ecosystem on which my business is currently run, but I don't want to create work for myself when there are lots of NO CODE alternatives out there.

Hello, the issue was not resolved, up until now, I still could not manage to get my website chatbot to work using Vertex AI and the Dialogflow CX interface. Some months have passed, to be honest I have given up on this project, it should work but for some reason it simply did not (I tried most of the things other people suggested here but without success...)