Agent Builder | Complications using structured metadata with advanced website indexing

[NicolaiRoland]

I am currently working on a project utilizing Vertex AI Agent Builder's website functionality in order to create a site search application.

Currently i have been able to enrich the website agent with some of the metatags from our website. However, i am currently facing issues adding opengraph og:image metatag. Looking at the documentation for Agent builder: link it clearly describes how common metatags indexable inlcudes og:image.

i have tried a update of the data store schema using the following method before recrawling the site (project and datastore id excluded): 

curl -X PATCH \
-H "Authorization: Bearer $(gcloud auth print-access-token)" \
-H "Content-Type: application/json" \
"https://eu-discoveryengine.googleapis.com/v1beta/projects/project_id/locations/eu/collections/default_collection/dataStores/datastore_id/schemas/default_schema" \
-d '{
  "structSchema": {
  "type": "object",
  "properties": [{
    "og_image": {
      "type": "array",
      "items": {
        "type": "string",
        "searchable": true,
        "retrievable": true,
        "indexable": true,
        "siteSearchStructuredDataSources": ["METATAG"]
      }
    }
  }],
  "$schema": "https://json-schema.org/draft/2020-12/schema"
}
}'

Has anyone had any experience adding structured metadata to their datastore, thus able to provide any inputs on the issue?

Thanks in advance!

[ibaui]

Hi @NicolaiRoland,

Welcome to Google Cloud Community!

It seems you're trying to enrich your website's metadata in Vertex AI Agent Builder, specifically focusing on adding Open Graph (OG) images as part of the structured metadata for site search. You've taken the step of updating the schema of your datastore to include the og:image metatag so that it can be indexed and retrieved during searches. Despite following the documentation, you're facing challenges in making this work. With regard to the issue you are encountering, here are some things you can consider to address the issue: 

Verify the og:image meta tag on your webpage: Ensure the <meta property="og:image" content="..."> tag is correctly present within the HTML source code of the pages you're trying to index. Double-check the URL within content is fully correct and accessible.

Data Store Crawling: Ensure that the data store is being properly crawled after you've updated the schema. You mentioned that you tried recrawling the site, but verify that the crawl process is completing successfully and picking up the newly added og_image data. Check the logs to ensure there are no issues with the crawling process.

Test with a simple page: Create a very simple HTML page with only the essential meta tags, including og:image. Index this test page to isolate if the problem is in your website's complexity or the Agent Builder configuration.

Just to add, the 'Use structured data for advanced website indexing' is currently in Preview, which means it is still under development and may not yet offer the expected quality. Preview features often come with certain limitations and might not provide the full range of quality or functionality available in the final product. It may have bugs or unexpected behaviors. You can expect the quality to improve as the feature matures. 

I hope the above information is helpful.