PAGEMAP & migrating from Google Custom Search Site Restricted Json API to Search Vertex AI Search

[evanfr]

I understand Google Custom Search Site Restricted is being depreciated.  We searched for results from specific domains of websites and used microdata, such as PAGEMAP to extract descriptions and images:

PageMaps - Programmable Search Engine Help (google.com)

Providing Structured Data  |  Programmable Search Engine  |  Google for Developers

if 'pagemap' in crawler_object and crawler_object['pagemap']:
pagemap = crawler_object['pagemap']
if 'metatags' in pagemap and pagemap['metatags']:
metatags = pagemap['metatags']
if 'XXXXX' in metatags[0] and metatags[0]['XXXX'] and not "lookaside" in metatags[0]['XXXXX']:
thumbnail_url = metatags[0]['XXXXXX']
elif 'XXXXX' in metatags[0] and metatags[0]['XXXXX'] and not "lookaside" in metatags[0]['XXXXX']:
thumbnail_url = metatags[0]['XXXXX']
if not thumbnail_url and 'cse_thumbnail' in pagemap:
if len(pagemap['cse_thumbnail']) and 'src' in pagemap['cse_thumbnail'][0]:
thumbnail_url = pagemap['cse_thumbnail'][0]['src']

As I migrate, is this data still available? And which do I choose?

Website data OR Structure/Unstructured data?

https://cloud.google.com/generative-ai-app-builder/docs/preview-search-results

https://cloud.google.com/generative-ai-app-builder/docs/preview-search-results#structured-unstructur...

website data:

https://cloud.google.com/generative-ai-app-builder/docs/preview-search-results#web