Gen App Builder - BigQuery Structured Data Import Failure

[yauhofung]

When importing structured data from BigQuery into the Gen App Builder data store, 126 out of the 1354 rows failed.

The error logs contain an "Unexpected value of string type. Expected number type." error message.

Upon previewing the table I imported, I realized the import failed because Gen App Builder misinterpreted floating-point numbers as strings.

Note how the BigQuery schema correctly defines the relevant columns as NUMERIC.

[lsolatorio]

Hi @yauhofung, 

Welcome and thank you for your interest with Generative AI.

I understand that you are having challenges importing data from BigQuery due to a numerical format mismatch.

Unfortunately I wasn't able to find a document that specifies this issue but looking at your samples it appears that Generative AI accepts floating-point numbers in this format [integer].[decimal] like 1.43 and is having difficulty processing decimal numbers starting with zero (0) such as 0.4 hence treating it as a string.

You can check this hypothesis by looking at your successfully imported data, see if there are accepted decimal numbers starting with an integer and if there are none starting with zeros. Once validated, a good workaround for this is to convert these floating-point data to STRING prior importing then just have it converted back to FLOAT once successfully loaded.

You can also have this submitted as an issue or feature request or reach out to Vertex AI Support for better assistance.

[yauhofung]

Greetings, @lsolatorio:

Interesting hypothesis, but it doesn't seem to hold true.

Row 96 contains only one floating-point number (8.75), but Gen App Builder still imports it as a string.

Here is a corresponding snippet of the error log:

{"code":3,"message":"Unexpected value of string type. Expected number type.","details":[{"@type":"type.googleapis.com/google.rpc.ResourceInfo","resourceName":"gs://695418092167_us_import_custom/staging9030139258281936279/discovery_custom_documents_000000000000.json:96"}]}
{"code":3,"message":"Unexpected value of string type. Expected number type.","details":[{"@type":"type.googleapis.com/google.rpc.ResourceInfo","resourceName":"gs://695418092167_us_import_custom/staging9030139258281936279/discovery_custom_documents_000000000000.json:97"}]}
{"code":3,"message":"Unexpected value of string type. Expected number type.","details":[{"@type":"type.googleapis.com/google.rpc.ResourceInfo","resourceName":"gs://695418092167_us_import_custom/staging9030139258281936279/discovery_custom_documents_000000000000.json:102"}]}
{"code":3,"message":"Unexpected value of string type. Expected number type.","details":[{"@type":"type.googleapis.com/google.rpc.ResourceInfo","resourceName":"gs://695418092167_us_import_custom/staging9030139258281936279/discovery_custom_documents_000000000000.json:103"}]}
{"code":3,"message":"Unexpected value of string type. Expected number type.","details":[{"@type":"type.googleapis.com/google.rpc.ResourceInfo","resourceName":"gs://695418092167_us_import_custom/staging9030139258281936279/discovery_custom_documents_000000000000.json:104"}]}

Thanks for the suggestions.