INT values when importing CSV data from Funnel to BigQuery

[Nathan_Alves]

I have a rather unexpected problem with BigQuery

When importing CSV data from Funnel to BigQuery. I see that BigQuery changes the values of my INT columns during the load process.

I use Funnel to export data to BigQuery, and Funnel correctly sends data in CSV format. However, after the data is loaded into BigQuery, I notice changes in the values of my INT columns.

I have done extensive checks and I am convinced that the problem is not on Funnel's side, as the CSV data sent by Funnel is correct. INT values are correctly formatted and show no issues when exported from Funnel.

However, when importing into BigQuery, the values of the INT columns are changed. I don't see this problem with other data types like STRING or DATE, which suggests the problem is specific to INT columns.

I wanted to know if anyone could know why I have this error and why BigQuery changes the values of my INT columns when importing CSV data from Funnel.

Thanks in advance

CSV sent from then Funnel :

Data CSV

Data received by BigQuery :

[ms4446]

I see, that's puzzling indeed. The integers for the "Impressions" column are in plain digit format without any commas or other symbols, and there don't appear to be any missing or NULL values.

The schema for this data should be something like:

• Date: DATE
• SMC___Name: STRING
• Impressions: INT64

The integer values in your sample data are not particularly large, so INT64 should be more than sufficient to handle them.

Here are a few additional steps you can take:

1. Try INT64: If you're not already using INT64 for the "Impressions" column in your BigQuery schema, try changing it to INT64.

2. Check for hidden characters: Sometimes, CSV files can contain hidden characters or extra whitespace that isn't visible when you're looking at the data. This could potentially be causing issues. Try using a tool to inspect the CSV file for any hidden characters.

3. Try a different import method: If you're currently using the BigQuery web UI to import the data, try using the bq command-line tool instead, or vice versa. This can sometimes help to identify if the issue is related to a specific import method.

4. Create a new table: Create a new table in BigQuery with the correct schema and try importing the data into this new table. This can help to rule out any issues related to the existing table.