BigQuery Subscription writes FLOAT and BOOLEAN data incorrectly to BigQuery

[cagriyonca]

Hello guys,

I'm working with Pub/Sub's BigQuery Subscription on different kinds of data types and there are several ambiguities here:

- When we send float data with a 0.0 value, it is stored as Null at BigQuery table (data is stored in FLOAT column). How will we know if incoming data has come as 0.0 or it hasn't arrived?

- When we send float data other than 0.0, sometimes it is getting additional decimals. For example, I'm sending 0.1 as a value but BigQuery stores it as 0.10000000149011612. Where does this precision come from and why? What kind of logic rounds value to an uncertain decimal value?

- When we send boolean data, if it's not true, BigQuery stores it as null. How will we know if incoming data has come as false or null?

Best regards,
Cagri

[cagriyonca]

Is there anybody from Google Dev Team?

[josegutierrez]

There is a current issue regarding what you mention, the engineering team is aware of this and is working on it.

[doctormiguel]

any news on this?

I am having this exact problem with pubsub / subscription / bigquery.

The field in my proto schema looks like this:

syntax = "proto3";

float selection = 14;

 in bigquery the definition looks like this

{
  "name": "selection",
  "type": "FLOAT64",
  "mode": "NULLABLE"
},

.. when I push a JSON encoded message with a field like this

"selection": 1.825,

in bigquery I get something like:

1.8250000476837158