Hi All,
I am trying to create BigQuery Subscription from Pub/Sub topic to ingest data into BigQuery based on https://cloud.google.com/pubsub/docs/create-subscription#subscription
First, I created an empty table on BigQuery:
CREATE OR REPLACE TABLE tibrahim_debezium.mysql_inventory_products (
id INT64 NOT NULL,
name STRING,
description STRING,
weight FLOAT64,
__op STRING,
__table STRING,
__source_ts_ms INT64,
__deleted STRING
)
Then, I created a Pub/Sub schema using this command:
gcloud pubsub schemas create mysql.inventory.products-schema \
--type=AVRO \
--definition='
{
"type" : "record",
"name" : "MysqlInventoryProductsSchema",
"fields" : [
{
"type": "int",
"optional": false,
"name": "id"
},
{
"type": "string",
"optional": false,
"name": "name"
},
{
"type": "string",
"optional": true,
"name": "description"
},
{
"type": "float",
"optional": true,
"name": "weight"
},
{
"type": "string",
"optional": true,
"name": "__op"
},
{
"type": "string",
"optional": true,
"name": "__table"
},
{
"type": "int",
"optional": true,
"name": "__source_ts_ms"
},
{
"type": "string",
"optional": true,
"name": "__deleted"
}
]
}'
The schema successfully created.
I continue created Pub/Sub topic with above schema.
gcloud pubsub topics create mysql.inventory.products --message-encoding=json --schema=mysql.inventory.products-schema
The topic successfully created.
Next, I created the BigQuery subscription using below command:
gcloud pubsub subscriptions create mysql.inventory.products-bq-sub --topic mysql.inventory.products --bigquery-table=<PROJECT_ID_REDACTED>.<DATASET_REDACTED>.mysql_inventory_products --use-topic-schema
It returned error: ERROR: Failed to create subscription [projects/<PROJECT_REDACTED>/subscriptions/mysql.inventory.products-bq-sub]: Incompatible schema type for field weight: DOUBLE vs. FLOAT.
The doc https://cloud.google.com/pubsub/docs/bigquery#avro-to-zetasql showing that float should be fine.
What could be the issue here?
Solved! Go to Solution.
Yes, confirming this is now fixed.