Solved: Alternative after termination of IoTCore service u... - Page 2

fuji1 · 04-06-2023 06:03 PM

Hello.

I am currently sending data from an edge device to BigQuery with the following configuration.
Edge device ➡ (MQTTS) ➡ IoTCore ➡ Pub/Sub ➡ BigQuery

However, I am looking for an alternative as IoTCore will be out of service in 2023/8.
I found this article but could not find a solution.

I have two questions.
(1) If I follow this article and build a standalone MQTT broker in Compute Engine or GKE, how does Pub/Sub connect to the MQTT broker?
(2) Are there any other ways to send data from an edge device to BigQuery using MQTT?

Thank you in advance for your help.

pthiagar

Hello @fuji1

There's a GCP provided Dataflow streaming pipeline template that reads MQTT topic messages and writes them to Pub/Sub. See documentation here: https://cloud.google.com/dataflow/docs/guides/templates/provided/mqtt-to-pubsub

Alternatively, you could define your own batch or streaming data pipeline using Apache Beam and run the pipeline in Dataflow.

Hope this helps.

View solution in original post

Alternative after termination of IoTCore service using MQTT