Solved: BigQuery Storage Write API and multiple tables - Page 2

diegobx · 08-14-2023 08:41 PM

Hi everybody,

I am implementing a service in GO that will stream incoming logs (from other services) into BigQuery, using the new Storage Write API.

My question is how to handle writing to different tables (the destination table is decided by the GO service based on some attributes of the incoming logs).

Should the GO service open a separate BigQuery write stream for each table, or it's possible to write to different tables within the same write stream? Thanks!

Diego

ms4446

When using the BigQuery Storage Write API, each write stream is associated with a specific table. Therefore, if you want to write to multiple tables, you'll need to manage multiple write streams, one for each table.

You cannot write to different tables within the same write stream. Each stream is bound to a specific table at the time of its creation.

For implementing this in Go, you'd use the BigQuery Storage Write API's Go client library. The actual methods and usage might differ from the standard BigQuery API. I recommend referring to the official Google Cloud documentation and the Go client library's reference for accurate methods and examples.

View solution in original post