BigQuery efficient data extraction

[DimaJuravel]

Hi! 

I want to read data from BigQuery table to 3'rd party application to process the data.

I'm trying to understand which way is cheaper:

• BigQuery Read Storage Streaming
• BigQuery REST API

on official documentation says that Query on BigQuery (as I understand BigQuery REST API uses a Query mechanism) will cost $6.25 per TiB (for US)

and Streaming reads (BigQuery Storage Read API) will cost $1.1 per TiB read for (US)

I have a couple questions:

1) Does the price of Query include data transportation or it's just price for computing query on Google's server and I need to pay something else to get out it from Google space?

2) I found that Storage Read API Data Transfer to Worldwide Destinations (excluding Asia & Australia) will cost $0.08 per GiB and customers can read up to 300 TiB of data per month at no charge. It means that after 300TiB I'll be able to read data by $0.08 per GiB? Or 300 TiB has another meaning? and what about price $1.1 per TiB read?

I cannot guarantee that BigQuery instance and third party app will be in the same location  and what amount of data I need to process). But let's imagine the worst case: different countries and 500 TiB per month

[jangemmar · Accepted solution]

Hi @DimaJuravel,

Welcome to Google Cloud Community!

To answer your first question, query costs in BigQuery do not include external data transfer charges. The $6.25 per TB price you mentioned covers the computational cost of processing your query within BigQuery's infrastructure.

When you transfer data out of BigQuery to an external destination like your 3rd party application, egress charges apply. The specific cost depends on:

• Source location of your BigQuery data
• Destination location of your 3rd party application
• Network used (Google Cloud's network is generally cheaper than going over the public internet)

Regarding your second question, you’re right, you won’t be charged for the first 300 TB of data you transfer from BigQuery within a month. After that, the $0.08 per GB rate kicks in. The $1.1 per TB read is the cost for using the Storage Read API itself to access your data. 

Note: Leverage Google Cloud's network when transferring data within Google Cloud services for lower costs.

I hope the above information is helpful.

[jangemmar · Accepted solution]

Hi @DimaJuravel,

Welcome to Google Cloud Community!

To answer your first question, query costs in BigQuery do not include external data transfer charges. The $6.25 per TB price you mentioned covers the computational cost of processing your query within BigQuery's infrastructure.

When you transfer data out of BigQuery to an external destination like your 3rd party application, egress charges apply. The specific cost depends on:

• Source location of your BigQuery data
• Destination location of your 3rd party application
• Network used (Google Cloud's network is generally cheaper than going over the public internet)

Regarding your second question, you’re right, you won’t be charged for the first 300 TB of data you transfer from BigQuery within a month. After that, the $0.08 per GB rate kicks in. The $1.1 per TB read is the cost for using the Storage Read API itself to access your data. 

Note: Leverage Google Cloud's network when transferring data within Google Cloud services for lower costs.

I hope the above information is helpful.

[DimaJuravel]

Hi, Jangerman! Thanks for your reply!

Just to clarify: if I will use Storage Read API Data I'll pay $0.08 per GB + $1.1 per TB after first 300 TB? Or should I always pay $1.1 per TB for using the Storage Read API? 

And is the first  300 TB available for transfer data out of BigQuery to an external destination like 3rd party application? Or is it just for Google internal purpose?

[jangemmar · Accepted solution]

  @DimaJuravel  you can read up to 300TB of data per month at no charge by using the BigQuery Storage Read API. The $1.1 per TB read will kick in beyond the 300TB. The $0.08/GB is the batch export data transfer pricing (Asia - Asia). You can refer to this table for detailed pricing for each location.