BQ Connection parameters to set default dataset project_id

[crazyrevol]

Hi, 

Am looking for some parameters in BQ connection that lets me define billing project_id AND default dataset project_id separately. Can you please let me know what params to use and an example if possible. 

Thank you

[dawnberdan]

Hi @crazyrevol,

Welcome to Google Cloud Community!

Yes, you can define a billing project ID and a separate default dataset project ID in BigQuery connections, though the exact parameters and how they interact can vary slightly depending on the client library or tool you're using.

• Billing Project ID (Cost Center): This is the project where the query execution costs (compute, data processed) will be charged. This is often the primary project associated with your connection.
• Default Dataset Project ID (Data Location/Context): This is the project where BigQuery will look for datasets and tables when you refer to them without explicit project qualification (e.g., SELECT * FROM my_dataset.my_table or CREATE TABLE my_dataset.my_table). If you don't specify a project in your connection parameters, this typically defaults to the billing project. You can also refer to this documentation for more information.

The key to separating them is often by setting the client's default project to your default_dataset_project_id, and then overriding the billing project either at the client level or per query.

Here are a few helpful resources that explain how to configure separate billing and dataset projects in BigQuery:

1. Google Cloud Billing Export to BigQuery: This document will guide you through exporting billing data to BigQuery, including how to set up the billing project and dataset project separately. It’s especially useful if you're managing billing data across multiple projects.
2. Stack Overflow Discussion: This thread dives into using different projects for billing and data access via the Python SDK. It includes examples of setting the quota_project_id to control which project is billed, even when querying datasets in another project.

Was this helpful? If so, please accept this answer as “Solution”. If you need additional assistance, reply here within 2 business days and I’ll be happy to help.