Influence Looker’s Roadmap and join the BI Customer Council (BICC)
Would you like to influence Google Cloud’s BI roadmap more? I’d like to invite you to join the Google Cloud BI...
Data Analytics
Want more from your data? Have questions about products like BigQuery, Dataflow, or Pub/Sub? You're in the right place. Join conversations around data warehouse modernization, streaming analytics, business intelligence, and more.
Focused on Looker? Engage with the Looker Community here.
Focused on Looker? Engage with the Looker Community here.
Forum Posts
Datastream unsupported events
So I have been using MySQL to GCS Datastream pipeline. When I was inspecting the job , I found out that as sho...
Difference between Basic and Enterprise Cloud Data Fusion Edition
Dear team,As per the details given in the pricing page link (https://cloud.google.com/data-fusion/pricing) not...
MIgrating from Dataform Web to BQ Dataform
I am migrating from Dataform web to BQ Dataform on GCP. I have created the Dataform repository, connected to m...
Unknown problem with connection to Airflow Database, server closed connection unexpectedly.
I have daily ingest pipeline as production runs on Cloud Composer, the code is unchanged for a while. One day ...
Could I achieve schema on read approach when loading data to Google BQ (BigQuery)?
So I am exploring of fully using BQ as my primary storage for Data Lakehouse architecture pattern.However, one...
Repository could not be reached, Dataform connected to Gitlab
HiI have implement data transformation pipeline on Google Cloud Composer, by using1. "DataformCreateCompilatio...
Discrepancy between Traffic by API and API Requests
As shown in the screenshots above there is a huge difference between the number of requests for a 30 day perio...
How to transform Table in us-central1 to table in US data location
Hello,I have a table in us-central1 which is referenced in Dataform. I want to apply some transformation on it...
What are Dataform's key features?
Dataform core: an open source, SQL-based framework to manage data transformations and configure new tables tha...
Dataflow: Failed to start the VM
HelloIs the Dataflow service available? I'm getting the error ": "Failed to start the VM, launcher-20231004082...
Please help me understand billing, slots, and auto-scaling
How can we turn off auto-scaling in BigQuery of slots (or other way to limit usage of slots to MAX 100).We hav...
Can't create BigQuery query jobs in nextjs project
Someone please help me , i'm desperate and I'm just a coop student working for a startup... being stucked for ...
Big Query Update
My company has established a table in a BigQuery sandbox account to store data pertaining to various site even...
Need (workaround) solution to improve BQ table reads in Apache beam sdk 250 java [Urgent]
It takes 1min 5 sec to read 19 rows of BQ full table scan that has around 10 columns (size is 15kb) with Apach...
Can create a procedure in BigQuery
Hello!I have the error during creating procedure and can't understand why.
Data Lakehouse Medallion Architecture: bronze/silver/gold: Silver Zone and data persistency
When it comes to Data Lakehouse Architecture and promoting data from Bronze (raw) to Gold (Curated), what are ...
Unable to View BigQuery Editor
Hi,I am trying to use BigQuery Soundbox but am unable to access the BigQuery explorer pane and editor tab afte...
stored procedure to refresh reporting data with data transferred in after the fact?
Hi All,Use case: My current process (inherited) involves a daily data transfer from sa360 for a number of digi...
GA4 Export issue
Hi all ,I am trying to analyse the number of transactions from the big query export of ga4 analytics to big qu...
pubsub emulator: how to set up dead-letter queue
Hi, "dead-letter queues" have been supported by the emulator since July 2021: https://cloud.google.com/sdk/doc...
BigQuery -> Firestore
Hi, a number of our teams want to export some data being ingested and processed in BigQuery to Firestore, AKA ...
Dataform external configuration
Hello everyone,My team is grappling with the challenge of scaling our Dataform code both robustly and as autom...
Total processed Billed Bytes at processing data at table object level in Bigquery
Hi,I see information schema JOBS view has "total_bytes_billed" at query/job level. May i possibly find the sam...
Query Google Trend using Big Query
I need help with google BigQuery. I am using big query to query data from Google Trends.I want to get data usi...
BQ I/O Connector - Storage Read API (Direct table access) - Performance degrade seen in Apache B2.50
Hello, We are seeing Dataflow pipelines taking 2x to 3x more time to run in Apache beam SDK ver 2.50 compared ...
biq.blue: Destroy Your Big Query Costs !
Hi!I've been working on Biq Blue a tool engineered to analyze your Google BigQuery tables, storage, and reques...
Date of Birth Compare Current date
Hi, im working on a bigquery to extract accurate total days, months and years from date of birth to current da...
Query google trend using google BigQuery
I need help with google BigQuery. Am using big query to query data from Google Trends. now I want to get data ...
Dataproc, Error when submit a file to dataproc serverless
Hi, when I try to submit a python file to batch I had the next error:The problem was on the path created for t...
Will Dataform support BQ ML as well?
My organization started utilizing BQ ML. While I assume it would be possible to run BQ ML commands with `type:...
Articles from Product Experts and Googlers
BigQuery Data Preparation with AI Agents: Smarter, faster, and code-free
Ready to unlock the secrets hidden in your product data? This tutorial demonstrates how BigQuery Data Preparation can clean and transform your raw data into actionable business intelligence, using a realistic example from the Fashion and Beauty industry.
Launched: ABAP SDK v1.10 unleashing powerful AI/ML, eventing, and storage capabilities
Supercharge your SAP systems! ABAP SDK v1.10 unlocks BigQuery AI/ML, simplified event-driven integration, Cloud Storage content repositories & access to 290+ Google Cloud APIs. See what's new.
From idea to viral video: the GenAI workflow you need
What if you could create professional-quality videos with just a fraction of the effort? This blog introduces a radical new GenAI workflow on Vertex AI, using models like Gemini, Imagen3, MusicLM and Veo2 etc to make video creation accessible to everyone.
