Influence Looker’s Roadmap and join the BI Customer Council (BICC)
Would you like to influence Google Cloud’s BI roadmap more? I’d like to invite you to join the Google Cloud BI...
Data Analytics
Want more from your data? Have questions about products like BigQuery, Dataflow, or Pub/Sub? You're in the right place. Join conversations around data warehouse modernization, streaming analytics, business intelligence, and more.
Focused on Looker? Engage with the Looker Community here.
Focused on Looker? Engage with the Looker Community here.
Forum Posts
Dataflow requires minimum 4 minutes time for resources provision for processing small data jobs
I have a requirement, where I will be getting a encrypted zip file in GCS bucket. I need to decrypt and unzip ...
Data Loss after sGTM Capi Integration
Hi everyone!We're facing with an issue on new user tracking. We have BigQuery integration and the situation is...
Hi guys I am getting this error runing dataflow with flex-template
Workflow failed. Causes: S30:Read TransferResultDetails from MySQL/Create.Values/Impulse+Read TransferResultDe...
Create operation on Cloud Composer 2 failed
Hello,I'm trying to create Composer2 environment but getting this error, see printscreen, I didn't find any an...
Seeking Guidance: Error Connecting to AWS S3 via BigQuery External Connections
I am currently working on a project that involves leveraging the BigQuery - External Connections feature to ac...
storing the pubsub received messages in firestore database
I want to store the message received from the pubsub topic via pull subscription in firestore database in fire...
Data Discrepancy - Search Ads 360 to BigQuery Data Transfer
Dear All,We are using the BigQuery Data Transfer product to collect data from the Search Ads 360 platform to G...
[BigQuery] Fetching results into a Go struct
I am trying to fetch a string result into a Go struct. I am following the instructions mentioned here under `E...
User does not have permission to access results of another user's job.
Hello,User A is creating a query in bq and then user B wants to read the data (using job ID and location). Use...
Adding Created & updated Timestamps columns in Data Fusion pipeline using wrangler
Hi GCP Community,I'm currently working with Data Fusion and have set up a pipeline to transfer data from SQL s...
Read data from hudi using trino
I need help to read data from hudi using trino in dataproc cluster.
Google Ads Datatransfer transfering way less products than show in the dashboard
Hello everyone,I have a bigquery google ads datatransfer, I'm only interested in the products table.Each day I...
Data Fusion Current Timestamp issue
Hi Google,I'm creating a data fusion pipeline where my source is GCS bucket and I'm processing a CSV file from...
Looker: data error when looker studio send scheduled report
I scheduled looker studio report since January, and it didn't have any issue until last month. But some reason...
Group by aggregate in Cloud data fusion failing
Hi all,I am trying to build a ETL pipeline in Cloud Fusion.When i do an sum aggregate on a field from a bigque...
INFORMATION_SCHEMA.JOBS view does not show referenced_tables for views
I'm trying to find a way to find references to our warehouse views. I see that when a view is referenced in a ...
DATASTREAM MYSQL HIGH LATENCY
Hello, I have been implementing a flow from MySQL in RDS to BigQuery. Everything is well configured: connectio...
Json with Select Distinct
Is it possible to utilize 'Select Distinct' with a table containing a JSON column, especially when dealing wit...
Pipeline provisioning status takes too long
Hi, I'm using CDF with ephemeral cluster on dataproc. The machine has this typology:Master Machine Type:e2, Nu...
Group by, count plug-ins
Hi When I use a pipeline with a "group by" or/and "count" plug-in for more then 1000 records, it fails. Could ...
Asset report from google ads data transfer data in BQ
I need to create a report on asset performance from google ads transfer data in bigquery. The report should be...
Big Query Materialized view : bytes_billed is different when used in a sub query
HiI've got a Materialized view partition by date.When I query one day on the materialized view only I get :- B...
calling a pyspark stored procedure in bigquery takes a long time
is there a way to speed this up? the script itself is very fast, but the action of actually calling the stored...
BigQuery csv file upload error: "Invalid value at 'job.configuration.load.schema.fields[0]' (Map)"
Hi,I have tables for which I upload csv files on a regular basis where I append the csv files to existing tabl...
Not matching results from GA4: campaign_name and purchase_revenue in BQ
Hello there! So lately I've been trying to pull out the data from campaign_name and purchase_revenue_value in ...
Identify most used columns to create partitions and clusters in existing BigQuery workloads
In our BigQuery environment, we have reporting tables that are mostly OBT tables. But they are not partitioned...
Incomplete job metadata when quering BigQuery using Node client
Hey,We use the Node.js client to query BigQuery with a query job. Afterward, we access `job.metadata.statistic...
Load JSON to BigQuery with varying schema
I have a requirement to load 5M Json files to be loaded to BigQuery.This is a nested and repeated column in th...
issue with Writing in BigQuery Table
I have implemented https://cloud.google.com/bigquery/docs/write-api-batch-load Batch load using this and when ...
Google PubSub Schemas Compatibility
Hi, Currently, we are using Google PubSub schemas either to validate events or with BigQuery subscriptions. We...
Articles from Product Experts and Googlers
BigQuery Data Preparation with AI Agents: Smarter, faster, and code-free
Ready to unlock the secrets hidden in your product data? This tutorial demonstrates how BigQuery Data Preparation can clean and transform your raw data into actionable business intelligence, using a realistic example from the Fashion and Beauty industry.
Launched: ABAP SDK v1.10 unleashing powerful AI/ML, eventing, and storage capabilities
Supercharge your SAP systems! ABAP SDK v1.10 unlocks BigQuery AI/ML, simplified event-driven integration, Cloud Storage content repositories & access to 290+ Google Cloud APIs. See what's new.
From idea to viral video: the GenAI workflow you need
What if you could create professional-quality videos with just a fraction of the effort? This blog introduces a radical new GenAI workflow on Vertex AI, using models like Gemini, Imagen3, MusicLM and Veo2 etc to make video creation accessible to everyone.
