Influence Looker’s Roadmap and join the BI Customer Council (BICC)
Would you like to influence Google Cloud’s BI roadmap more? I’d like to invite you to join the Google Cloud BI...
Data Analytics
Want more from your data? Have questions about products like BigQuery, Dataflow, or Pub/Sub? You're in the right place. Join conversations around data warehouse modernization, streaming analytics, business intelligence, and more.
Focused on Looker? Engage with the Looker Community here.
Focused on Looker? Engage with the Looker Community here.
Forum Posts
Should I limit the size of the bigquery table raw_changelog table?
I linked it with Google data studio using "Stream Collections to BigQuery," which is a firebase extension.One ...
In Looker can I rename the "Totals" row that results from checkbox
See screenshot below. When I check off the Totals box, it produces a new row that sums up the values of the co...
How to check the data present in a Pub/Sub topic.
How can check whether a message is present in a Pub/Sub topic without using a subscriber or listener?
Is there a way to label multiple Bigquery tables at the same time using SQL?
As tables within a Dataset do not inherit the Dataset labels, I am looking for ways to label multiple tables.I...
Avro library in dataproc
I am using a Dataproc cluster with hadoop 3.2 and spark 3.1. I have a python code to read avro files from GCS....
Unable to connect Cloud Data Fusion to Kafka on GCE
Hello Everyone,I am reaching out here for guidance as am unable to create a successful connection between the ...
No pipelines needed. Stream data with Pub/Sub direct to BigQuery
Great feature for #BigQuery Community, kudos #GoogleCloudCommunity.https://cloud.google.com/blog/products/data...
Cloud Data Fusion: Multiple MongoDB collections to BigQuery
This is a repost of something I wrote on Stack Overflow a few days ago, but without any takers. Guess there's ...
AWS SNS equivalent in GCP
I have my data pipelines on GCP dataproc and want to implement notification in it. Is there any AWS SNS equiva...
Logs Explorer to Big Query
I have an API hosted on GCP, I am wondering if there is a way to do a one time dump of Logs Explorer data into...
How to compare current row values with next row values in BigQuery Stored Procedure
Hi,In Big Query stored procedure, I need to compare the current row values with next row values. How can I ach...
How to increase limit of Storage Write API?
This page:https://cloud.google.com/bigquery/quotas#write-api-limitslists a 10MB limit on Storage Write API req...
How to get job_id inside a pyspark job running on dataproc cluster ?
Is there any provision or any way to get the job_id of the pyspark job inside the pyspark code while running o...
Data Fusion - Preventing Duplicated from sinking to Bigquery & Denormalise
Please assist? I would like to prevent duplicates entering into my Google Bigquery db. I have used the Distinc...
Bigquery premature EOF
I have a table in bigquery with around 32 million rows and 85 columns with size of 22 GB. (It is a physical ta...
BigQuery Console UI un-nesting all structs
Something changed in the way results are displayed in the UI. All structs and arrays are automatically un-nest...
Big query alter table add column after a specific column
Is it possible to alter table and add a column after a specific column in Big query ? something like Alter tab...
transfer a file from a container to another
I'm trying to find the best way to communicate between 2 containers (cont1 and cont2) whenever a file is uploa...
Extract Telemetry Message Content and Respond With IOTCore Command to Device
Hello,I'm new to GCloud and am looking for a way to extract telemetry messages sent by devices to IOTCore, and...
com.google.cloud.bigquery.BigQueryException: Error getting access token for service account: Error w
We've run into a very weird issue while using the bigquery java client library. The method com.google.cloud.bi...
⭐️ Google Cloud Data Heroes Series: Meet Tomi, a data engineer based in Germany 🇩🇪
Hi Google Cloud Data Analytics Community! I'm Grace, your community manager flying in with the fourth Google C...
How do you orchestrate and schedule Dataproc serverless batch jobs ?
I have multiple pyspark jobs with dependencies. I want to orchestrate and schedule it. How it is possible to d...
Google Ad Manager: Big Query
How to transfer data transfer files to database? I understand that data transfer files from Google Ad Manager ...
Bigquery automated store proc for ddl creation
Hi,Do we have store proc to create complete ddl for bigquery
👉 Don't miss the Data Engineer Spotlight on July 20!
Join the Data Engineer Spotlight to connect with your fellow data engineers, learn best practices of Google Cl...
GCP Datastream new source connections
Hi guys , I need to get CDC from databases which are of (Mysql , Mongodb , Postgres) type and store it in GCS ...
I can not run any query on BigQuery anymore (transient error n°4233314)
I can no longer query nor preview any table nor run any job in my BigQuery (region europe-west2). When I run a...
Wrangler - Data Fusion Exception in task 0.0 in stage 0.0 (TID 0)
Please assist with this error in Wrangler. See log: https://drive.google.com/file/d/1BAVlLmeaHBZuUVr7_i3nA1iVk...
Pub/Sub ordering guarantees
I am going to use Pub/Sub platform for rtdn processing. I am a bit confused of some guarantees (if they exist,...
Google Data Catalog issue with permissions(Maybe a bug in GCP?)
I am using the Data Catalog and I need be enable to change the resource entry overview, like this overview in ...
Articles from Product Experts and Googlers
BigQuery Data Preparation with AI Agents: Smarter, faster, and code-free
Ready to unlock the secrets hidden in your product data? This tutorial demonstrates how BigQuery Data Preparation can clean and transform your raw data into actionable business intelligence, using a realistic example from the Fashion and Beauty industry.
Launched: ABAP SDK v1.10 unleashing powerful AI/ML, eventing, and storage capabilities
Supercharge your SAP systems! ABAP SDK v1.10 unlocks BigQuery AI/ML, simplified event-driven integration, Cloud Storage content repositories & access to 290+ Google Cloud APIs. See what's new.
From idea to viral video: the GenAI workflow you need
What if you could create professional-quality videos with just a fraction of the effort? This blog introduces a radical new GenAI workflow on Vertex AI, using models like Gemini, Imagen3, MusicLM and Veo2 etc to make video creation accessible to everyone.
