Influence Looker’s Roadmap and join the BI Customer Council (BICC)
Would you like to influence Google Cloud’s BI roadmap more? I’d like to invite you to join the Google Cloud BI...
Data Analytics
Want more from your data? Have questions about products like BigQuery, Dataflow, or Pub/Sub? You're in the right place. Join conversations around data warehouse modernization, streaming analytics, business intelligence, and more.
Focused on Looker? Engage with the Looker Community here.
Focused on Looker? Engage with the Looker Community here.
Forum Posts
Issue with properly setting up a Spark Session to my Apache Iceberg BigLake tables
Hello all,I've successfully set up an Iceberg table with a BigLake Metastore as per this documentation:https:/...
Inquiry about IP Addresses for BigQuery Data Transfer Service for S3
Hello,I am looking to use Google BigQuery Data Transfer Service to read files stored in AWS S3. However, our S...
Cloud Composer - run same task in parallel with different arguments
Hi @ms4446 and allI just wanted to ask if this is a good practice. I have an Airflow DAG being run in Cloud Co...
duplicated subnet
in google cloud console i tried to create an network endpoint group and when i try to choose the subnetwork i ...
DataFusion Backup
I need to know how to take backup of DataFusion Instanc.Can anyone help me.
Looker PDT Help - PDT's Persist, Regenerator Does Not Run
I'm looking for some help with some PDT's I've created.The PDT's do not clear themselves out even with persist...
Missing reporting dashboards
We are missing all of our reporting dashboards. They disappeared and they were not deleted. Appreciate any gui...
How to handle exceptions in Apache Beam (python), for reading from JDBC and writing to BigQuery
I was able to successfully read from JDBC source, and write the output back to BigQuery. However, I'm still st...
Time and Geographically clustered joins in BigQuery
Hi All,I am working with astronomical data, using Bigquery Geogpoints data to represent RA/Dec, and am testing...
PubSub to BQ - Dataflow Standard Templates to upsert the records
Would like to know whether we have standard pubsub to BQ dataflow template to insert the record if new and upd...
GCP Dataform data quality scripts failed to send notification
Is there a way to send notifications with error messages for each script inside the workflow like the one we u...
Emmanuel Katto Uganda : How can I start learning data analytics from scratch?
Hello everyone,I hope this message finds you well. My name is Emmanuel Katto from Uganda, and I'm based in Uga...
CSVs to dataframe to BigQuery - GCS best practice
HiConsider the code below. Here I am doing the following steps:1) Reading several dataframes from csvs in a bu...
Cloud Composer 3 pricing explanation
Cloud Composer 3 pricing is based on DCU (data compute unit), https://cloud.google.com/composer/pricing#compos...
GCP PubSub : Message Ack Latency has Increased with Java17
I'm running service on GAE to consume the message from GCP Pubsub with below configuration and it takes only ~...
How to observe pubsub push window changing with failing requests?
I have a cloud run endpoint that gets triggered by a push type pubsub subscription and the cloudrun api is a c...
Dataflow requires minimum 4 minutes time for resources provision for processing small data jobs
I have a requirement, where I will be getting a encrypted zip file in GCS bucket. I need to decrypt and unzip ...
Data Loss after sGTM Capi Integration
Hi everyone!We're facing with an issue on new user tracking. We have BigQuery integration and the situation is...
Hi guys I am getting this error runing dataflow with flex-template
Workflow failed. Causes: S30:Read TransferResultDetails from MySQL/Create.Values/Impulse+Read TransferResultDe...
Create operation on Cloud Composer 2 failed
Hello,I'm trying to create Composer2 environment but getting this error, see printscreen, I didn't find any an...
Seeking Guidance: Error Connecting to AWS S3 via BigQuery External Connections
I am currently working on a project that involves leveraging the BigQuery - External Connections feature to ac...
storing the pubsub received messages in firestore database
I want to store the message received from the pubsub topic via pull subscription in firestore database in fire...
Data Discrepancy - Search Ads 360 to BigQuery Data Transfer
Dear All,We are using the BigQuery Data Transfer product to collect data from the Search Ads 360 platform to G...
[BigQuery] Fetching results into a Go struct
I am trying to fetch a string result into a Go struct. I am following the instructions mentioned here under `E...
User does not have permission to access results of another user's job.
Hello,User A is creating a query in bq and then user B wants to read the data (using job ID and location). Use...
Adding Created & updated Timestamps columns in Data Fusion pipeline using wrangler
Hi GCP Community,I'm currently working with Data Fusion and have set up a pipeline to transfer data from SQL s...
Read data from hudi using trino
I need help to read data from hudi using trino in dataproc cluster.
Google Ads Datatransfer transfering way less products than show in the dashboard
Hello everyone,I have a bigquery google ads datatransfer, I'm only interested in the products table.Each day I...
Data Fusion Current Timestamp issue
Hi Google,I'm creating a data fusion pipeline where my source is GCS bucket and I'm processing a CSV file from...
Looker: data error when looker studio send scheduled report
I scheduled looker studio report since January, and it didn't have any issue until last month. But some reason...
Articles from Product Experts and Googlers
Introducing DeepSeek R1 Model-as-a-service on Vertex AI Model Garden
Deploying Deepseek-R1-0528, a 671B parameters model, typically necessitates at least 8x H200 GPUs for a single API request. Such extensive resources are not accessible to everyone.To make Deepseek R1 more accessible to developers, we're excited to announce that DeepSeek R1 is now available as a fully managed API on Vertex AI in Preview.
Untangle the API maze with Apigee API Hub
Mirror mirror on the wall, What do I do with my API sprawl, Where can I see them all, How to make sure they listen to my call, And to ensure their compliance doesn’t fall. API Hub is the one to rule them all!
Next-Gen Data Pipelines: Airflow 3 Arrives on Google Cloud Composer
We are thrilled to announce that Google is the first hyperscaler to provide selected customers with access to Apache Airflow 3 on our fully managed Cloud Composer 3 service. Discover how powerful new features like DAG Versioning, a modern React-based UI, and scheduler-managed backfills can help you innovate faster and more reliably.
