Influence Looker’s Roadmap and join the BI Customer Council (BICC)
Would you like to influence Google Cloud’s BI roadmap more? I’d like to invite you to join the Google Cloud BI...
Data Analytics
Want more from your data? Have questions about products like BigQuery, Dataflow, or Pub/Sub? You're in the right place. Join conversations around data warehouse modernization, streaming analytics, business intelligence, and more.
Focused on Looker? Engage with the Looker Community here.
Focused on Looker? Engage with the Looker Community here.
Forum Posts
workflow executed successfully, but data is not inserted into destination table.
My work flow was working fine, but yesterday i update the corn timing.Though log and result shows data inserte...
Invoke Dataform single sqlx using Cloud workflows
Hi All,I'm invoking Dataform using Cloud workflows. I can invoke single sqlx file using tags(includedTags). Bu...
Log Explorer - Improper Log Parsing
I have a Flink job running in a dataproc cluster. It's exporting logs to Log Explorer but I don't think Log Ex...
Writing streaming data to BigQuery via I/O connector using a Dataflow job
I am using the `STORAGE_WRITE_API` method using BQ I/O connector to write data to BQ from a DataFlow job. I am...
Using BigQueryTemplate for streaming ongoing inserts
From a java application (spring) i need to stream (insert) ongoing data (1TB per day).Can i use for this BigQu...
Cloud Functions vs AWS Lambda
Does Cloud Functions Supports bot layer alike AWS Lambda Functions. If not what is a workaround if we still wa...
Load 130.000 csv files to BigQuery. Limitation loading jobs
Hello, I have 130.000 csv files in GCS. I am uploading them via bigquery.Client() . But the problem is that ap...
How to track app download by using utm tags link?
Hi there, Hope you will be able to help me. How to i track app download by using UTM tag link with Android pla...
Dataproc Alert policy
custom/cpu/utilizationcustom/disk/percent_usedhow are the above cloud dataproc batch metric useful when settin...
cloud pub sub
Can anyone tell following logic is possible or not:I have one publisher and one subscriptions. I've sent 10 me...
Dataproc Alert
I need help regarding creating Dataproc alerts.What all type of alert we can create for Dataproc.
Using a Shared Subscription with Multiple Pull Subscribers in Google Cloud Pub/Sub
Create One publisher and Two Subscriber using the same one Subscription ( Pull ). Pull the Message using the f...
Execution order of Dataform workflows with the same tag name.
Hello Community,I have the following situation in my project: More than one Dataform workflows under the same ...
Error while creating Dataflow Custom Templates
Hi Google,I have a requirement to read data from BigGuery and write data to the gcs bucket for that I created ...
SQL Spark connection in juptyerlab
Hi,I have a Dataproc cluster and using the web interface (Jupyterlab) inside the cluster, I am trying to read ...
POST API data to Big Query
I am trying to figure out what is the easiest way to connect our CRM's REST API. The CRM we use in Inline Data...
Concurrent UPDATES or INSERTS to a BQ table
Hello Community,I am building a validation process for checking whether a .csv file was successfully ingested ...
Looker Instance Went Down
We have a legacy looker instance that doesn't appear to be responding, and we don't see any information from g...
Google Analytics 4 (standard) overcome data retention period
Google Analytics 4 (standard) has only a 14-month data retention period we need to extend it quite long and ne...
Airflow DAG: Web scrapping using selenium
In my pipeline, I have a step wherein I am doing some web scrapping with selenium. That is: using selenium to ...
Pushing CDC data from pubsub to BQ
Hi All,I would like to understand what are the ways by which I can handle the CDC data present in pubsub & pus...
Are the BigQuery API calls like getting table metadata free?
I'm struggling to find information on how BigQuery API calls are charged. Is calling tables.get with STORAGE_S...
Unable to Execute Dataform Workflow due to "IAM permission denied for service account"
HelloI've been facing an issue trying to execute a workflow in Dataform. Every time I attempt to run it, I'm m...
Dataplex > Explore > Spark SQL: Script Failed in Execution
Has anyone been getting this error in Dataplex > Explore > Spark SQL?___Script failed in execution.org.apache....
Why are certain BigQuery jobs not allocated to the reservation?
I've established a reservation specifically for query jobs. However, why are certain BigQuery jobs not being a...
TABLE_STORAGE_TIMELINE view details
Hey, We can't find any information about this view (TABLE_STORAGE_TIMELINE), anyone can give us some more deta...
I can't see the whole exported rows from BigQuery to the Google Sheets, I have almost 12k rows ?
I can't see the whole exported rows from BigQuery to the Google Sheets, I have almost 12k rows .I only see 31 ...
Dataform and BI Engine
It seems that queries run from Dataform are not able to utilize the BI Engine.Is there a way to use the BI Eng...
Physical billing related: Time Travel bytes question
I'm analyzing my BQ project to see if switching to Physical Bytes Pricing model will be beneficial and I ran t...
Dataform Development
Hi All,I am trying to set up my teams' infrastructure in BQ and currently, we are looking to have 2 projects 1...
Articles from Product Experts and Googlers
Untangle the API maze with Apigee API Hub
Mirror mirror on the wall, What do I do with my API sprawl, Where can I see them all, How to make sure they listen to my call, And to ensure their compliance doesn’t fall. API Hub is the one to rule them all!
Next-Gen Data Pipelines: Airflow 3 Arrives on Google Cloud Composer
We are thrilled to announce that Google is the first hyperscaler to provide selected customers with access to Apache Airflow 3 on our fully managed Cloud Composer 3 service. Discover how powerful new features like DAG Versioning, a modern React-based UI, and scheduler-managed backfills can help you innovate faster and more reliably.
Configuring GKE rollout sequencing
You can manage the order of automatic cluster upgrades across Google Kubernetes Engine (GKE) clusters in multiple environments using rollout sequencing. For example, you can qualify a new version in pre-production clusters before upgrading production clusters.
