Influence Looker’s Roadmap and join the BI Customer Council (BICC)
Would you like to influence Google Cloud’s BI roadmap more? I’d like to invite you to join the Google Cloud BI...
Data Analytics
Want more from your data? Have questions about products like BigQuery, Dataflow, or Pub/Sub? You're in the right place. Join conversations around data warehouse modernization, streaming analytics, business intelligence, and more.
Focused on Looker? Engage with the Looker Community here.
Focused on Looker? Engage with the Looker Community here.
Forum Posts
Datastream_to_bigquery dataflow template : Invalid date problem
I am using Datastream_to_bigquery dataflow template to ingest data from GCS to Bigquery. The avro files in GCS...
Google Analytics Data Export to BigQuery resulting in null Value in User ID
Hello, we have an application that uses Firebase and Google Analytics and then exports the data to BigQuery. O...
Getting Bindings information on different level
I am creating application which collect role & member information for analysis purpose. I want this informatio...
CUD expiry alert
Hi Team , I need to send alerts about expiry of CUD(Committed Use Discounts) at before 30 days via email. Ther...
Remote repository couldn't be reached in GCP
Hi there, Hi there,Hope someone will be able to help me. I built simple pipleline in Dataform repo (connected ...
Dataform: incremental table: stored procedure name
HI,Dataform generates code with procedure name like: EXECUTE IMMEDIATE"""CREATE OR REPLACE PROCEDURE `??????.d...
Not able to write datafram to BigQuery using Dataflow
I have an excel file on GCS I'm trying to read in Dataflow them write it out to BigQuery.I was successfully ab...
Transfer csv file from BigQuery to GCS without loading it into memory
Consider the following code: #Initialize GCS client gcs_client = storage.Client() # Define the destination URI...
GCP Integration with Dynatrace
I would like to integrate GCP with Dynatrace monitoring service to analyze the logs and metrics from gcp. For ...
Firestore -> BigQuery
Hi Folks,My company has a mobile app with lots of data stored in Firestore collections. We'd like to run data ...
Equivalent of Insert Overwrite strategy in DataForm
Hello, i'm testing DataForm instead of DBT, and i'm missing a feature we use a lot. In DBT incremental mode we...
Stream data availability
Data is available for real-time analysis using GoogleSQL queries immediately after BigQuery successfully ackno...
Dataflow Provisioning time and optimization
We are using dataflow for batch workloads which are small in future roadmap we want to enable streaming worklo...
Composer pipeline to upload files from Google BigQuery to an S3 bucket
I have an s3 bucket (amazon) and I want to write a composer pipeline to upload csv files into the S3 bucket on...
Data Engineering vs Cloud Architect
Hi, I’m currently studying to become a data analyst for a telecoms provider in the UK. Just working on my PDP ...
BigQuery Temporary Tables - Storage & Billing
1) BigQuery Temporary tables are scoped within the session / multi statement query. Will its storage as a data...
Access to street number metadata in google maps?
Hi,When I'm viewing a location on google maps, I can see the street numbers overlaid on building locations. Do...
UDF error: java.lang.NoSuchMethodException: No such function
Hi,I'm using dataflow, and I've added UDF which I uploaded in Cloud Storage bucket. The compute service accoun...
Release config ignores custom branch
Not sure what I am doing wrong here. I have a `test` workspace that is pushed to `test` branch in Github. When...
BigQuery datasets containing detailed usage cost data
Hi,"For BigQuery datasets containing standard usage cost data or detailed usage cost data, if I configure the ...
DataStream connection to Google CloudSQL timing out with no error
I am attempting to set up a DataStream from a CloudSQL to CloudStorage. I enter the network and authentication...
The caller does not have permission Cloud Composer
Hello, so currently I want to create cloud composer in service project using shared VPC from host project.I al...
Why does the viewed BigQuery data change after a certain period of time?
The January data I viewed in February is different from the January data I viewed in March.The query is the sa...
"Include advertising identifiers for mobile app streams" is ticked but no identifiers in BigQuery
We've ticked "Include advertising identifiers for mobile app streams" while linking Firebase (GA4) to BigQuery...
Have a dynamic parameter as a JS function
Hello everybody,We are trying to start using Dataform to orquestate and versionize our BigQuery queries but in...
Unable to start de Dataflow job due to : [UNAVAILABLE] HTTP_503
We are having problems to start a new JOB using gcloud commands.After successful creation , the job hangs with...
Dataflow module not found error while using Jupyter Notebook
Team,I just tried to create dataflow job using jupyter notebook available under workbench on Dataflow. If i wa...
Ingest Data from GCS to Big Query. Error - google.api_core.exceptions.Forbidden: 403 POST
I am trying to ingest data from GCS to Big Query. I can do it when I run it locally but when I put my function...
FAILED_PRECONDITION: request failed: Failed to convert table schema to Avro record schema
Hello community,I'm facing below error while trying to read BQ table data using storage read apiFAILED_PRECOND...
How to prevent a Dataform workflow in case a pre defined condition is not met.
Hello Community,I have the following scenario:1. Ingest daily multiple .csv files into designated BQ tables (e...
Articles from Product Experts and Googlers
BigQuery Data Preparation with AI Agents: Smarter, faster, and code-free
Ready to unlock the secrets hidden in your product data? This tutorial demonstrates how BigQuery Data Preparation can clean and transform your raw data into actionable business intelligence, using a realistic example from the Fashion and Beauty industry.
Launched: ABAP SDK v1.10 unleashing powerful AI/ML, eventing, and storage capabilities
Supercharge your SAP systems! ABAP SDK v1.10 unlocks BigQuery AI/ML, simplified event-driven integration, Cloud Storage content repositories & access to 290+ Google Cloud APIs. See what's new.
From idea to viral video: the GenAI workflow you need
What if you could create professional-quality videos with just a fraction of the effort? This blog introduces a radical new GenAI workflow on Vertex AI, using models like Gemini, Imagen3, MusicLM and Veo2 etc to make video creation accessible to everyone.
