Data at Google Cloud Next '24!
Answer this question in the comments to earn a special badge: In your journey, what's one overlooked data skil...
Forum Posts
Influence Looker’s Roadmap and join the BI Customer Council (BICC)
Would you like to influence Google Cloud’s BI roadmap more? I’d like to invite you to join the Google Cloud BI...
BigQuery returning `There is no data to display.`
I've setup a Datastream CDC into BigQuery stream. The stream started without issues and is currently showing t...
Issue in Implement Datastream and Dataflow for analytics
I was adding a new source in Datastream whose base db is in Postgres I’ve setup the pipeline where the destina...
Datastream_to_bigquery dataflow template : Invalid date problem
I am using Datastream_to_bigquery dataflow template to ingest data from GCS to Bigquery. The avro files in GCS...
DataStream connection to Google CloudSQL timing out with no error
I am attempting to set up a DataStream from a CloudSQL to CloudStorage. I enter the network and authentication...
Datastream Issues
We are encountering a problem with the synchronization of data from MySQL to BigQuery through Datastream. An e...
Schema not updating in Datastream destination table
I have a Datastream set up that is streaming several tables from one of our cloud SQL Postgres databases into ...
Problem with partitioned table in BigQuery and streaming buffer from Datastream
Hello,I have created a datastream from MySql to BigQuery, a made table in Bigquery partitioned.It looks like i...
Not matching results for total revenue and manual content
Hey guys,Currently, I'm working on filtering campaign results based on the content within the UTM parameters. ...
Does Datastream support SqlServer as a source?
I found some documentation for setting up SqlServer as a source in Datastream but I don't see it as an option ...
Forward-SSH Tunnel Datastream Option Trouble
I have chosen the “Forward-SSH Tunnel” option for my Datastream connection profile after reading up on the doc...
GCP Datastream doesn't replicate new tables from RDS Postgres
Hello,I have a datastream that replicate RDS postgres to Bigquery dataset.Everything is set correctly, the dat...
Spark Streaming with PubSub Lite
I want to create a Spark Streaming application in DataProc that has a subscription to PubSub Lite or PubSub (I...
BigQuery Data Transfer Service for Google Ads - table ads_PaidOrganicStats
Hello Community,We have connected our Google Ads Account with BigQuery via Google Ads API https://console.clou...
Datastream got updated and now timing out
We have three environments using Datastream. On Sunday they all got "updated" and started failing immediately ...
Duplicate records in bigquery not able to delete.
1. Our source is Oracle and the Destination is a big query we are using Datastream to migrate the data. In the...
How many MySQL connections does datastream consume?
Good day allI am in the process of backfilling and streaming a large number of tables (>1000) using datastream...
How do you manage schema changes for BigLake tables?
Hi everyone!I am streaming data from a source DB to cloud storage (.avro format) using datastream, and then re...
Data ingestion and transformation and storage
I want to execute a cloud function (say transformation) which gets triggered on gcp file upload/create. It doe...
Using cloud SQL (postgres) server as SSH host to connect from Datastream
Hi! I am trying to connect to my Cloud SQL PostgreSQL database from Datastream (to get data to BigQuery). I am...
Google Cloud's Datastream: Not replicating deletes to BigQuery
I have a stream setup with Datastream where a PostgreSQL database on Cloud SQL replicates to BigQuery. When da...
Best way to batch load / historic backfill bigQuery table?
We've got a mysql server running at another cloud provider, and the plan is to use datastream to pull the near...
Not matching results from GA4: campaign_name and purchase_revenue in BQ
Hello there! So lately I've been trying to pull out the data from campaign_name and purchase_revenue_value in ...