Data at Google Cloud Next '24!
Answer this question in the comments to earn a special badge: In your journey, what's one overlooked data skil...
Forum Posts
Influence Looker’s Roadmap and join the BI Customer Council (BICC)
Would you like to influence Google Cloud’s BI roadmap more? I’d like to invite you to join the Google Cloud BI...
Using a Shared Subscription with Multiple Pull Subscribers in Google Cloud Pub/Sub
Create One publisher and Two Subscriber using the same one Subscription ( Pull ). Pull the Message using the f...
Pushing CDC data from pubsub to BQ
Hi All,I would like to understand what are the ways by which I can handle the CDC data present in pubsub & pus...
cloud pub sub
Can anyone tell following logic is possible or not:I have one publisher and one subscriptions. I've sent 10 me...
How to make avro schema fields optional ?
Already tried below schema.{ "type": "record", "name": "MyRecord", "fields": [ { "name": "requiredField", "typ...
i can't change any organization policies.
I am trying to change the organization policy domain restricted sharing. But unfortunately I get an error ever...
Deduplication if subscriber application is not idempotent and we can't change it.
If Subscriber application is legacy and not idempotent, Can we handle deduplication (in case of redelivery, ac...
Ensuring Sequential Delivery with Multiple Ordering Keys in Pub/Sub
I'm working on a Pub/Sub system where I need to ensure that messages are received in the exact sequence they a...
Read Google Workspace log in Google cloud
Hi all,I need to get Google workspace events in Google Cloud so that with api I can see them for analytics sco...
Android management api set pub/sub for notification
Hi Team,I've been utilizing the Android Management API for managing notifications in my app, employing the pub...
Recommendations for the pull mechanism of Google pub/sub notifications
I am using EMM (Android Management) in my project. For each new enterprise, I am creating a new topic and a su...
Real time streaming of analytics table in BQ to Pub/Sub
Hi, Am currently working on a POC. We are leveraging Google analytics for our website and capturing the traffi...
Issue in Implement Datastream and Dataflow for analytics
I was adding a new source in Datastream whose base db is in Postgres I’ve setup the pipeline where the destina...
Webhook for Google Reviews
Hello, I'm currently working on setting up a webhook or some form of notification within my Google Business Ac...
GCP Integration with Dynatrace
I would like to integrate GCP with Dynatrace monitoring service to analyze the logs and metrics from gcp. For ...
UDF error: java.lang.NoSuchMethodException: No such function
Hi,I'm using dataflow, and I've added UDF which I uploaded in Cloud Storage bucket. The compute service accoun...
Relay messages from Java Messaging Service (JMS) to Pub/Sub
Hi, I have a JMS topic and would like to relay messages from that topic to Pub/Sub. I see a Dataflow template ...
How to stream MongoDB cdc changes to BigQuery using Dataflow?
Hi folks so I have a usecase where I need a realtime pipeline to push mongodb cdc data to big query for analyt...
how to pull all unacknowledged messages from pull subscription process and ack in batch
I Have a requirement to pull all the messages from the pull subscription and process ( write to GCS file and w...
BigQuery Subscription with proto3: Wrong writing of null values to BigQuery table
Hi all,It appears that the BigQuery Subscription writes NULL values to the BigQuery table when the proto3 defa...
PubSub: Consuming message fron Subscriber Acknowledged Message using message id
Team,I consumed message from Pubsub using my subscriber. Can i re consume same message using message id though...
Subscribers don't reconnect to the subscription when there is no traffic
I am using StreamingPull mechanism to receive messages from the Pub/Sub.Here is only 2 overriding for default ...
Is it possible to implement multi region on pub sub?
Hi Everyone,I am new in GCP pub-sub exploring and analyzing few real time problem scenario. As per my knowledg...
Pubsub Emulator UI
Hello everyone, I just wanted to share a project that I was working on that will help people who want to use t...
Pub/Sub to GCS/BigQuery
Hi ,i am having a scenario like only when Data quality check is success i want to persist the data from pub/su...
Pushing data from BQ to Pubsub in Real-time
Hi,I would like understand the best way to capturing CDC data from bq to publish into Pubsub. Primarily, I wan...
autoscale cloud function based on no. of messages in pubsub
is there a way to autoscale cloud function based on number of messages in pubsub. My understanding of scaling ...
Google pubsub push subscription ack
I have a cloud run endpoint that gets triggered by a push type pubsub subscription and it recives the user spe...
My subscriber receives a message from GCP subscription with exactly 60 seconds delay sometimes
I have an issue which happends periodically in my system, but causes major problems.We use GCP Pub/Sub and som...
Spark Streaming with PubSub Lite
I want to create a Spark Streaming application in DataProc that has a subscription to PubSub Lite or PubSub (I...
google busiiness pub sub
Hello I'm trying to be notified when a new review is added on my Google Business Profile or updated profilleA...