Get hands-on experience with 20+ free Google Cloud products and $300 in free credit for new customers.

Offset management in PubSub Lite spark connector

I was following  the example https://cloud.google.com/pubsub/lite/docs/samples/pubsublite-spark-streaming-from-pubsublite

But I have a question, In my test when query.stop() is called, it just interrupt the pipelime immediately. My question is how do I make sure everything I have ingested are committed in subscription so when next time I start the job it start from where I have processed?

0 7 725
7 REPLIES 7