I Have a requirement to pull all the messages from the pull subscription and process ( write to GCS file and write to BQ) and acknowledge these messages.
I want to run a cloud run job that is scheduled to run every 15 min. So for every 15 min i need to get all the unack messages from pull subscription in a single pull request process it and acknowledge.
There are 2 options.
1) Using synchronous pull : With This we cannot guarantee that all unack messages will be pulled in single pull request. Also it has a limitation that max_messages is 1000.
2) Using streaming pull API pull all the messages create a list , process these message in batch and acknowledge in batches.
Above 2 approaches cannot guarantee that all the unack messages in subscription will be processed.
How can we know how many unack messages are available (ex: 15 min)?
LinkedIn
Twitter
