This website uses Cookies. Click Accept to agree to our website's cookie use as described in our Privacy Policy. Click Preferences to customize your cookie settings.
Accept
Reject
Preferences
Log in to ask a question
Get hands-on experience with 20+ free Google Cloud products and $300 in free credit for new customers.
Log in to ask a question

Dataflow requires minuemiun 4 minutes time for resources provision for processing small data jobs

Posted on 02-04-2024 03:57 PM
askreddii
Bronze 3
Bronze 3
Reply posted on --/--/---- --:-- AM

Hi,

When we submit a new job in Dataflow, the service automatically allocates the necessary infrastructure, including worker instances, to execute the specific job for my project production runs. but we are facing issues with the dataflow provision time. it currently takes 4 minutes for the resource provision and 1 minute for the processing job. Not sure, how can we reduce the dataflow resource allocation time here ? I'm expecting the data processing job and worker nodes provision should be completed in 1 minute time. would it be possible in Google Dataflow?? thanks

 

Regards,

Reddy

 

 

0 8 3,025
Topic Labels
0 Likes
8 REPLIES 8
Top Solution Authors
View all