Get hands-on experience with 20+ free Google Cloud products and $300 in free credit for new customers.

Dataflow HttpError 429 when requesting to trigger a dataflow template

Hi all,

   We created a dataflow template and running dataflow pipelines from it since a month or two. Today we received a weird error related to it which is "HttpError 429 when requesting <url> returned Internal service error: Failed to configure shuffle service for job <job_id>". But the next time we ran it triggered the pipeline and ran fine. I was going through the source of this error but unfortunately i couldn't find any. Can some one help me know what is this about.

 

  Job details : It is a batch dataflow pipeline. Source BQ query, Sink Pub/Sub.

 

Thank you in advance

0 1 248
1 REPLY 1