Get hands-on experience with 20+ free Google Cloud products and $300 in free credit for new customers.

Cloud Run Job failing with no logs 1 of every 5ish times

We have a cloud run job that has been running successfully for about a year. In the last few days, when we go to run the job maybe 1/5 times it fails. If I execute it from a local terminal using gcloud, it's about the same. Try verbosity and don't see anything interesting in the back and forth from gcloud.

The Task column says "Failed with Errors". There are no logs anywhere. Nothing shows up in the error reporting.

When the job runs, I see everything I'm expecting, including logs.

I've made it so the first thing on startup I have a log so I should see something - so it seems like a GCP thing?

I've upped the memory of the job, just in case, but no help there.

I'm going to try moving it to a new region (currently in us-west-2) as that has helped when some things went off like cloud scheduler in the past.

 

2 1 477
1 REPLY 1

And now it is magically fixed (no errors in 20+ runs).

So there ya go...