Starting yesterday (Saturday May 17) all prediction jobs are failing and this is the error:
<<<
Batch prediction job test1prediction encountered the following errors:
System error. Please try this operation again. If the issue persists please visit https://cloud.google.com/support-hub to view your support options. Google Cloud Dataproc Agent reports job failure. If logs are available, they can be found at: https://console.cloud.google.com/dataproc/batches/us-south1/llm-batchprediction-spark-job-3091061712... gcloud dataproc batches wait 'llm-batchprediction-spark-job-3091061712091086848-0' --region 'us-south1' --project 'ma156d6cdfeeb30ca-tp' https://console.cloud.google.com/storage/browser/dataproc-staging-us-south1-683156645640-o20l4air/go... gs://dataproc-staging-us-south1-683156645640-o20l4air/google-cloud-dataproc-metainfo/17417650-250b-46be-bf0d-704eca6834b6/jobs/srvls-batch-110e0abd-e97a-4830-b236-4e49f358a5bb/driveroutput.*
>>>
Even when the batch has only one request it will fail.
As a test I copied a request from a previous batch that succeeded and created a new batch file using that request and it failed also.
Lastly, I manually created a batch job and it failed also.
I know I'm not exceeding any quotas and my billing is in good shape.
I'm desperate to process batches as I have an important deliverable first thing Monday morning. Can anyone lend some help or information on this/
Thank you.
LinkedIn
Twitter
