vertex ai | custom training job | cancel creates duplicate job

[Toobs4]

Hi. I have set up a custom training job on Vertex AI. Each time I cancel it, Vertex AI stops the job but creates a new job shortly after. Similarly, after the job is stopped and I delete the job, Vertex AI creates another new job. Has anyone experienced this behavior? Do you have any solution?

[cassandramae]

Hi @Toobs4,

Welcome to Google Cloud Community!

If you're stopping and cancelling the job using the console, could you try the following methods and see if the issue persists? This could help us isolate if the behavior happens only in the console.

• Cancel a custom job
• Delete a custom job

In addition, it's possible that some configuration within your pipeline might include a retraining policy that automatically restarts jobs under certain conditions which you may check on your set up.

If the issue persists, I suggest to reach out to Google Cloud Support and see if this could be a potential bug or project-specific issue only.

Was this helpful? If so, please accept this answer as “Solution”. If you need additional assistance, reply here within 2 business days and I’ll be happy to help.