I got error when training the model

[minhkimnhut]

I got error "Training pipeline failed with error message: Internal error occurred. Please retry in a few minutes. If you still experience errors, contact Vertex AI " when training the model

Detail:

Status Failed

Training pipeline ID 7341302730499031040

Created Mar 25, 2024, 3:34:09 PM

Start time Mar 25, 2024, 3:34:09 PM

Budget (original) 8 node hours

Elapsed time 8 hr 13 min

Regionus-central1

Encryption type Google-managed

Dataset chest_xray_unbalance

Annotation set chest_xray_unbalance_icn

Data split Randomly assigned (80/10/10)

Objective Image classification (Single-label)

Can anyone help me to explain the issue?

[d4np4n]

I follow the topic. Same issue (I am following Udacity as well). More than 8h and after I get same issue, this against the 1hour forecasted.

[nceniza]

Can you try running your pipeline in a different region ?  Internal Error message is vague to specify the error. If this persist I would suggest contacting support for better investigation of your project usage and/or Quota Limits. - https://cloud.google.com/contact

[minhkimnhut]

I tried to train on same region. Fortunately, it was complete.