New Version of Document AI fails Internal Error

[Securis]

New Versions fail and have been for several days.

[Sitron]

Same here, training a new model fails most of the times since 2024-09-06 for me. It occasionally works and manages to train models for a while (can sometimes work ok for a few hours at a time). Models stop training with the error "Internal error encountered.", code 13, after about 15 minutes (approximately, sometimes longer: e.g. I had one today that failed after 45 minutes). I'm training Custom Extractor models, in EU.

EDIT: I've now finally managed to train a new model, the last successful one was trained a week ago. However, the very next attempt at training a model (another processor) failed. It's weird that it randomly works and that the solution, from my testing, has been to just try many times.

[Securis]

No wonder Google is losing the AI arms race.  This is my first time using Google Cloud for anything, and I'm not impressed.

[Securis]

Found this and thought it might be helpful - https://issuetracker.google.com/issues/353425610?pli=1

[steven_tan10]

Hello, I'm having a similar problem. Is there any enlightenment from google?

[Securis]

No Nothing.  As we say in America...Radio Silence  

[jaia]

Hello,

Thank you for reaching out to us with your request.

We have duly noted your feedback and will thoroughly validate it. While we cannot provide an estimated time of implementation or guarantee the fulfillment of the issue, please be assured that your input is highly valued. Your feedback enables us to enhance our products and services.

Regards,
Jai Ade

[Securis]

@jaia This isn't feedback. Your product is broken. You want us to purchase something, and it's broken, and you can't or won't give us any response other than something provided by another AI.  

[AndrewB]

@Securis I want to apologize for the frustration this issue is causing. I'm escalating this issue to our engineering team to get more information about what's causing the "Internal error encountered" message and error code 13, and to understand what's being done to resolve it.

I'll keep this thread updated with any information I receive from engineering.

[Securis]

https://issuetracker.google.com/issues/353425610?pli=1

[Securis]

@AndrewB How do we get an update on this?

[AndrewB]

Hi everyone,

Just wanted to let you know we're making progress on this issue. Thanks to @Securis's help, our engineers have identified the cause and are working on a solution. We understand this bug is impacting many of you, and we're committed to resolving it as quickly as possible. We expect to have a fix deployed by the end of next week and will keep this thread updated.

[AndrewB]

Hi everyone - I'm happy to report that a fix has been deployed and should be live for all users now. We encourage you to test it out and let us know if you encounter any further problems.

[shaikhsharmeen4]

If you're encountering an "Internal Error" in Document AI's new version, it could be due to various factors such as incorrect configurations, insufficient resources, or internal service issues. Try the following:

1. Check Logs: Use Google Cloud's Cloud Logging to view detailed logs and identify the root cause.
2. Verify Inputs: Ensure your training/testing data is correctly formatted and meets minimum requirements.
3. Retry: Sometimes internal errors resolve after retrying the job.
4. Contact Support: If issues persist, contact Google Cloud support for assistance.

[Sidharth05]

I still encounter the same issue while training my model. Have been retrying it for the past 2 days.

{

"name": "projects/435734813656/locations/us/operations/13343185919240393728",

"done": true,

"result": "error",

"response": {},

"metadata": {

"@type": "type.googleapis.com/google.cloud.documentai.uiv1beta3.TrainProcessorVersionMetadata",

"commonMetadata": {

"state": "FAILED",

"createTime": "2024-10-07T06:44:19.501896Z",

"updateTime": "2024-10-07T06:52:24.959021Z",

"resource": "projects/435734813656/locations/us/processors/d5d53b3436bf86cc/processorVersions/b92c84e5f85153d9"

},

"trainingDatasetValidation": {},

"testDatasetValidation": {}

},

"error": {

"code": 13,

"message": "Internal error encountered.",

"details": []

}

}

[akshayAttri]

@AndrewB We are also facing similar issue since yesterday 11-10-2024. 
{
  "error": {
    "code": 13,
    "message": "Internal error encountered.",
    "details": []
  }
}