AbortedException in Empty Commits

[lingling-134]

Hi! I'm currently developing a Java program using Google Cloud Spanner, utilizing batch inserts within manual transactions via a multithreaded approach. Each thread in the application is responsible for generating and accumulating its own mutation array. As part of the process, we add an insert mutation to the local array and perform empty commits. However, I've encountered an issue where AbortedException arises during these empty commits, and not specifically when the batch size is met for a commit to the database. The transactions are handled using TransactionManager and TransactionContext. I am looking for insights into why this exception occurs specifically at the point of empty commits. Thank you so much!

[robertcarlos]

Hi @lingling-134,

Welcome to Google Cloud Community!

Before we could proceed, I would like to ask a few questions:

• Can you provide a sample code/snippet that we could try?
• Can you provide a specific error message that you receive?
• Can you also provide some screenshots that could be helpful in this case?

Hoping to hear from you soon.

[lingling-134]

Hi @robertcarlos ,

Thank you for the response. I have created a repo for a simpler version of my application, would you mind checking it out in the repo? I have attached a detailed README file, which explains the context in the application. Thank you!

Basically, the application utilizes a batch processing approach in the SpannerClient class for handling database insert operations. In this process, each thread creates mutations for insert operations, which are stored in a bufferedMutations ArrayList<Mutation>. These mutations are collected until they reach a set batch size. Upon reaching this limit, the batch is sent to the Google Cloud Spanner database in a single transaction.

Some basic errors occur are (more detail printed out when applciation is run locally) two types:

---------------------------------------------------------------------------------------------------------------
Thread-138 in SpannerClient.commit() method- AbortedException occurred.
Current size of mutation buffers array: 94.
Stack Trace:com.google.cloud.spanner.AbortedException: ABORTED: io.grpc.StatusRuntimeException: ABORTED: Database schema has changed



---------------------------------------------------------------------------------------------------------------
Thread-0 in SpannerClient.commit() method- AbortedException occurred.
Current size of mutation buffers array: 77.
Stack Trace:com.google.cloud.spanner.AbortedException: ABORTED: io.grpc.StatusRuntimeException: ABORTED: Transaction was aborted.
retry_delay {
  nanos: 38655229
}

Thank you very much!

[robertcarlos]

Hi @lingling-134,

Thank you for your response. Unfortunately I wasn't able to replicate your issue. I would suggest filing this one as a bug through this Github issue link or Google Cloud issue tracker. I would also suggest checking out this troubleshooting documentation as this may contain solutions/workarounds.

I would also suggest reaching out to Google Cloud support hub for additional support.

Hope this helps.

[lingling-134]

Hi @robertcarlos ,

Thank you for the response. May I ask under what circumstances we might encouter the 'ABORTED: io.grpc.StatusRuntimeException: ABORTED: Database schema has changed' excpetion while we are not doing any modifications to the database schema at all? Also, I suppose the retry_delay error occurs because there is tension caused by the multi threading environment? Thank you so much!

[robertcarlos]

Hi @lingling-134,

For ABORTED error messages, this is usually encountered due to concurrency issues and that there are conflicts within requests based on this documentation Cloud Spanner error codes. It was also reported previously as per this Github issue link and an update was previously released to fix this issue. That is why I suggested filing a bug through the links I provided so that engineers could provide a fix for this issue.

Hope this address your concern.