Solved: Error integrating Secret manager with Dataflow - Page 2

Pravish · 09-07-2023 07:33 AM

Apache Beam v.2.48
Secret Manager API v.2.22
Here's a simple code to fetch secrets from Secret Manager. When running with this on Dataflow, I run into an error (see below)

public class GCPSecretManagerClient implements Serializable {

    private static final Logger LOG = LoggerFactory.getLogger(GCPSecretManagerClient.class);

    private ProjectName projectName;
    private SecretManagerServiceClient secretManagerServiceClient;
    public GCPSecretManagerClient(String secretsProjectName) throws IOException {
        this.projectName = ProjectName.of(secretsProjectName);
        secretManagerServiceClient = SecretManagerServiceClient.create();
    }

    public String getSecret(String secretName) {

        GetSecretRequest request =
                GetSecretRequest.newBuilder()
                        .setName(SecretName.of(projectName.getProject(), secretName).toString())
                        .build();
        Secret response = secretManagerServiceClient.getSecret(request);
        return response.toString();

    }

    public void closeClient(){
        secretManagerServiceClient.close();
    }

}

Error while creating secret manager client -

exec.go:66","message":"java.lang.IllegalAccessError: class com.google.iam.v1.TestIamPermissionsRequest tried to access method 'com.google.protobuf.LazyStringArrayList com.google.protobuf.LazyStringArrayList.emptyList()' (com.google.iam.v1.TestIamPermissionsRequest and com.google.protobuf.LazyStringArrayList are in unnamed module of loader 'app')
exec.go:66","message":"\tat com.google.iam.v1.TestIamPermissionsRequest.\u003cinit\u003e(TestIamPermissionsRequest.java:127)
exec.go:66","message":"\tat com.google.iam.v1.TestIamPermissionsRequest.\u003cclinit\u003e(TestIamPermissionsRequest.java:918)
exec.go:66","message":"\tat com.google.cloud.secretmanager.v1.stub.GrpcSecretManagerServiceStub.\u003cclinit\u003e(GrpcSecretManagerServiceStub.java:211)
exec.go:66","message":"\tat com.google.cloud.secretmanager.v1.stub.SecretManagerServiceStubSettings.createStub(SecretManagerServiceStubSettings.java:349)
exec.go:66","message":"\tat com.google.cloud.secretmanager.v1.SecretManagerServiceClient.\u003cinit\u003e(SecretManagerServiceClient.java:180)
exec.go:66","message":"\tat com.google.cloud.secretmanager.v1.SecretManagerServiceClient.create(SecretManagerServiceClient.java:162)
exec.go:66","message":"\tat com.google.cloud.secretmanager.v1.SecretManagerServiceClient.create(SecretManagerServiceClient.java:153)

ms4446

The error message you are seeing indicates that there is a problem with the Java class com.google.iam.v1.TestIamPermissionsRequest. This class is from the IAM library, but the error message does not necessarily mean that there is a problem with the IAM API itself. The error is more likely due to a Java class/method access issue caused by library conflicts.

Here are some things you can check to troubleshoot this issue:

Make sure that all of your dependencies are using the same version of the IAM library.
Make sure that there are not multiple versions of the IAM library in your classpath.
Try using a different version of the IAM library.
Try running your Dataflow job in a different region.
Try using a different service account.

If you have checked all of the above and you are still getting the error, then please provide more information about your Dataflow job, such as the pipeline code and the steps that you are taking to run the job. I may be able to provide more specific help if I have more information.

Additional notes:

The java.lang.IllegalAccessError is a Java runtime error that usually occurs when there is a version mismatch between compiled classes or libraries. It is not necessarily tied to permissions in the IAM sense.
The mention of the Google IAM API v1 in the original error message is likely because the com.google.iam.v1.TestIamPermissionsRequest class is from the IAM library. However, this does not mean that the IAM API itself is causing the problem or that it is not enabled.
The steps for checking permissions or the status of the IAM API are still relevant, but they should be considered separately from the java.lang.IllegalAccessError. IAM permission errors can also cause problems with Dataflow jobs, but they are typically manifested in different ways.

View solution in original post