Apigee Hybrid - Workload Identity Federation to replace GCP Service account Keys

[Ashk1]

Hi @dchiesa1 ,

I am trying to help out someone with a java callout code. and when we deploy the callout proxy, the deployment seems to be successful but when the API is triggerred we see the following errors:
I am not even a newbee in Apigee related stuff so not able to debug correctly.

{"level":"SEVERE","thread":"Apigee-Main-61","mdc":{"apiName":"test-api-v1","org":"xx-apigeehyb-test-app-cbe6","messageId":"xxxxx-2271-4cbc-bc1d-xxxxxx","env":"glb-test1-3","revision":"1","trackingId":"xxxx-872e-4f6a-a32a-xxxxx"},"className":"com.apigee.flow.execution.AbstractAsyncExecutionStrategy$AsyncExecutionTask","method":"logException","severity":"SEVERE","message":"Exception caught with message: Failed to execute JavaCallout. loader constraint violation: when resolving method \u0027org.slf4j.ILoggerFactory org.slf4j.impl.StaticLoggerBinder.getLoggerFactory()\u0027 the class loader com.apigee.messaging.resource.JavaResourceClassLoader @579d033b of the current class, org/slf4j/LoggerFactory, and the class loader \u0027app\u0027 for the method\u0027s defining class, org/slf4j/impl/StaticLoggerBinder, have different Class objects for the type org/slf4j/ILoggerFactory used in the signature (org.slf4j.LoggerFactory is in unnamed module of loader com.apigee.messaging.resource.JavaResourceClassLoader @579d033b, parent loader com.apigee.messaging.resource.DeprecatingClassesLoader @303669ef; org.slf4j.impl.StaticLoggerBinder is in unnamed module of loader \u0027app\u0027)","formattedDate":"2024-03-05T18:52:12.465Z"

I also see a bunch of SEVERE message before this error, could this be an after math of permission denied error or vice versa.

More logs:

{"level":"SEVERE","thread":"Apigee-Main-61","mdc":{"apiName":"test-api-v1","org":"xx-apigeehyb-test-app-cbe6","policyName":"JavaCallout.AWS.STSToken","messageId":"xxxx-2271-4cbc-bc1d-xxx","env":"glb-xxxx-3","revision":"1","trackingId":"xxxx-872e-4f6a-a32a-xxxx"},"className":"com.apigee.securitypolicy.SecurityPolicy$3","method":"run","severity":"SEVERE","message":"Permission denied (\"java.io.FilePermission\" \"/application/opt/apigee/apigee-runtime/webapps/api/org/slf4j/impl/StaticLoggerBinder.class\" \"read\") ProtectionDomain\u003dProtectionDomain (file:/application/opt/apigee/apigee-runtime/data/work/contract_glb-xxxx-3_264/a/shared-flow-v1__38/java/awsclient-1.0-SNAPSHOT.jar/ \u003cno signer certificates\u003e)\n JavaResourceClassLoader /application/opt/apigee/apigee-runtime/data/work/contract_glb-xxxx-3_264/a/shared-flow-v1__38/java\n \u003cno principals\u003e\n java.security.Permissions@36556a8d (\n (\"com.apigee.securitypolicy.RestrictedSocketPermission\" \"RestrictedSocketPermission\")\n (\"java.io.FilePermission\" \"/usr/lib/jvm/java-11-openjdk-amd64/-\" \"read,readlink\")\n (\"java.io.FilePermission\" \"/application/opt/apigee/apigee-runtime/data/work/contract_glb-xxxx-3_264/a/shared-flow-v1__38/java/../../../java/-\" \"read,readlink\")\n (\"java.io.FilePermission\" \"/tmpfs/src/gfile/jre/-\" \"read,readlink\")\n (\"java.io.FilePermission\" \"${javacallout.org.dir}/-\" \"read\")\n (\"java.io.FilePermission\" \"/application/opt/apigee/apigee-runtime/data/work/contract_glb-xxxx-3_264/a/shared-flow-v1__38/java/../../../../../java/-\" \"read,readlink\")\n (\"java.io.FilePermission\" \"/application/opt/apigee/apigee-runtime/data/work/contract_glb-xxxx-3_264/a/shared-flow-v1__38/java/awsclient-1.0-SNAPSHOT.jar/-\" \"read\")\n (\"java.io.FilePermission\" \"/application/opt/apigee/apigee-runtime/data/work/contract_glb-xxxx-3_264/a/shared-flow-v1__38/java/-\" \"read,readlink\")\n (\"java.lang.RuntimePermission\" \"accessClassInPackage.*\")\n (\"java.lang.RuntimePermission\" \"modifyThreadGroup\")\n (\"java.lang.RuntimePermission\" \"getProtectionDomain\")\n (\"java.lang.RuntimePermission\" \"getClassLoader\")\n (\"java.lang.RuntimePermission\" \"modifyThread\")\n (\"java.lang.RuntimePermission\" \"enableContextClassLoaderOverride\")\n (\"java.lang.RuntimePermission\" \"setContextClassLoader\")\n (\"java.lang.RuntimePermission\" \"stopThread\")\n (\"java.net.NetPermission\" \"specifyStreamHandler\")\n (\"java.util.PropertyPermission\" \"*\" \"read\")\n (\"java.net.URLPermission\" \"ftp:*\" \"*:\")\n (\"java.net.URLPermission\" \"https:*\" \"*:\")\n (\"java.net.URLPermission\" \"sftp:*\" \"*:\")\n (\"java.net.URLPermission\" \"http:*\" \"*:\")\n (\"com.apigee.securitypolicy.AllDenyAndLogExcept\" \"Alldenyandlogexcept\" \"java.io.FilePermission,java.net.SocketPermission,java.util.PropertyPermission,java.net.URLPermission,java.net.NetPermission:specifyStreamHandler,java.lang.RuntimePermission:modifyThread/stopThread/modifyThreadGroup/getProtectionDomain/getClassLoader/setContextClassLoader/enableContextClassLoaderOverride,java.io.SerializablePermission:enableSubclassImplementation\")\n (\"java.io.SerializablePermission\" \"enableSubclassImplementation\")\n)\n\n Grant\u003djava.security.Permissions@4e48096c (\n (\"com.apigee.securitypolicy.RestrictedSocketPermission\" \"RestrictedSocketPermission\")\n (\"java.io.FilePermission\" \"/usr/lib/jvm/java-11-openjdk-amd64/-\" \"read,readlink\")\n (\"java.io.FilePermission\" \"/application/opt/apigee/apigee-runtime/data/work/contract_glb-xxxx-3_264/a/shared-flow-v1__38/java/../../../java/-\" \"read,readlink\")\n (\"java.io.FilePermission\" \"/tmpfs/src/gfile/jre/-\" \"read,readlink\")\n (\"java.io.FilePermission\" \"${javacallout.org.dir}/-\" \"read\")\n (\"java.io.FilePermission\" \"/application/opt/apigee/apigee-runtime/data/work/contract_glb-xxxx-3_264/a/shared-flow-v1__38/java/../../../../../java/-\" \"read,readlink\")\n (\"java.io.FilePermission\" \"/application/opt/apigee/apigee-runtime/data/work/contract_glb-xxxx-3_264/a/shared-flow-v1__38/java/-\" \"read,readlink\")\n (\"java.util.PropertyPermission\" \"*\" \"read\")\n (\"java.net.NetPermission\" \"specifyStreamHandler\")\n (\"java.lang.RuntimePermission\" \"modifyThreadGroup\")\n (\"java.lang.RuntimePermission\" \"accessClassInPackage.*\")\n (\"java.lang.RuntimePermission\" \"getProtectionDomain\")\n (\"java.lang.RuntimePermission\" \"modifyThread\")\n (\"java.lang.RuntimePermission\" \"getClassLoader\")\n (\"java.lang.RuntimePermission\" \"enableContextClassLoaderOverride\")\n (\"java.lang.RuntimePermission\" \"stopThread\")\n (\"java.lang.RuntimePermission\" \"setContextClassLoader\")\n (\"java.net.URLPermission\" \"https:*\" \"*:\")\n (\"java.net.URLPermission\" \"ftp:*\" \"*:\")\n (\"java.net.URLPermission\" \"sftp:*\" \"*:\")\n (\"java.net.URLPermission\" \"http:*\" \"*:\")\n (\"com.apigee.securitypolicy.AllDenyAndLogExcept\" \"Alldenyandlogexcept\" \"java.io.FilePermission,java.net.SocketPermission,java.util.PropertyPermission,java.net.URLPermission,java.net.NetPermission:specifyStreamHandler,java.lang.RuntimePermission:modifyThread/stopThread/modifyThreadGroup/getProtectionDomain/getClassLoader/setContextClassLoader/enableContextClassLoaderOverride,java.io.SerializablePermission:enableSubclassImplementation\")\n (\"java.io.SerializablePermission\" \"enableSubclassImplementation\")\n)\n","formattedDate":"2024-03-05T18:52:12.462Z","logger":"SERVICES.SECURITY_POLICY_SERVICE"}

{"level":"WARNING","thread":"Apigee-Main-61","mdc":{"apiName":"test-api-v1","org":"xx-apigeehyb-test-app-cbe6","policyName":"JavaCallout.AWS.STSToken","messageId":"xxxx-2271-4cbc-bc1d-xxx","env":"glb-xxxx-3","revision":"1","trackingId":"xxxx-872e-4f6a-a32a-xxxx"},"className":"com.apigee.messaging.resource.LogOnce","method":"lambda$warnLog$0","severity":"WARNING","message":"Non Apigee class loaded from Gateway ClassLoader : org.slf4j.impl.StaticLoggerBinder","formattedDate":"2024-03-05T18:52:12.464Z","logger":"CONFIG-CHANGE"}

{"level":"SEVERE","thread":"Apigee-Main-61","mdc":{"apiName":"test-api-v1","org":"xx-apigeehyb-test-app-cbe6","messageId":"xxxx-2271-4cbc-bc1d-xxx","env":"glb-xxxx-3","revision":"1","trackingId":"xxxx-872e-4f6a-a32a-xxxx"},"className":"com.apigee.flow.execution.AbstractAsyncExecutionStrategy$AsyncExecutionTask","method":"logException","severity":"SEVERE","message":"Exception caught with message: Failed to execute JavaCallout. loader constraint violation: when resolving method \u0027org.slf4j.ILoggerFactory org.slf4j.impl.StaticLoggerBinder.getLoggerFactory()\u0027 the class loader com.apigee.messaging.resource.JavaResourceClassLoader @579d033b of the current class, org/slf4j/LoggerFactory, and the class loader \u0027app\u0027 for the method\u0027s defining class, org/slf4j/impl/StaticLoggerBinder, have different Class objects for the type org/slf4j/ILoggerFactory used in the signature (org.slf4j.LoggerFactory is in unnamed module of loader com.apigee.messaging.resource.JavaResourceClassLoader @579d033b, parent loader com.apigee.messaging.resource.DeprecatingClassesLoader @303669ef; org.slf4j.impl.StaticLoggerBinder is in unnamed module of loader \u0027app\u0027)","formattedDate":"2024-03-05T18:52:12.465Z","logger":"MESSAGING.FLOW","exceptionStackTrace":"com.apigee.kernel.exceptions.spi.UncheckedException{ code \u003d steps.javacallout.ExecutionError, message \u003d Failed to execute JavaCallout. loader constraint violation: when resolving method \u0027org.slf4j.ILoggerFactory org.slf4j.impl.StaticLoggerBinder.getLoggerFactory()\u0027 the class loader com.apigee.messaging.resource.JavaResourceClassLoader @579d033b of the current class, org/slf4j/LoggerFactory, and the class loader \u0027app\u0027 for the method\u0027s defining class, org/slf4j/impl/StaticLoggerBinder, have different Class objects for the type org/slf4j/ILoggerFactory used in the signature (org.slf4j.LoggerFactory is in unnamed module of loader com.apigee.messaging.resource.JavaResourceClassLoader @579d033b, parent loader com.apigee.messaging.resource.DeprecatingClassesLoader @303669ef; org.slf4j.impl.StaticLoggerBinder is in unnamed module of loader \u0027app\u0027), associated contexts \u003d []}\n\tat com.apigee.kernel.exceptions.spi.UncheckedException.setPrintStack(UncheckedException.java:147)\n\tat com.apigee.steps.javacallout.JavaCalloutStepDefinition$SecurityWrappedExecution.execute(JavaCalloutStepDefinition.java:290)\n\tat com.apigee.steps.javacallout.JavaCalloutStepDefinition$CallOutWrapper.execute(JavaCalloutStepDefinition.java:108)\n\tat com.apigee.messaging.runtime.steps.StepExecution.execute(StepExecution.java:175)\n\tat com.apigee.flow.execution.AbstractAsyncExecutionStrategy$AsyncExecutionTask.call(AbstractAsyncExecutionStrategy.java:107)\n\tat com.apigee.flow.execution.AsyncExecutionStrategy.execute(AsyncExecutionStrategy.java:30)\n\tat com.apigee.flow.MessageFlowImpl.execute(MessageFlowImpl.java:617)\n\tat com.apigee.flow.MessageFlowImpl.resume(MessageFlowImpl.java:433)\n\tat com.apigee.flow.execution.ExecutionContextImpl.lambda$resume$0(ExecutionContextImpl.java:143)\n\tat com.apigee.threadpool.ThreadPoolManager.submitOnMainThreadPool(ThreadPoolManager.java:235)\n\tat com.apigee.flow.execution.ExecutionContextImpl.resume(ExecutionContextImpl.java:151)\n\tat com.apigee.messaging.adaptors.http.configuration.MessageProcessorHttpSkeletonFactory$MessageProcessorRequestListener.lambda$onHeaders$1(MessageProcessorHttpSkeletonFactory.java:517)\n\tat com.apigee.threadpool.RunnableWrapperForMDCPreservation.run(RunnableWrapperForMDCPreservation.java:22)\n\tat com.apigee.threadpool.QueueMetricsAwareTask.run(QueueMetricsAwareTask.java:31)\n\tat java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)\n\tat java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)\n\tat java.base/java.lang.Thread.run(Thread.java:829)\n"}

In my pom.xml I do not have any slf4j dependency, but I do see that my aws related code is dependent on it. So I am not planning to include slf4j-simple jar in pom file and try again tomorrrow.

Anything else you can suggest?
Please help.