Java 11 App Engine Standard not sending profiling information to Cloud Profiler

[SkylerWittman]

Hello everyone,

I'm running GAE standard, we've recently migrated from java8 to java11 with bundled legacy services. So we are still deploying via appengine-web.xml, not app.yaml.

So far with java11 things have been pretty smooth, except our servers are running out of memory faster due to unknown memory leaks since upgrading to java11. I'm using the largest server size for backend and still hitting memory ceilings often.

I'd like to profile my application using Cloud profiler to understand where memory leaks are occurring and fix them. I've followed the guide here for java11 to profile my application: https://cloud.google.com/profiler/docs/profiling-java#starting_your_program

1. I enabled the following env_var in appengine-web.xml: <env-var name="GAE_PROFILER_MODE" value="cpu,heap" />

2. I deployed it to my dev service and promoted it so our dev website points there, I used it for a few minutes

3. I made sure that the service account has cloud profiler permissions.

But my cloud profiler is empty and shows a welcome message. 

Does anyone have some suggestions or troubleshooting I can follow to understand why my profiles aren't showing up?

Thanks.

[SkylerWittman · Accepted solution]

I've solved this. The problem in my case appears to be the fact that I am using the java8 way of deploying which is to use appengine-web.xml to set all configs but I am using java11 and second gen runtimes. Therefore the env variable for GAE_PROFILER_MODE was being passed into the app.yaml generated for java11 but that's not the expected way to declare it.

What I did to fix it was the following:

1. Build my app as normal with no GAE_PROFILER_MODE, which generated an app.yaml file.

2. I modified that app.yaml to include the following line from the documentation here the line being 

  JAVA_TOOL_OPTIONS: "-agentpath:/opt/cprof/profiler_java_agent.so=-logtostderr,-cprof_enable_heap_sampling=true"

3. Then I deployed the app as normal, and now the memory profile appears in Google Profiler