Dataflow PubSub to Elastic Text Transform Does Not Execute [Resolved]

[krm104]

I am attempting to use the PubSub to Elasticsearch Dataflow Template:
https://cloud.google.com/dataflow/docs/guides/templates/provided/pubsub-to-elasticsearch

I do not seem to be able to get a text transform function to operate correctly. I set the js file bucket path and specify the function name. When I push a message onto the topic, the message is consumed, but no document results in the elasticsearch index. There are no logs in the worker to indicate that the transform function ran. 

I have successfully used just an index and id function, I have put print statements in those functions and the worker logs show those prints. However, the text transform function just consumes the message and doesn't seem to actually run the js or pass the message through.

I have tried just using the pass through function that parses the input and stringifys the object right back, but again there are no logs or any indication of what the problem with the text transform function is.

Has anyone successfully used a text transform function with the Pubsub to Elasticsearch dataflow template or has any thoughts on how to debug the text transform function without any worker logs being printed?

[krm104 · Accepted solution]

I was able to resolve the issue. It was a syntax 'error' in the javascript that was preventing the logic from running in the Nashorn engine. I followed the steps in the template documentation and launched the engine in a shell to confirm functionality and was able to make the necessary modifications to enable execution.

https://cloud.google.com/dataflow/docs/guides/templates/create-template-udf#javascript-udf