Get hands-on experience with 20+ free Google Cloud products and $300 in free credit for new customers.

Disable Dataproc cluster logs

Hi!

I have a Dataproc Single Node Cluster that is used as a Spark History Server. 

However I notice in Cloud Logging that this is creating quite a lot of logs. 
Looking at the documentation I can see that indeed there are a number of components that will generate logs: https://cloud.google.com/dataproc/docs/guides/logging#cluster_logs_in

But how do I alter/modify the behaviour of these logs? Ideally I want to set the log level to ERROR/OFF if possible. 

Of course I could always create a Exclusion Rule on the Sink level but that feels like I'm fixing the problem at the wrong end. 

Any ideas, since it isn't obvious from the documentation what is possible

0 1 1,031
1 REPLY 1