Hello again,
I've been trying all day to custom the level of the logging Python library in my Beam data pipeline. I've been sending a parameter DEBUG through the UI, and according to the value, set something like:
if DEBUG:
log.setLevel(level=logging.DEBUG)
else:
log.setLevel(level=logging.INFO)
At the end of the day, I realized Dataflow has the total control of logging, so I'm using log.info for everything and controlling it accordingly.
Has this happened to you?
I'm using Apache Beam Python 3.8 SDK 2.37.0
LinkedIn
Twitter
