Updating a streaming engine enabled dataflow pipeline for security patches of worker container OS

[Sathish_tu_ns]

As per the below doc on streaming engine, 
https://cloud.google.com/dataflow/docs/streaming-engine
Its mentioned as "Improved supportability, since you don't need to redeploy your pipelines to apply service updates."

Does this mean, if there is a security vulnerability in the dataflow workers VM instance as per the security bulletin page, then Dataflow backend service will take care of the update of the workers VM instances OS and we don't have to do anything manually as per the updating a pipeline page.

thanks in advance for your answers.

[ms4446]

Yes, you are correct. Google Cloud Dataflow service does update the VM image used by the workers to include mitigations for identified vulnerabilities. This was confirmed by a release note from Google Cloud Dataflow stating that the VM image had been updated to address multiple vulnerabilities. According to the note, jobs started on or after the specified date would run VM instances using this updated image​

https://cloud.google.com/dataflow/docs/release-notes

This means that when Google Cloud Dataflow Streaming Engine is enabled, you do not need to manually update the worker VMs for service updates, including security patches. The backend service handles these updates automatically, which could include OS updates for the VM instances. This provides enhanced supportability, as mentioned in your initial quote.

However, please remember that this does not negate the need for good security practices. Always adhere to the principle of least privilege, monitor for unusual activity, and keep informed about new vulnerabilities and best practices for mitigation.

Keep in mind that while Google Cloud Dataflow takes care of updates related to its own services, you might still need to take action for other components of your Google Cloud setup depending on the nature of the security bulletin.

[Sathish_tu_ns]

Hi, thanks for answering,

its mentioned in several places in the release notes page that dataflow jobs created on or after a date will new images, 

but its never mentioned in anywhere where it says that enabling streaming engine will update the running jobs to use patch OS for VM instances, 

if you dont mind, can you please share whether there is any documentation that clearly mentions this, 

Please let me know if you need further details, 

thanks for your time again