Dockerised Flask App Failing with Multiple Replicas

[joshua-n1]

Hi, I'm deploying a simple Flask app which uploads some data, processes it and then plots some graphs. I deploy it using Load Balancer. It works fine with 1 replica/pod in the deployment but as soon as I increase the number of pods, some of the graphs randomly no longer appear. I couldn't find an answer anywhere, please help!

[SebastianG]

Would be nice to see your deployment file content.

By the sounds of it this is a highly coupled type of service, LB > GKE > Ingress > Service > Pod(s).

[glen_yu]

It sounds like all your pods/containers have their own independent storage/data for graphs.  Hence when you only have 1 pod, it's always hitting that single pod which contains all your data.

example:
data a --> pod 1, data b --> pod 1, data c --> pod 1, etc.

But when you increased the number of pods and you don't have a single source of truth for your data (i.e. shared volume with PV & PVC), then data that gets written and read becomes fragmented.

example:
data a --> pod 1, data b --> pod 2, data c --> pod 1, data d --> pod 2, etc.

I already mentioned it, but what you need is a shared volume mount via PersistentVolumes (PV) and PersistentVolumeClaims (PVC) and have your data written to and read from there so no matter which pod traffic gets load balanced to, it's writing and reading from a single source of truth.