Connection reset while reading file from GCS bucket

[MariuszW]

Hi Everyone

I'm experiencing weird issue while reading text file (CSV) stored in GCS bucket. It is being uploaded correctly but during processing it I'm getting two types of errors:

read: connection reset by peer 

or 

ERR: unexpected EOF

 Sometimes occurs one, sometimes another (the file processing is being stopped) and sometimes it works without problem (the file processing ends successfully).

I have GKE cluster version 1.22.15-gke.2500, I'm authenticating by service account HMAC and the SA has Storage Admin role assigned. I'm also using minio-go/v7.0.46 client to connect from pod to GCS bucket.

Does anyone has idea what could be the reason of the issue?

Thank you in advance.

[dionv]

Hello MariusZW,

If you find that connections drop frequently then I think it is because you are hitting rate limits. You can adjust the respective quotas through GCP Cloud Console by navigating to IAM & admin --> Quota. You can check this documentation for reference.