Transfer Issue with tsop-agent v3.0.5219: Update to v3.0.5235

[reezz]

Hello everyone,

We recently encountered a problem with our file transfer configuration using the **tsop-agent** for Google Cloud Storage. Here is a summary of the situation:

#### Problem Description

- **Observed Error**:
E0621 13:52:09.606702 13863 handler.go:168] failed to initialize upload: generic::unknown: retry budget exhausted (10 attempts): Post "https://www.googleapis.com/upload/storage/v1/b/your_bucket_name/o?alt=json&ifGenerationMatch=0&uploa...": dial tcp: lookup www.googleapis.com on IP_ADDRESS_of theDNS:53 no such host

- **Scenario**:
1. Launch an STS job to synchronize a POSIX filesystem on-prem to a GCP bucket.
2. The synchronization consistently fails since 06/20.

#### Environment

- **Environment**: Production
- **GCP Service**: Storage Transfer Service / Storage Service
- **DNS Configuration**: DNS resolution for Google URLs (googleapis.com) is intentionally disabled by our security policies.
- **Proxy Configuration**: All HTTP requests are routed through a proxy that can resolve and route these requests.

#### Diagnosis

- **DNS**: No resolution for `www.googleapis.com` and `google.com`, as expected.
- **Proxy**: Correctly configured, `curl` commands work via the proxy.
- **STS Logs**: Transfers fail for files approximately 35 MB or larger.

#### Solution

After various tests and verifications, we discovered the problem stemmed from the version of the **tsop-agent**. An automatic update occurred, upgrading the agent from version **v3.0.5219** to version **v3.0.5235**, which resolved the issue.

#### Question

Has anyone else experienced this problem with **tsop-agent** version **v3.0.5219**? Do you know what was fixed in version **v3.0.5235** to resolve this issue? I searched the documentation but couldn't find the release notes to compare the two versions.

Thanks in advance for your help!

#### Additional Details for Context:

1. **DNS Configuration**: DNS resolution for Google URLs is intentionally disabled by our security policies.
2. **Proxy Configuration**: Verified and correct, with an initial warning about Boto configuration resolved.
3. **File Size**: Specific issue with files around 35 MB or larger.
4. **Logs**: Consistent error message on DNS resolution.

Thank you for your attention, and I hope someone can provide insights on the differences between these two versions of the **tsop-agent**.

[sathil]

Hello , 

Welcome to Google Cloud Community!

From internal links I can see that you raised the similar question via a support case. Will summarize the update briefly from the ticket for better visibility on this community, So that if any one refers to this issue they may be benefited from it.

As per the update please note that The agent will self recover from transient network failures (as much as possible).And there were some bug fixes b/w v3.0.5219 to v3.0.5235. For future references when encountering similar network failures issues it is recommended to update agent to the latest version which might resolve the issue.