Download file aborted using axios nodejs in Google Cloud Run

[zj]

■Background

In Google Cloud Run using AXIOS with node.js for file downloading from atlassian Confluence,

when the content-length of the response is greater than 100M,

there is a high probability that the download will not succeed.

■What I did

Read response data in chunks. It found that a request oborted exception occurred when it reached the end.

The response data was not read completely.

In the local environment (out of Google Cloud Run), a 100M file can be downloaded successfully.

    const writer = fs.createWriteStream(filepath);

    const response = await Axios({

        url: fileUrl,

        method: "GET",

        responseType: "stream",

        validateStatus : null,

        httpAgent: new http.Agent({ keepAlive: true }),

        httpsAgent: new https.Agent({ keepAlive: true })

    }).catch(err => {

        console.log("axios error: "+ err);

    });

    const contentLength = response.headers['content-length'];

    console.log("contentLength: " + contentLength);

    let chunklength = 0;

    response.data.on("data", (chunk) =>  {          

        chunklength = chunklength + chunk.length;

        console.log("chunk-length-sum: "+ chunklength);

        writer.write(chunk);                

    });        

    response.data.on("aborted", () =>  {

    });

    response.data.on("close", () => {

    });

    response.data.on("end", () => {

    });

    writer.on("finish", async () => {

    })

■What I expect

Used the AXIOS with node.js can download files.

■Environment

node.js 18.7

AXIOS 0.26.2

Google Cloud Run

■Question

1)Are there some limits in the Google Cloud Run?

Best regards

[Andre_Fiesco]

Do you know if your service is running under HTTP/1 or HTTP/2? According to this documentation, Cloud Run by default downgrades HTTP/2 requests to HTTP/1 when those requests are sent to the container. If you want to explicitly set your service to use HTTP/2 end-to-end, with no such downgrading, you can configure it for HTTP/2.

Here the limiting factor is that, if your service is running HTTP/1, HTTP/1 limits the request limit to 32 MB and that limit cannot be changed.  Something that can be implemented is to configure your service to HTTP/2, since for HTTP/2 there is no request limit.  In case HTTP/1 it's what's causing the issue, the first link that was provided in this post can be used to implement HTTP/2 to your service.