Long running tasks

[thinrhino]

Hello,
We recently migrated our Standard App Engine code from Python 2.x to Python 3.7 + Flask + Gunicorn

Previously, if a service / module is manually scaled the timeout for a task was 24 hours or so. We used to run all our long running tasks (upto 10 mins or so) inside a manually scaled service.

With Python 3.7 we realised that timeout is defined by the timeout parameter to Gunicorn. 

We previously used deferred.defer(), moved some of those to taskqueue.add(), but they still fail as per timeout of Gunicorn.

I wanted to know how we can run our long running tasks in the new scheme of things.

[christianpaula]

Hi @thinrhino,

The default timeout of gunicorn is 30 seconds when the entrypoint field is specified. Workers silent for more than this many seconds are killed and restarted. If your handler takes more than 30 seconds, you may face  WORKER TIMEOUT error. To ensure that the request successfully completes, based on your use-case, you can use the -- timeout flag in the entrypoint field to increase the timeout.

Value is a positive number or 0. Setting it to 0 has the effect of infinite timeouts by disabling timeouts for all workers entirely.

Generally, the default of thirty seconds should suffice. Only set this noticeably higher if you’re sure of the repercussions for sync workers. For the non sync workers it just means that the worker process is still communicating and is not tied to the length of time required to handle a single request.

You may check this documentation about gunicorn timeout.

Thanks

[thinrhino]

@christianpaula Thanks for your response.

In my mind, setting gunicorn timeout to > 60s is not a good solution.

What is the Google suggested way to run tasks which need more than, say 10 mins?

We run our backend on GAE. Our mobile app talks to a service with is auto scaled and all these long running tasks are run on a service which is manual / basic scaling. We have been using GAE for over 7 years and recently migrated all our code from Py 2.7 to Py 3.7.

Thanks.

[christianpaula]

Hi @thinrhino,

Thanks for the response. I would highly suggest to contact Google Cloud Support for this matter.

Thank you.