Increase maximum size of an individual part

[henriquedac]

I'm trying to upload a file via API to my google drive account, but there is a 5 GiB limit on the maximum size of an individual part. Is it possible for google to increase this limit? How to make?

[Marramirez]

Hi @henriquedac,

Welcome to the Google Cloud Community!

If you're referring to this documentation: Method: files.insert, then 5,120 GB is equivalent to 5.12TB. It is also stated in storage and upload limits for Google Workspace that the maximum individual file size that you can upload or synchronize is 5TB.

Let me know if it helped, thanks!

[henriquedac]

Hi @Marramirez,

Sorry if any words seem confusing, I'm using google translate.

I'll try to explain it better below:

Google Drive allows uploading files up to 5TB, as you said. I created an API so that files from my local computer are sent via a script created in Python, directly to my Google Drive.

However, when I try to send these files via API, it gives me an error message saying that the size limit for a single file has been exceeded. That is, although I have plenty of space in my Drive, the fact that a single file has a size that it considers large, the API is unable to send.

To be clear, the files that the limit error occurs are 400GB, 200GB, 50GB files, and finally I tried with a 4GB file.

I did a test by sending these files directly from the Google Drive web interface, and I was able to send them successfully.

The size limit message is only when I try to send via API, as I mentioned earlier.

Did you understand me?

[hectorjacales]

This seems like an buffer/channel size issue, Are you using the python Google Drive (Workspace) client?. Are you using resumable uploads? [2]

Can you share more details on the error message, like a screenshoot ? 

[henriquedac]

Hi @hectorjacales,

The script is using the Google Drive REST API, with OAuth 2.0 authentication through a service account, to interact with Google Drive.

The log file has the following message:

File SERVERCTRL01_Pessoal_FULL_20230814_013000.bak compressed.
Traceback (most recent call last):
File "C:\BkpsGoogleDrive\BkpFortesCompleto_Python.py", line 49, in <module>
File "C:\Users\Administrador\AppData\Local\Programs\Python\Python311\Lib\site-packages\googleapiclient\_helpers.py", line 130, in positional_wrapper
return wrapped(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Administrador\AppData\Local\Programs\Python\Python311\Lib\site-packages\googleapiclient\http.py", line 1022, in next_chunk
raise ResumableUploadError(resp, content)
googleapiclient.errors.ResumableUploadError: <HttpError 403 when requesting None returned "The user's Drive storage quota has been exceeded.". Details: "[{'message': "The user's Drive storage quota has been exceeded.", 'domain': 'usageLimits', 'reason': 'storageQuotaExceeded'}]">

[kumards]

Hi @henriquedac 

Because this question is about Google Drive, consider cross-posting to the Google Workspace community forum, as well, for visibility to more experts who might be visiting that forum: https://www.googlecloudcommunity.com/gc/Google-Workspace/ct-p/google-workspace.

[henriquedac]

Dear,

Summarizing my case, I explain once more:

I'm trying to send some files to Google Drive via a script developed in Python and I'm not succeeding.

I have a paid Google Workspace account and there I have over 30TB of free storage. I also checked that the size limit for a single file is 5TB.

I have an 18GB file and I sent it through the browser, with the purpose of testing, and the sending was successful.

My problem is when I try to send via Python script. It brings me in the error log:

Error sending FILE.zip: <HttpError 403 when requesting None returned "The user's Drive storage quota has been exceeded.". Details: "[{'message': "The user's Drive storage quota has been exceeded.", 'domain': 'usageLimits', 'reason': 'storageQuotaExceeded'}]">
403
Forbidden
b'{\n "error": {\n "code": 403,\n "message": "The user\'s Drive storage quota has been exceeded.",\n "errors": [\n {\n "message": "The user\'s Drive storage quota has been exceeded.",\n "domain": "usageLimits",\n "reason": "storageQuotaExceeded"\n }\n ]\n }\n}\n

The error indicates that the Google Drive storage quota for the user or service account being used has been exceeded. Which is not the case.

The library used in Python to communicate with Google Drive is "GoogleAPIClient". This library is part of the Google Client SDK and allows interaction with Google services, including Google Drive.

Thanks a lot if anyone can help me.

[hectorjacales]

It seems the root cause is the use of the service account (1).  I guess you can try to use your personal user account or impersonate the sa with an user account.  Or talk with Workspace to increase the quota for SA.

https://developers.google.com/drive/api/guides/api-specific-auth

1 - https://stackoverflow.com/questions/40535355/how-to-fix-the-storage-exceeded-issue-for-google-drive-...