500000 item limit in GDrive

[svetcov]

How do we get the limit of 500000 items removed for a GDrive Directory?

[KAM]

@svetcov I thought the limit was 400K items in a shared drive. I'm not aware of anyway to raise the limit but you can just create another shared drive. -KAM

[svetcov]

It is 500K (https://support.google.com/a/users/answer/7338880?hl=en). The limit appears arbitrary, it is not clear why it could not be 500001 or 499999 instead. 

The use case does not lend itself easily to just splitting up a directory as that would incur a large amount of re-programming and then we would need to track the data in each shard and monitor if any individual shard (shared drive) needs to be again split. 

[svetcov]

I've seen this feature request (https://www.googlecloudcommunity.com/gc/Feature-Ideas/Increase-Item-Limit/idc-p/900117); however, it appears to be moribund with no comments over the last year until I commented on it today. I don't think this should be a feature request, it should be a configuration option - I'm just wondering how we change the configuration - 500,000 seems artificially low when competitors have the ability to handle millions of files.

[KevinApodaca]

You do not change the configuration.

[svetcov]

Not a problem, I'm fine with Google changing the configuration as long as the limit is changed. If Google would like to change the configuration to 5 Million, that would be helpful (still less than competitor limits, but certainly better than 500,000).

By the way, your response is fantastic. At least we know this isn't a configuration we can change, but Google can.

[svetcov]

@KevinApodaca - Just checking in. I know you have indicated that I cannot change the configuration - but you were very specific that "you" meaning "me" cannot change the configuration. Can Google go ahead and do so? That would be very useful.

[KevinApodaca]

There is no immediate plan to change this. The actionable next step would be to add your feedback and use case to the feature request linked above.

[svetcov]

@KevinApodaca - Your response continues to be unclear. Let me try to parse what you are saying based upon your comments here and above:

1. Yes, this is a configuration option and end user admins are not able to make the change.

2. This is currently a configuration option that Google could change.

3. Google has no plan to make this change.

4. You are suggesting that we make a feature request (which has existed for over a year with no action) and which I have already added myself to.

My comment - If this is a configuration option on Google's part, why wouldn't Google make this configuration change. This doesn't seem to be a feature request as you've already indicated that it is a configuration change Google could make? We cannot request a feature for you to make this configurable as it already is - it's just that Google is refusing to make the change. A feature request also shouldn't be, "We need a feature to change Google's corporate position on a configuration change." That seems like a stupid feature to request. 

What I am requesting isn't a feature change, it is a configuration change that Google has reserved to itself. You do not have a mechanism for requesting configuration changes that Google has reserved for itself - and thus I'm asking via Q&A if you can make the configuration change - which does seem to be the correct mechanism to request this. 

Question - Will you change the configuration from 500000 files to 5000000 files?

[KevinApodaca]

> Question - Will you change the configuration from 500000 files to 5000000 files?

No

[svetcov]

@KevinApodaca - Why not and why was 500000 selected arbitrarily? Can I appeal this decision to not change the configuration? If so, who can I appeal this decision to?

[StephenHind]

---

@svetcov wrote:

Why not and why was 500000 selected arbitrarily? Can I appeal this decision to not change the configuration? If so, who can I appeal this decision to?

---

It used to be 400,000 and only recently changed, so I imagine it's to do with performance.

Before Shared Drives was a thing we had a client who had 1.7 million files in a My Drive hierarchy: we needed to add a group to the entire hierarchy and it took over a week for the permissions to propagate.

I don't work for Google but I have used its products for an awful long time and these kind of restrictions are normally always a benefit to you to keep things working as it should.  This kind of restriction is normally always necessary (else why would it be there).

[svetcov]

@StephenHind, thanks for your comment. I'm moving over from Box and have a directory with 1.9M items. If Box can handle it, one would presume that Google could as well. It could very well have to do with performance, I would welcome @KevinApodaca or someone else at Google to chime in. We have had it confirmed that it isn't something that would be resolved through a feature request - although one has been out there for over a year (with no comments or progress). So, getting Google to comment on the configuration request seems reasonable. 

I'm kinda stuck - I want to migrate over and my users don't want multiple directories. So, I'm stuck dealing with Box (which otherwise we aren't using) because they don't want to re-write their application to deal with sharding across directories....I want to eliminate unnecessary solutions (which Box would be if Google would allow a larger number of files in a single directory).

[StephenHind]

---

@svetcov wrote:

If Box can handle it, one would presume that Google could as well

---

Well you could always migrate to a My Drive structure, but as already mentioned that's really not appropriate.

We're just about to help a client in a similar migration and the simple answer is to use multiple Shared Drives: you will probably need to due to how the permissions work differently in Drive compared to Box.

You're under the impression this can be resolved with a feature request: you've misunderstood.  It's not something that can be granted via a feature request: you need to make the request, others vote on it (for months), Google considers it (more months), if Google decides to add then development. 

Since Google has only just changed this recently it's unlike to change again any time soon, especially when ti's just released permission blocking.

[svetcov]

I don't think this is appropriate for a feature request. The capability already exists. This is a request for a configuration change. I don't think it is ever appropriate to request a configuration change through a feature request process.

[KevinApodaca]

This is a feature request 🙂

[svetcov]

The feature already exists - put X files into a single directory. The configuration of X needs to change, the feature remains the same. I'm not sure how you define a feature, but I would suggest that if merely changing X is a feature request, Google has a lot of configuration changes going through as feature requests.

This is like saying, I'm sorry, I'm not going to change from 8 character passwords to 10 character passwords unless we have a feature request and a user story and the item is placed into the backlog and sprint points are associated with it and someday in the far distant future, we might get a developer to work on it. No, you change the configuration from 8 characters to 10 characters and call it a day. It is crazy to send a configuration change through a feature request pipeline.

[KevinApodaca]

The feature request is to increase the limit from X to Y. Feature requests are not just for net-new functionality and certainly encompass improvements to existing limits/toggles/colors/etc

Just like previously, as Stephen mentioned, the feature request was to increase from 400,000 to 500,000.

I'm not sure there is very much actionable feedback that can be gathered from this discussion as it seems circular in nature.

[svetcov · Accepted solution]

I'm fine agreeing to disagree. I'm requesting a configuration change from 500000 to 5000000 items and you incorrectly believe that changing this configuration is a feature change. Good on you. I'm still requesting this as a configuration change - especially given that no comment or note of the feature request that has existed for a year has been made by Google. So, you can call this whatever you please, it is merely a change in configuration.

If you would like to call it a rose or anything else, have at it