Using GCS mount in Cloud Run at medium scale

[jldupont]

I am architecting an application and I am considering using a GCS fuse mount in Cloud Run. I am already using this capability in another project but for this new application, I need to understand the scaling properties.

1. One GCS fuse mount point only
2. The first "directory" level in the hierarchy would need to scale 100,000's directories
3. I do need to "list" the directories at the top level of the mount point (I would be using Firestore to keep track of the directories)
4. I would need to "list" the objects inside each top level directory though

So practically speaking, does this scale ? i.e. each time I add a directory, what are the impacts?

Thanks !

[yrhsk]

IMHO, GCS Fuse isn't designed for high-performance directory listing, especially with 100,000+ directories. Each directory listing can be slow and might impact performance as you scale. Using Firestore to track directories helps, but listing objects inside each directory could still be a bottleneck. For better scalability, consider using GCS APIs directly, which are more suited for handling large numbers of objects and directories efficiently.

[jldupont]

I've looked at the GCS Fuse source code but that's only 1 piece of the equation: what about the Cloud Run service itself: does it perform some `stat` operations ?

Come to think about this whole thing, I'll just use Rendezvous Hashing on a small set of directories at the root of a bucket and go through the API directly. This way, I have better control on the outcome I am looking for.

[jeerahalli]

Hello,

Thank you for contacting the Google Cloud Community.

I have gone through your reported issue, however it seems like this is an issue observed specifically at your end. It would need more specific debugging and analysis. To ensure a faster resolution and dedicated support for your issue, I kindly request you to file a support ticket by clicking here[1]. Our support team will prioritize your request and provide you with the assistance you need.

For individual support issues, it is best to utilize the support ticketing system. We appreciate your cooperation!

[1]: https://cloud.google.com/support/docs/manage-cases#creating_cases