Storage Basics

[JeremyBenson11]

I have some basic questions about storage. Is the boot disk the only storage on my startup instance with one of the smaller computer configurations? Basically I'm wondering how file storage works for user uploads. Is there a difference between using buckets, or a simple uploads folder? Users will be uploading job files for employees to download on the site. 

Thanks,

Jeremy.

[KirTitievsky]

Sounds like you are building a thing where human users can go to a website and upload file for processing. Then employees -- other humans -- can do things to these files and track their progress in some way.  Maybe it's just "download."   It also seems like you are considering three options:

1. Build an app that runs on a virtual machine and stores the data on local disk.

2. Storage the files in GCS.

3. "Simple  uploads folder."  I don't quite know what this means.

My guess is that to build this, you'd want to use GCS for storage and a database to keep track of users and files they load. For a database, an interesting starting configuration is Firebase DB which will also give you a nice front end for GCS.  The UIs would be served by, say, Cloud Functions (or Firebase functions if you go the Firebase route).  If you make it super basic, you can make your website be a cloud function that services a website for users and generates a signed, short lived signed  URL for uploads  (https://cloud.google.com/storage/docs/access-control/signed-urls#signing-resumable).  With this approach your app does not need to save anything itself -- no database.

[JeremyBenson11]

Thanks for the reply. By uploads folder I meant when users upload a file, I would process it via php, and store it in a folder outside root. An uploads folder. I was worried about not having "infinite" storage going this way, as I thought it might be on the Boot Disk which may have a max size. Google's Google Cloud Storage is a better option, why? As a side question is there a max size to Boot Disk on a Virtual Instance?

Thanks,

Jeremy.

[KirTitievsky · Accepted solution]

If "better" means "better than local disk," then with GCS you don't have to worry about managing the disk size, replication, or even keeping the machine  up.  You can host your PHP clients in a serverless environment (say, Cloud Run) and pay next to nothing for small amount of usage. At the same time, your data will be very safe.   As a side effect, when your internal users access the data, you can use granular access control management, audit logging, etc. out of the box with GCS. With bare disk, you need to manage all that yourself.

For boot disk sizes, see this. (TL;DR 64TiB)

[JeremyBenson11]

Yeah, I'm starting to see all that. Thanks for the reply.

[JeremyBenson11]

Is it okay to ask another question in here? I've added a couple of drives to my VM. I increased permissions of the folders made for them, mounted, added fstab entry, added read write permissions to the folders via 'sudo chmod a+w'. When I reset the VM the mounts were still there. So that's good. In a php script I'm trying to do a simple is_file() block on an xml file in one of the folders. I can't access the file no matter what amount of dots I put after __DIR__. I'm not sure what I could be doing wrong. I really don't think it's the path, I double checked, copy and pasted. Am I missing something?

My code is simple:

if(is_file(__DIR__ . '[path]')){echo 'is file';}