Get hands-on experience with 20+ free Google Cloud products and $300 in free credit for new customers.

gcsfuse mount without -implicit-dirs in Batch

We are mounting a Bucket to our Batch Tasks and passing in gcsfuse mount options.

When we look at the GCP Logs, we see that GCP is adding `implicit-dirs` which we do NOT want. Is there any way to have greater control over this?

```

Executing script runnable script:{text:"# This script helps mount an existing Google Cloud Storage as a volume\n# to a given path with provided mounting options.\n# It has been tested successfully with the following operating systems:\n# - Debian GNU/Linux 10 (buster, amd64 built on 20220317, supports Shielded VM features)\n# - Debian GNU/Linux 11 (bullseye, amd64 built on 20220822, supports Shielded VM features)\n# - CentOS 7 (x86_64 built on 20220406, supports Shielded VM features)\n# It may or may not work correctly with other operating systems and versions.\n\nOSID=\"$(. /etc/os-release && echo \"$ID\")\"\nif [ ! \"$OSID\" = \"debian\" ] && [ ! -f /etc/centos-release ] && ! grep -qi cos /etc/os-release; then\n echo '[Batch GCS Mount] Warning: this script has not been fully tested on the current operation system, it may fall.'\nfi\n\nif grep -qi cos /etc/os-release; then\n export PATH=/var/lib/google:$PATH\nfi\n\nif ! type gcsfuse >/dev/null 2>&1 ; then\n echo '[Batch GCS Mount] gcsfuse not found: cannot handle GCS mount.'\n exit 200\nfi\n\nif ! type fusermount >/dev/null 2>&1 ; then\n echo '[Batch GCS Mount] fusermount not found: cannot handle GCS mount.'\n exit 201\nfi\n\nif [ ! -d /mnt/disks/data-processor-data ]; then\n mkdir -p /mnt/disks/data-processor-data\nfi\nif grep -qi cos /etc/os-release; then\n # Set temporary directory for local GCS object copies to be /mnt/stateful_partition instead of the default /tmp.\n # This can guarantee gcsfuse to use the largest /dev/sda1 partition in the VM with Batch Container-Optimized OS image.\n gcsfuse -o allow_other --file-mode 777 --dir-mode 777 -client-protocol http1 --max-conns-per-host 1000 --experimental-local-file-cache --billing-project bborie-billing-test-2 -implicit-dirs --app-name google-batch --temp-dir /mnt/stateful_partition dataprocessor-e33f8bb /mnt/disks/data-processor-data > /dev/null 2>&1\nelse\n gcsfuse -o allow_other --file-mode 777 --dir-mode 777 -client-protocol http1 --max-conns-per-host 1000 --experimental-local-file-cache --billing-project bborie-billing-test-2 -implicit-dirs --app-name google-batch dataprocessor-e33f8bb /mnt/disks/data-processor-data > /dev/null 2>&1\nfi\n"} for Task action/STARTUP/0/0/group0 in TaskGroup group0 of Job sentinel2-mosaic-2-70b928f4-7a30-47a40.

```

 

Solved Solved
0 1 1,175
1 ACCEPTED SOLUTION

Found a reasonable solution for my needs. Instead of letting Batch manage the mounting of gcsfuse to the VM, the privileged Docker Container is now responsible

View solution in original post

1 REPLY 1

Found a reasonable solution for my needs. Instead of letting Batch manage the mounting of gcsfuse to the VM, the privileged Docker Container is now responsible