Get hands-on experience with 20+ free Google Cloud products and $300 in free credit for new customers.

BQ Loader - Multiple Service Account

Dear All,

I need a support to understand how I can use the following command:

  • bq show
  • bq load

in my shell script using 2 different service account dynamically.

At the moment I didn't find any solution and I notice that the command connect to the current active account, so the only way is to switch every time between the 2 account, but for me is not a suitable solution.

The goal is to load a file into dedicated table specifying the SA that can access on this table

Could you please support me?

Details:

  • Last gcloud CLI
  • 2 Service Account
  • 2 JSON file for authentication (one for each service accounts)

Best Regards

Simone

Solved Solved
2 5 1,217
1 ACCEPTED SOLUTION

When running shell scripts like Shell1.sh with environment variables such as GOOGLE_APPLICATION_CREDENTIALS being set within the script, concurrency issues you're concerned about typically do not arise due to how environment variables are handled in Linux environments. 

Each time a shell script is executed, it runs in a separate process. When you set an environment variable within a script (using export in bash, for instance), it affects only the current shell and its child processes. This means that:

  • If User 1 and User 2 run Shell1.sh simultaneously in your application, each invocation of the script runs in its own process.
  • When Shell1.sh sets GOOGLE_APPLICATION_CREDENTIALS using export, it sets the variable only for that script's process and any processes spawned by it, not for the entire system or other concurrent instances of the script.

Example Scenario 

  • User 1 Calls Shell1.sh: This script sets GOOGLE_APPLICATION_CREDENTIALS to SA1_KEY_FILE. This change is local to the process started for User 1.
  • User 2 Calls Shell1.sh Simultaneously: A new, separate process is started for User 2. When this script sets GOOGLE_APPLICATION_CREDENTIALS to SA2_KEY_FILE, it does so only within its process.

Implications

  • No Interference: The GOOGLE_APPLICATION_CREDENTIALS set by User 1's process does not interfere with the one set by User 2's process. They are entirely isolated.
  • Concurrency Safety: You can safely run multiple instances of your script concurrently without worrying about them overwriting each other's environment variables.

Best Practices 

While the environment variable approach is safe from concurrency issues in the scenario you described, consider the following to ensure overall system robustness:

  • Permissions: Ensure each service account has only the permissions necessary for the tasks it needs to perform to adhere to the principle of least privilege.
  • Security: Secure your service account key files and consider using secrets management solutions to handle them, especially in a multi-user environment.
  • Monitoring: Implement logging and monitoring to track the usage of service accounts and detect any unexpected access patterns or failures.

Your approach of dynamically setting GOOGLE_APPLICATION_CREDENTIALS within separate instances of a script is safe from concurrency issues due to the process isolation provided by the operating system.

View solution in original post

5 REPLIES 5