System Design: Resource Management Best Practices

omkarsuram_ · ‎12-13-2021

In this article, you'll find recommendations and best practices focused on the topic of Resource Management, as part of the System Design Pillar of the Google Cloud Architecture Framework.

Throughout this article, we often refer to the cloud resource management documentation. We suggest you review this documentation to learn basic concepts before evaluating the following assessment questions and recommendations.

Resource management

How will your organizational structure map to the Google Cloud resource hierarchy?

Create an Organization node with your domain. Define your resource hierarchy that will map to your organization’s business needs and requirements on Google Cloud, (e.g. isolation of business units or departments, such as HR, Finance, etc. or functional separation).
A Google Workspace or Cloud Identity account can have exactly one Organization provisioned with it. When a user with a Google Workspace or Cloud Identity account creates a Google Cloud Project, an Organization resource is automatically provisioned for them.
Set policies at the Organization level and at the project level rather than at the resource level. As new resources are added, you may want them to automatically inherit policies from their parent resource. For example, as new virtual machines (VMs) are added to the project through autoscaling, they automatically inherit the policy on the project.

Create an Organization node with your domain. Define your resource hierarchy that will map to your organization’s business needs and requirements on Google Cloud, (e.g. isolation of business units or departments, such as HR, Finance, etc. or functional separation). A Google Workspace or Cloud Identity account can have exactly one Organization provisioned with it. When a user with a Google Workspace or Cloud Identity account creates a Google Cloud Project, an Organization resource is automatically provisioned for them. Set policies at the Organization level and at the project level rather than at the resource level. As new resources are added, you may want them to automatically inherit policies from their parent resource. For example, as new virtual machines (VMs) are added to the project through autoscaling, they automatically inherit the policy on the project.

How is your billing organized? Is it by department?

Establish your cost and billing processes and requirements, i.e. cost center, projects, and budgets linked to departments, invoicing, and POs.
Assign labels to resources to allow for granular cost and billing reporting. Labels can be based on additional attributes besides the integrated reporting structures, such as per project or per product type. This can help to allocate consumption to cost centers, departments, or specific projects, or for internal recharge mechanisms. Refer to the Cost Optimization Pillar for more details.

Establish your cost and billing processes and requirements, i.e. cost center, projects, and budgets linked to departments, invoicing, and POs. Assign labels to resources to allow for granular cost and billing reporting. Labels can be based on additional attributes besides the integrated reporting structures, such as per project or per product type. This can help to allocate consumption to cost centers, departments, or specific projects, or for internal recharge mechanisms. Refer to the Cost Optimization Pillar for more details.

Do your resources need multi-tenancy?

Use labels and tags to apply granular access and management principles to different tenant resources and services.

Use labels and tags to apply granular access and management principles to different tenant resources and services.

How will you define your folder structure?

Define and specify a folder structure within your Google Cloud resource hierarchy that will map to your business needs. The folder structure is flexible and easily extensible, so you can start simple, and add more levels when needed.
Environment: Dev, Test, Prod (a simple, flat folder structure)
Function and Environment: apps, IT, etc. | Dev, Test, Prod (a hierarchical folder structure)
Business Unit, Function, and Environment: engineering, HR, finance etc. | apps, IT, security, etc. | Dev, Test, Prod (a hierarchical folder structure)

Define and specify a folder structure within your Google Cloud resource hierarchy that will map to your business needs. The folder structure is flexible and easily extensible, so you can start simple, and add more levels when needed. Environment: Dev, Test, Prod (a simple, flat folder structure) Function and Environment: apps, IT, etc. | Dev, Test, Prod (a hierarchical folder structure) Business Unit, Function, and Environment: engineering, HR, finance etc. | apps, IT, security, etc. | Dev, Test, Prod (a hierarchical folder structure)

Do you plan to use resource isolation and separation?

Use folders, subfolders, and projects to separate resources from each other to reflect data governance policies within your organization, such as separating finance, HR, and engineering resources.
Use projects to group resources that share the same trust boundary. For example, resources for the same product or microservice can belong to the same project.

Use folders, subfolders, and projects to separate resources from each other to reflect data governance policies within your organization, such as separating finance, HR, and engineering resources. Use projects to group resources that share the same trust boundary. For example, resources for the same product or microservice can belong to the same project.

How will you cater application or service needs for various control requirements?

Quotas and limits are enforced at the project level. Decouple and isolate workloads or environments at the project level to ease quotas and limits management. Use granular access management controls for critical environments at the project level or resource level to ease administrative complexity.
Implement data classification based on data sensitivity and compliance requirements. As a note, this is different from the separation of duties needed at the business units level.
Enforce billing isolation so you can support different billing accounts and more easily view costs by workload and environment.

Quotas and limits are enforced at the project level. Decouple and isolate workloads or environments at the project level to ease quotas and limits management. Use granular access management controls for critical environments at the project level or resource level to ease administrative complexity. Implement data classification based on data sensitivity and compliance requirements. As a note, this is different from the separation of duties needed at the business units level. Enforce billing isolation so you can support different billing accounts and more easily view costs by workload and environment.

Do you plan to enforce a unique naming scheme to apply management controls?

Anonymize information in project names. Follow a project naming pattern like {company-initial-identifier}-{environment}-{app-name} where the placeholders are unique but don't reveal company or application names. Do not include attributes that might change in the future, such as the team name or technology used.

Anonymize information in project names. Follow a project naming pattern like {company-initial-identifier}-{environment}-{app-name} where the placeholders are unique but don't reveal company or application names. Do not include attributes that might change in the future, such as the team name or technology used.

How will you automate project creation, delegate billing, and set up Identity and Access Management (IAM) governance?

Prevent accidental deletion by using project liens for any critical or production environments.

Key Google Cloud services

Resource Manager: Hierarchically manage resources by project, folder, and organization
Policy Intelligence: Smart access control for your Google Cloud resources

Resources

What's next?

We've just covered Resource Management as part of the System Design Pillar of the Google Cloud Architecture Framework. There are several other topics within the System Design Pillar that may be of interest to you:

Geographic Zones and Regions
Resource Management (you're here!)
Compute
Networking
Storage
Database
Analytics