Get hands-on experience with 20+ free Google Cloud products and $300 in free credit for new customers.

Help me understand Storage pricing during migration from Standard type to Archive type

At the moment I have a bucket with 25.7 TiB (26,317 GiB) of data and 117 millions (approximately) of objects. It is small logs and activity files accumulated for years. Bucket's location is US multi-region and storage type is Standard. I want to optimize cost of this bucket because I'm not using these files in any way, but they must be kept. I'm planning to change storage type to Archive and change location to a single US region.

If I understand correctly:
- if I want to change only storage type, I can either change storage type of existing bucket or create new bucket with appropriate type and move all items into it
- if I want to change both storage type and location, the only I can is to create new bucket with appropriate location and type

Based on this I did pricing calculations:
- change storage type of existing bucket to Archive using Lifecycle manager: 117000000 ÷ 1000 × 0.1 [class A operations] ~ $12,000
- create new bucket in a single US region with Archive storage type and move all items into it using "TRANSFER DATA" button: (117000000 ÷ 1000 × 0.01) [storage.objects.list on the source table] + (117000000 ÷ 1000 × 0.005) [storage.objects.copy on the destination table] + (26317 * 0.02) [inner data transfer] ~ $2,300
- remove entire bucket: free
- keep current bucket as is: 26317 × 0.026 [GB price] ~ $700 [per month]
-
store items in the new bucket in a single US location and Archive storage type: 26317 * 0.0012 [GB price] ~ $32 [per month]
download current bucket using "gsutil cp": (117000000 ÷ 1000 × 0.01) [storage.objects.list] + (1000 × 0.12 + 9000 × 0.11 + 16317 × 0.08) [data transfer to US] ~ $3,600

Mostly I wonder on the first two calculations. Why difference of total costs is so big? These two actions achieve same purpose - they changing storage type. The second one is even better because I'm changing location to a single region. Because of such big difference I'm thinking that all my calculations are wrong. It is not really clear what operations of what class are involved when I'm using Lifecycle manager, TRANSFER DATA button, gsutil cp.

I'm asking this:
1. please verify or correct above calculations
2. perhaps is there any other way to optimize pricing of the source bucket? I just want to store all the items, there is no plans to interact with them in any way

Thanks!

Solved Solved
1 1 238
1 ACCEPTED SOLUTION

 

  • Lifecycle Management (Storage Class Change):
    • Your calculation of $12,000 is approximately correct, as it is class A operations. Each object storage class change results in a class A operation.
    • 117,000,000 objects / 1,000 * $0.10 (Class A operation) = $11,700.
  • "Transfer Data" (New Bucket, Storage Class, and Location):
    • List Operations: 117,000,000 objects / 1,000 * $0.01 = $1,170.
    • Copy Operations: 117,000,000 objects / 1,000 * $0.005 = $585.
    • Data Transfer (Inter-region): 26,317 GiB * $0.02 = $526.34.
    • Total "Transfer Data" cost: $1,170 + $585 + $526.34 = $2,281.34.
    • The total cost of the transfer data operation is thus around $2,300. Your previous calculation was relatively accurate.

The significant cost difference stems from the Lifecycle Manager's Class A operations (per-operation cost) being much higher than the Class B operations (list, copy) used in the "Transfer Data" approach. Lifecycle Manager applies the Class A operation to each object when transitioning storage classes, resulting in the higher cost. Creating a new Archive-storage, single-region bucket and transferring data via "Transfer Data" is indeed the most cost-effective approach for long-term, infrequent access storage.

 

Please let me know if you need more clarifications! 🙂

 

View solution in original post

1 REPLY 1

 

  • Lifecycle Management (Storage Class Change):
    • Your calculation of $12,000 is approximately correct, as it is class A operations. Each object storage class change results in a class A operation.
    • 117,000,000 objects / 1,000 * $0.10 (Class A operation) = $11,700.
  • "Transfer Data" (New Bucket, Storage Class, and Location):
    • List Operations: 117,000,000 objects / 1,000 * $0.01 = $1,170.
    • Copy Operations: 117,000,000 objects / 1,000 * $0.005 = $585.
    • Data Transfer (Inter-region): 26,317 GiB * $0.02 = $526.34.
    • Total "Transfer Data" cost: $1,170 + $585 + $526.34 = $2,281.34.
    • The total cost of the transfer data operation is thus around $2,300. Your previous calculation was relatively accurate.

The significant cost difference stems from the Lifecycle Manager's Class A operations (per-operation cost) being much higher than the Class B operations (list, copy) used in the "Transfer Data" approach. Lifecycle Manager applies the Class A operation to each object when transitioning storage classes, resulting in the higher cost. Creating a new Archive-storage, single-region bucket and transferring data via "Transfer Data" is indeed the most cost-effective approach for long-term, infrequent access storage.

 

Please let me know if you need more clarifications! 🙂

 

Top Labels in this Space
Top Solution Authors