Cannot change dataset to physical storage billing

[banhmitrung]

When I tried to change through the storage billing I got these

Using SQL

ALTER SCHEMA does not support the option: storage_billing_model.

Using CLI

Exception raised in update operation: Storage billing model configuration is not available for the project.

Using Rest API:

{
"error": {
"code": 400,
"message": "Storage billing model configuration is not available for the project.",
"errors": [
{
"message": "Storage billing model configuration is not available for the project.",
"domain": "global",
"reason": "invalid"
}
],
"status": "INVALID_ARGUMENT"
}
}

Any idea how to fix this? Our data compress really well and we don't need time travel for these dataset. so would be super cost effective to use physical storage billing.

[Joevanie]

Follow this documentation. You should be able to do it in SQL, BQ, or API. To quote, you can update a dataset's storage billing model to use physical bytes instead of the default logical bytes when calculating storage charges. When you change a dataset's billing model, it takes 24 hours for the change to take effect. If you change a dataset's storage billing model to use physical bytes, you can't change it back to using logical bytes.

[kolban]

If Joevanie's post doesn't solve the puzzle, please post the exact command you ran (replace any specific / sensitive data).

[banhmitrung]

Hi @Joevanie and @kolban 

Yes, I had tried those  exacts 3 commands like I mention In my old post. I never changed it once before. This is the first time I ever tried to switch after seeing a massive bill.

You can see in below screenshot.

Then I tried using `bq` cli, I got this error:

Desparately, I tried hit the API directly 

[mbiebrich]

I'm having the exact same issue and get the same errors. 

I'm not sure if it matters, but we have multiple projects, they are billed under an organization. I've tried running the cli command while within the billing organization as well as multiple separate projects and still get the error. 

[banhmitrung]

Hi @Joevanie Yes, please see my reply to @kolban below. i tried all of 3 ways and none of them worked. I never changed it before, it was always default "logical billing". This is the first time I ever attempted to changed, and it was unsuccesful.

[kolban]

I've got an update ... I trawled through Google's known cases and found one called b/273219570.  This report exactly matches your story.  The last update on that case was 2023-03-17 (2 days ago) when the engineers reported "still investigating".   Bottom line, there doesn't appear to be anything obvious you are doing wrong.  This feature is flagged as pre-GA  (beta) so it looks like the rough edges are still being polished.  What I would suggest is to reach out to your own Google account team and point them to this post.  Show them the case ID listed in this post and have them raise a new case (or piggyback on the existing one) referencing them together.  That way you will be immediately notified when the issue is resolved.

I updated the existing case to point to this topic to say "Look ... another customer with the same issue".

[banhmitrung]

Thank @kolban It was great to hear we are not the only one.

[joshuaessex]

We're experiencing the same exact issue right now, preventing us from changing the billing model on a couple of large datasets that compress well and would see meaningful cost reduction through the change. We'd love to get this unblocked as soon as possible.

[kolban]

Sadly I'm not a BQ specialist but had a dig around.  What I seem to have found is that this physical billing storage technology was released as "Preview".  What I am reading is that the "Preview" has completed and Google is no-longer enrolling BigQuery consumers into the program.  What I am finding is that this capability is slated for full General Availability in 3Q 2023.  It will require the leveraging of BigQuery Editions.

[taarak18]

Is there any update on when this is going to be GA? Also any details on any performance gaps on moving from logical to physical billing?