Get hands-on experience with 20+ free Google Cloud products and $300 in free credit for new customers.

DATAFLOW: Weird graph in Estimated Cost feature from Dataflow

Hello people!

I love the Estimated cost feature since its a quick way to, not only get an estimated of the cost, but to compare to past version of the same job (a historic view would be nice!).

My question is:

Mostly, the graph looks like this, which is fine:  

Screen Shot 2023-05-26 at 10.06.46.png

  

 But this other graph looks weird, don't you think?

Screen Shot 2023-05-26 at 09.44.27.png

Context:
Both graphs are from the same job but different runs. 

 

 First oneSecond one
Elapsed time
 Succeeded
Running
Elapsed time
3 hr 59 min
3 hr 37 min
machine typen1-standard8n1-standard8
job typebatchbatch
Job region us-central1us-central1
Dataflow Prime DisabledDisabled
Runner v2 Enabled Enabled
Dataflow Shuffle Enabled Enabled
Current vCPUs
8
8
Total vCPU time
 63.347 vCPU hr
27.761 vCPU hr
Total memory time 
237.551 GB hr
104.188 GB hr
Current HDD PD 
25 GB
 25 GB
Total HDD PD time 
197.959 GB hr
87.032 GB hr
Total Shuffle data processed 
11.62 GB
4.88 GB
Billable Shuffle data processed 2.9 GB1.22 GB

--
Best regards
David Regalado
Web | Linkedin | Twitter

0 2 152
2 REPLIES 2

Good day @davidregalado25,

Welcome back to Google Cloud Community!

Unfortunately, you can't change this in the dataflow UI, its current visualization is set to Stacked Area Chart and you can only edit the time range for its charts and logs by clicking the dropdown (currently set to MAXTIME). However, you can edit the chart in the metrics explorer and save it to a dashboard by clicking the options button (three dots) and clicking (view in Metrics Explorer). You should be able to see different widget types (E.g. Line chart, stacked bar chart, etc.) that you can choose, and you can try editing the MQL query to your need, in this case, estimated cost per run. 

You can use this link for more information: https://cloud.google.com/monitoring/mql/query-editor
https://cloud.google.com/monitoring/mql/query-language

Hope this helps!

Hi!

Please, take a closer look, there are spaces with no data in that chart.

--
Best regards
David Regalado
Web | Linkedin | Twitter