How to find failed disk snapshot under all gcp projects

[Sahilsoni]

Hi Team,

I am unable to fetch all the failed snapshot of all disk under all projects. Tried alot but failed

Any one who can help me to fetch the same with through command or script ?

[kolban]

Can you possibly elaborate on your question?   What do you mean by a failed snapshot?  I think you are talking about "Persistent Disk Snapshots".   Are you saying that the request to create a snapshot is somehow failing?   Do you have any information you can share on what the nature of a failure looks like?

[Sahilsoni]

In the VM we have one boot disk and other storage disk.

In my company we have approximately 600 Servers and that servers have more than 2500 Attached Persistent Disks.

We have attached snapshot backup schedule which run at 11 pm CST and daily few disk snapshot getting failed. 

I want to fetch the data in one script. I am facing challenges in creation

[kolban]

Ideally, we can solve the puzzle of why ANY disk snapshots are failing and cure those.  However, I now understand your question.  You have 2500 persistent disks that are having snapshots taken of them and some fail to have those snapshots.  We now need to determine "which" disks have failed.  My immediate thinking is that a failure to snapshot a disk should result in a record being written to Cloud Logging.  What I think we want to do is have a script that executes a query against Cloud Logging that returns the (possibly empty) set of records that correspond to the failures.  A first thought is to use gcloud logging read with a suitable filter to retrieve the records.  Once we get the records, we can then process them.   For example, gcloud logging read can output either CSV or JSON.   Another thought is that Cloud Logging can additionally direct filtered output to BigQuery or GCS.  These might be alternate places to record failures.

I think I'd like to ask that you find an example log record and post it back here in JSON format.  Redact anything you consider sensitive in the output as this is a public forum.  That will help us formulate a query AND give us a clue to what the absolute underlying issue may be.