Solved: Using GCS get within a Workflow returns 404 - Page 2

mr_cech · 01-26-2024 10:24 PM

Using a Google Workflow and armed with information from the Google Storage Connector, I had a goal of retrieving the application/json file that was stored via YAML. However, when I try to use the seemingly simple task I am disappointingly met with a 404.

Here is my YAML source on the Google Workflow:

main:
  steps:
  - list_items:
    call: googleapis.storage.v1.objects.list
    args:
      bucket: "processed-items"
    result: item_list
  - get_single_item:
    call: googleapis.storage.v1.objects.get
    args:
      bucket: "processed-items"
      object: "item_2024-01-24.json"
    result: single_item_json
  - returnOutput:
    return: '${single_item_json.entities}'

I have verified that the json object is in the bucket, both visually and because the result of the 'list_items' step shows it in the response body.

Am I using the get method incorrectly?

alexmoore

So I just tried copy pasting your code into a sample project with a file of that name in a test bucket and the `get_single_item` step works as expected. Is the object definitely there? does the service account of the workflow have access to it?

The last step however might be an issue. By default the `get` call will return the metadata of the object and there is no value 'entities', so this will fail. To see what values there are see:

https://cloud.google.com/workflows/docs/reference/googleapis/storage/v1/Overview#object

If however you want to get the json itself, you will need to add the following `alt` parameter to the 'get' call:

  - get_single_item:
    call: googleapis.storage.v1.objects.get
    args:
      bucket: "processed-items"
      object: "item_2024-01-24.json"
      alt: "media"

As per: https://cloud.google.com/workflows/docs/samples/workflows-connector-storage?hl=en

Hope that helps,

Alex

View solution in original post