We are developing a workflow that will store the results obtained from a web API into files in a bucket.
The workflow ends up generating a JSON with this format:
[{entry},{entry},{entry}...]
Each entry has multiple keys with corresponding values.
Here is the challenge, we want to load these JSON files into BigQuery and it does not like the format, it wants a NDJSON instead, something like this should be the format:
The question is: How can I convert my JSON to NDJSON in a Workflow? Is it even possible?
If its not, can we call a Function, send the //bucket/file and have the function convert it to NDJSON? How would it work?