Declare sources dynamically

[dogvile]

Hi,

I was wondering whether there is a way to declare sources in separate file (sources.js) dynamically.
For example, to read a configuration table (BQ) and populate the arrays in file (sources.js)

declare({

  schema: "demo",

  name: "demo1"

});

declare({

  schema: "demo",

  name: "demo2"

});

declare({

  schema: "demo",

  name: "demo3"

});

Sources Table:

Schema | Name

demo    |  demo1

demo    |  demo2

demo    |  demo3

[jangemmar · Accepted solution]

Hi @dogvile,

Welcome to Google Cloud Community!

While Dataform offers flexibility and advanced use cases, declaring sources dynamically in a separate file is not available at the moment. Currently, Dataform allows you to declare multiple data sources in one JavaScript declaration file with the Dataform declare Javascript method.

Here are the steps in declaring a data source in a JavaScript file:

1. In your development workspace, in the Files pane, expand definitions/.
2. Select a Javascript file
3. Enter the following code snippet:
   

 declare({
   database: "DATABASE_PROJECT_ID",
   schema: "BIGQUERY_SCHEMA",
   name: "RELATION_NAME",
 });

To declare another data source in the same file, simply add an extra declare block to the file.

I hope the above information is helpful.

[dogvile]

Thank you for your reply. I have concerns since when you have multiple tables is not very efficient this way. In general, you have a configuration table so can prevent duplicates and so on.