This website uses Cookies. Click Accept to agree to our website's cookie use as described in our Privacy Policy. Click Preferences to customize your cookie settings.
Hi,I have created a cloud function in Google cloud and I will like to
call the function from inside a workflow I have in power automate, the
key issue is authorisation, the function is authorised using a Bearer
token which expires every 1 hour, I hav...
Hi All,I am trying to create a document that is meant to be part of our
semantic layer and the information I need can be found in the dataplex
search as it contains column information and description and also the
table descriptions, data lineage etc....
Hi All,I am trying to set up my teams' infrastructure in BQ and
currently, we are looking to have 2 projects 1 for prod and 1 for dev.
and in each of the projects we hope to set up our BQ with a medallion
structure. we hope to have the prod to be a c...
I am currently managing a dataform project with numerous
transformations. These transformations are executed through a
combination of workflow configuration and release configuration for
orchestration, scheduled to run daily, weekly, or monthly, depe...
Hi all,I have the configuration below in my dataform.json file{
"defaultSchema": "datalake_analytics", "assertionSchema":
"dataform_assertions", "warehouse": "bigquery", "defaultDatabase":
"cfc-2017-site-login", "defaultLocation": "EU"} the "defaultL...
Hello @novas and @ms4446 ,What does the Dag_ID stand for? is it the same
as a tag defined in the config file? So currently I have a dataform
project with multiple transformations, I currently run these
transformations using the workflow configuration...
So how do I handle this if I want to work with datasets stored in
multiple locations in GCP? Cos I noticed that dataform only runs for
datasets that are saved in the same location as the default location.
I have tried that before and when I did, it throws an error that says
location is not a parameter.I am currently thinking this feature is not
yet live in dataform.
@ms4446 wrote:config {type: "incremental",bigquery: {partitionBy:
"DATE(timestamp)",clusterBy: ["user_id",
"event_id"]}}SELECT*FROM${ref("your_source_table")}WHEREDATE(timestamp)
> (SELECT MAX(DATE(timestamp)) FROM ${self()}) So I must create the
tab...
