Data at Google Cloud Next '24!
Answer this question in the comments to earn a special badge: In your journey, what's one overlooked data skil...
Forum Posts
Influence Looker’s Roadmap and join the BI Customer Council (BICC)
Would you like to influence Google Cloud’s BI roadmap more? I’d like to invite you to join the Google Cloud BI...
Integrating External SQLX Models from GitHub Packages into Dataform Projects
I'm currently working with Dataform and facing a challenge regarding the integration of external SQLX models f...
Create Python Notebook button is unavailable in BigQuery
Hi there!I want to create a Python notebook on a BigQuery console but the button is unavailable. It also shows...
Dataform operations, validate if csv exist in bucket before trying to upload its content
Good day, I have the following sqlx file of type "operations", it is loading data from a csv file (in a bucket...
Executing a Dataform action only after all dependencies of this action are executed
@ms4446 @maffka123 Hello, I have a similar to this request : https://www.googlecloudcommunity.com/gc/Data-Anal...
Analyse cost per dataform schedule
Hi there,We are doing a project on cost optimisation and wanting to try and understand how much each schedule ...
How to add dependencies from .js script in Dataform or use wildcard/tag inside dependencies?
I have about 20 scripts to merge tables that I generate using .js inside Dataform (something like here https:/...
How to use Javascript function in SQLX query
Good day, im having trouble in understanding how to pass arguments to js functions at my sqlx queries, for exa...
Dataform - Unable to connect to your Git provider
We are trying to connect the Repos (Azure DevOps) as the main repository of Datafrom (GCP) and we are encounte...
workflow executed successfully, but data is not inserted into destination table.
My work flow was working fine, but yesterday i update the corn timing.Though log and result shows data inserte...
Invoke Dataform single sqlx using Cloud workflows
Hi All,I'm invoking Dataform using Cloud workflows. I can invoke single sqlx file using tags(includedTags). Bu...
Unable to Execute Dataform Workflow due to "IAM permission denied for service account"
HelloI've been facing an issue trying to execute a workflow in Dataform. Every time I attempt to run it, I'm m...
Dataform and BI Engine
It seems that queries run from Dataform are not able to utilize the BI Engine.Is there a way to use the BI Eng...
Dataform Development
Hi All,I am trying to set up my teams' infrastructure in BQ and currently, we are looking to have 2 projects 1...
dataform: Add assertions as dependencies
Hi Experts, I am following Add assertions as dependencies. I have created A.sqlx: config { type: "table", asse...
Dataform: generic::invalid_argument: The remote repository closed connection during remote operation
Hello,I've scheduled Dataform actions using workflow configurations, however I'm getting the following error i...
How to properly delete data from incremental table
Hi, I have a dataform incremental table where I would need to delete the past 3 days of data from the table. H...
DATAFORM - Load a table with data from CSV files using Dataform
Hi,Following this conversation, I decided to put my case as a new message here. My goal is to load a table wit...
Dataform - Documentation Generation
Hi, Does any know how we can generate documentation i.e. our model's documentation? Like in DBT, we can genera...
Using GCP Artifact Registry for Dataform Private Packages
Hello,Can we use GCP Artifact registry for Dataform private packages.
Dataform workflow orchestration
I am new to dataform and i have to implement a working Dataform workflow which i have to trigger from Airflow....
Dataform workflow orchestration
I have set up workflow to run a specific tag in a branch as per the guide here, but I'm seeing that the workfl...
Dataform<>Airflow Run Using Tags
I am currently managing a dataform project with numerous transformations. These transformations are executed t...
Dataform write different project ID
Hi everyoneI am currently new at working with dataform in bigqueryI would like to know how I can make my sqlx ...
Dataform - additional learning
Dataform just recently went into General Availability. My team is looking to move our transformations away fro...
Pivot Macro for Dataform
Hello everyone,I am currently evaluating Dataform and wondering if it is possible to dynamically pivot data us...
Incremental Table w/ Changing columns
Hi there - I use dataform to build an incremental model ("requests_snapshot") that stores snapshots on a model...
Hello! Help me please. I have a deck-type view, where primary header - name, summary column - Code.
Hello! Help me please. I have a deck-type view, where primary header - name, summary column - Code. Code is a ...
Dataform Plugin for neovim
Hello guys!For data engineers that like to use neovim, I created this dataform plugin with the following funct...
Unable to connect ADO with google-dataform
I am getting below mentioned error while connecting my ADO repository with google dataform using. Even after p...
Dataform: Unable to run incremental workflow
I have the following SLQX and relative table already created in my dataset:config { type: "incremental", schem...