Get hands-on experience with 20+ free Google Cloud products and $300 in free credit for new customers.

What are Dataform's key features?

  • Dataform core: an open source, SQL-based framework to manage data transformations and configure new tables that can be adopted incrementally and additively, without modifying existing code.
  • Cloud development environment to develop data assets with SQL. Define tables, fix issues with real-time error messages, visualize dependencies, commit the changes to Git, and schedule pipelines in minutes, from a single interface, without leaving your web browser. Connect your repository with third-party providers such as GitHub and GitLab. Commit changes and push or open pull requests from the IDE. 
  • Fully managed, serverless orchestration for data pipelines embedded in GCP.
3 15 2,135
15 REPLIES 15

RC1
Bronze 4
Bronze 4

@josiehall 

When it will be publicly available or preview availablility?  Is this similar to Dbt ?

 

Hi!

Dataform will be available publicly in Preview by the end of August! We are working to integrate the product experience inside GCP and launch in the BigQuery UI. The Preview features include a framework to define tables and dependency, a SQL IDE integrated with Github and managed deployment.

Hi there,

1. Are there plans to have GCP Dataform connect to Cloud Source Repository or Bitbucket?

2. Will there be support for displaying dependency trees, Documentaion, or DAGs in the near future (The way it does in Standalone Dataform)?

Hi Alex,

1. We have Bitucket support on our roadmap for Q1 next year.
2. Yes! We plan to launch the DAG later this year.

Hi @josiehall I'm also interested in this Bitbucket topic. Do you have any ETA updates?

Thx in advance.
Best,
Felipe Tancredo

Hi, and with " Cloud Source Repository", that is the internal GCP GIT Repo?

Hi there, Cloud Source repositories aren't supported yet.

Hello Team, I was expecting this release for a while. However, I don't see the test functionality yet in GCP documentation. Do you have this feature in your roadmap?

https://docs.dataform.co/guides/tests

Hi Nicolas,
Yes this is on our roadmap for H2 next year.

Thank you Josie, I am starting to use Dataform and try to spread it in my team. Do you have like a roadmap documentation or a git repository where I could follow the releases?

I'd also like to know this!

We're planning to add release notes to the doc site (https://cloud.google.com/dataform/docs/) early next year. We're also sending a monthly email with updates and plans. You can register by filling this form 

Is there an updated ETA for using Bitbucket as the cloud git repository? Thanks.

Any plans for typescript support? If not, I can transpile.

Have you all been seeing strong adoption? I have some concerns that the platform may not be adopted, limiting feature updates over time.