Data Fusion - Preventing Duplicated from sinking t... - Page 2

This website uses Cookies. Click Accept to agree to our website's cookie use as described in our Privacy Policy. Click Preferences to customize your cookie settings.

Reject

Preferences

Google Cloud
Google Workspace
AppSheet
Looker & Looker Studio
Google Cloud Security

Google Cloud Home
Cloud Forums
Groups
- Cloud FinOps and Cost Optimization Community
Learning & Certification Hub
Articles & Information
Community Resources
Cloud Events

cancel

Turn on suggestions

Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.

Showing results for

Search instead for

Did you mean:

Get hands-on experience with 20+ free Google Cloud products and $300 in free credit for new customers.

Google Cloud
Cloud Forums
Data Analytics
Data Fusion - Preventing Duplicated from sinking t...

Topic Options

Subscribe to RSS Feed
Mark Topic as New
Mark Topic as Read
Float this Topic for Current User
Bookmark
Subscribe
Mute
Printer Friendly Page

Solved

Data Fusion - Preventing Duplicated from sinking to Bigquery & Denormalise

Posted on 07-21-2022 03:36 AM

Share this topic

Twitter

yrstruly

Bronze 1

Post Options

Mark as New
Bookmark
Subscribe
Mute
Subscribe to RSS Feed
Permalink
Print
Report Inappropriate Content

Reply posted on --/--/---- --:-- AM

Post Options

Mark as New
Bookmark
Subscribe
Mute
Subscribe to RSS Feed
Permalink
Print
Report Inappropriate Content

Please assist?

I would like to prevent duplicates entering into my Google Bigquery db. I have used the Distinct plugin and noticed that my final data is more than 50% less as my original data. Can i trust this plugin(see screen shots)?

In this ETL Flow, i would like to denormalise some tables to make it more compact in Bigquery. I would like to use the Joiner plugin, see attached. The idea is to join 2 or more tables in the ETL flow and have one table written in Bigquery for it. Please advise?