About crazyrevol

crazyrevol · 04-07-2025

I have below script where I create or replace tables in my pipeline. I add PK and FK constraint after the table are created. How can I make the PK FK statements idempotent? I can't seem to add IF NOT EXISTS as indicated in the docs. CREATE OR REPLACE...

crazyrevol · 04-01-2025

We're using on-demand bigQuery model exclusively for all our projects. my observation is that in on-demand model, as long as we're below 2000 slots per project, we don't see any wait times in stages of jobs running in our project. So, I would conside...

crazyrevol · 06-20-2024

Hi, I tried to do some tests and its not very clear to me. So, wanted to ask here as a newbie to BigQuery.1. If I use a filter on partitioned date column(monthly partition), and I want to apply a function like below, will Big Query be able to do part...

crazyrevol · 06-16-2024

I'm trying to find a way to find references to our warehouse views. I see that when a view is referenced in a query, the referenced_tables columns in JOBS view shows the underlying tables but not the views that are actually referenced in the FROM cla...

crazyrevol · 06-12-2024

In our BigQuery environment, we have reporting tables that are mostly OBT tables. But they are not partitioned and clustered well. Most were created by analysts who didn't consider forecasted usage to identify good partitioning and clustering scheme....

crazyrevol · 04-17-2025

Hello @MarvinLlamas Thank you for your reply! The monitor Performance link you provided above, is not for BigQuery as fas as I understand. Is there is UI monitoring way I can set alerts if any bigquery job is taking longer than say twice or 10 times ...

crazyrevol · 04-17-2025

got it! Thank you for your reply @ibaui We are using this in pipelines where we "create or replace table" to create data assets in our gold layer yesterday. The pipelines do not have access to INFORMATION_SCHEMA. some kind of security think imposed b...

crazyrevol · 06-29-2024

Thank you for your reply @caryna . Clarifying question with regards to your answer to 1 and 2. 1. you said I can use EXTRACT function on partition column. My question was NOT if I can use the function(or any of the functions here) but "Does usage of ...

crazyrevol · 04-12-2024

@ms4446 wrote:WHEN NOT MATCHED BY SOURCE THEN DELETEclarifying question on this clause - If it doesn't find in the below part of source records, will it delete ? I would not want that. If I don't get any cdc records for an ID, that doesn't mean delet...

crazyrevol · 04-11-2024

Thank you for your reply @ms4446 .1. Incremental Materialization with MERGE: This option is very clear to me. I have implemented it. The CDC table has real-time updates from cdc extraction tool. I have partitioned the CDC table on extraction_ts colum...

My Stats

Sohail_El's Bio

Badges crazyrevol Earned

Recent Activity

PK and FK syntax to make my script idempotent

bigquery - on-demand vs enterprise edition auto-scaler

Question about filtering/billable bytes using date partition column with Function applied

INFORMATION_SCHEMA.JOBS view does not show referenced_tables for views

Identify most used columns to create partitions and clusters in existing BigQuery workloads

Re: bigquery - on-demand vs enterprise edition auto-scaler

Re: PK and FK syntax to make my script idempotent

Re: Question about filtering/billable bytes using date partition column with Function applied

Re: creating latest snapshot from cdc data of a source data table

Re: creating latest snapshot from cdc data of a source data table