Hi,
I have a dataform incremental table where I would need to delete the past 3 days of data from the table. However, it seems like the `delete from ${self()} where event_date >= event_date_checkpoint` is scanning the entire table (as it shows the script will process 930.32 GB). If I run the delete script on BQ UI, it will be only 0 byte.
This will be very costly when running the incremental table everyday if the delete function itself is processing 900 GB. May I know if the suggestion of process byte is a correct value? If yes, how do I optimize the delete function to only process the specific partition date
Any guidance or suggestions would be greatly appreciated!