Answer this question in the comments to earn a special badge: In your journey, what's one overlooked data skil...
Would you like to influence Google Cloud’s BI roadmap more? I’d like to invite you to join the Google Cloud BI...
Hello, I am using the managed version of Apache Airflow with Cloud Composer in version composer-2.6.1-airflow-...
Hi Folks,I have created a stored procedure in Postgres SQL named "source".add_missing_columns. I want to trigg...
In my pipeline, I have a step wherein I am doing some web scrapping with selenium. That is: using selenium to ...
Hi All,I am trying to set up my teams' infrastructure in BQ and currently, we are looking to have 2 projects 1...
I am currently managing a dataform project with numerous transformations. These transformations are executed t...
Hi Team,I'm trying to perform incremental load of data from AWS RDS (Entrata PostgreSQL) to Cloud SQL PostgreS...
Hello,After migration to Cloud Composer 2.6.0, we can see a password authentication has been added to the unde...
I have an s3 bucket (amazon) and I want to write a composer pipeline to upload csv files into the S3 bucket on...
Hello, so currently I want to create cloud composer in service project using shared VPC from host project.I al...
I want to do a live call transcription of a streaming audio. Is it possible in CCAI Agent assist?
Hey, guys,I need to know who ran a DAG on the GCP composer airflow(Version 1 and 2).Do you have any tips for k...
Title explains a lot.I am using Dataform and Composer. Both instances are very simple and I just used the quic...
I have a Cloud Composer environment running with the 1.20.11 Composer version and 2.3.4 Airflow, but I have so...
I am exploring options for copying/moving some files from SFTP to GCS, Google Cloud Storage.Does GCP for real ...
We are using Kubernetes and GCP for our client to setup the sGTM tagging. We are facing an console error issue...
In Dataproc, we have metadata option to install python packages in a dataproc cluster created through Composer...
Hello,I've created a few Composer environments through Terraform (same config, different projects). For the mo...
I am trying GKERunStartPodOperator to launch pods on existing GKE cluster user deferrable mode=true and do xco...
Please see my code for triggering Dataform. This is in my Airflow DAG and I am using the Composer Environment....
This is my Equivalent command line for Dataproc serverlessgcloud dataproc batches submit --project xyz2022 --r...
I have daily ingest pipeline as production runs on Cloud Composer, the code is unchanged for a while. One day ...
HiI have implement data transformation pipeline on Google Cloud Composer, by using1. "DataformCreateCompilatio...
My company has established a table in a BigQuery sandbox account to store data pertaining to various site even...
Hey Guys !!I want to trigger dataproc batch job (dataproc serverless job) via cloud composer(airflow) ...I am ...
Hi all,Recently we are preparing to upgrade our CC1-Airflow1 setup to CC2-Airflow2. For ease of development, w...
Hello there!Im currently working on this integration with Google Ads's offline conversions and was wondering w...
What backend does Cloud Composer use by default for xComs? Does it just use the database or does it use a cust...
w.r.t. this problem: We have an application that has a dependency DAG. To build the application we need to bui...
I am trying to schedule a data prep flow using composer. I have provided recipe_id/output_id and token_bearer....
Hello,I am using GCP Composer(airflow). What I want to do is to change Airflow, in all its components, to refl...