Answer this question in the comments to earn a special badge: In your journey, what's one overlooked data skil...
Would you like to influence Google Cloud’s BI roadmap more? I’d like to invite you to join the Google Cloud BI...
Hello, I am unclear about the Google-Managed Encryption Process.- How often does google rotate encryption keys...
I have a Flink job running in a dataproc cluster. It's exporting logs to Log Explorer but I don't think Log Ex...
custom/cpu/utilizationcustom/disk/percent_usedhow are the above cloud dataproc batch metric useful when settin...
I need help regarding creating Dataproc alerts.What all type of alert we can create for Dataproc.
Hi,I have a Dataproc cluster and using the web interface (Jupyterlab) inside the cluster, I am trying to read ...
Hi,I am currently converting some Alteryx workflows into pyspark and wanted to test the codes for debugging an...
I noticed this alert on the serverless batches page:"Starting March 31, 2024, special requirements apply to us...
Hi,I'm using Dataproc on VMs. Currently we are running in production two clusters: master VM typeworker VM typ...
Hi Folks,I'm trying to submit to submit a Dataproc serverless job using service account (have necessary permis...
Hi, I have a pipeline set up, it creates a cluster perform an ingestion and delete a cluster. In success logs ...
Hi,we have created a metastore and it has a database and table, post editing the metastore and enabling the Da...
When trying to use Dataproc image version 2.2. the gcloud CLI raises a validation error.Trying the same comman...
Hi,we're using hive tables with dataproc metastore synced to data Catalog, we could see that data catalog was ...
I have a dataproc cluster with livy installed and it is working.what is the command to run a 'hello world' scr...
is it possible to trigger a job in dataproc cluster from the compute engine vm?the sample 'hello world' job fi...
Hi!I have a Dataproc Single Node Cluster that is used as a Spark History Server. However I notice in Cloud Log...
Hi,While I'm trying to connect to hive-metastore(Dataproc Metastore) using thrift url within Spark Configurati...
Hi When we are using a dataproc cluster that needs a hive metastore,what benefits we have using the metastsore...
I got an issue happened since 3h30 UTC with Dataproc MetastoreAs I look into Logs Explorer, it happened after ...
HiI have successfuly deployed a job in airflow from google cloud composer.I'm trying now to do the same by pro...
I want to create a Spark Streaming application in DataProc that has a subscription to PubSub Lite or PubSub (I...
Im trying to build google data fusion pipeline to load data from MS SQL server to Big Query. The source (MS Sq...
when i am running the same pyspark job to read the cloud sql data using jdbc in pycharm on local machine where...
The cloud sql MSSQL which i created , i am able to connect it from sql server management studio on my laptop (...
Hello Hi realy would like to test a series of spark 2.4 jobs on dataproc serverless but there is only spak 3.x...
Dear Experts,we have configured two names space in the cloud data fusion, but from yesterday onwards not able ...
my quota is 4096 GB and its reached after I triggered the serverless spark job ( which failed) but its still s...
In Dataproc, we have metadata option to install python packages in a dataproc cluster created through Composer...
Hi,I have a dataproc serverless batch job using the 2.1 Runtime version.Looking to use the new autoscaling ver...
Hello community,I need to know who started a job in the dataproc cluster... I need to know if it was a service...