Need to create a Kubeflow pipeline for ML use-cases on GKE cluster, currently working on recommendation. Have made the Vertex AI pipeline for the same but vertex being serverless takes time to make the containers up and then run the pipeline. We needed a platform where the kubeflow pipelines can be executed faster - especially during development the results are needed instantly so that we can change inputs or tune the model.
After uploading the yaml file, creating a run for the pipeline. After the first component the file read fails to write on the given bucket with below error.
FileNotFoundError: [Errno 2] No such file or directory: '/gcs/bucket/reco_v2/637705bf-381e-40fa-8597-91089e700aaf/pipeline/reco_v2/637705bf-381e-40fa-8597-91089e700aaf/get-dataframe/df_path.csv'39F0203 13:57:24.664574 18 main.go:50] Failed to execute component: exit status 140time="2023-02-03T13:57:24.672Z" level=error msg="cannot save artifact /tmp/outputs/test_df_path/data" argo=true error="stat /tmp/outputs/test_df_path/data: no such file or directory"41time="2023-02-03T13:57:24.672Z" level=error msg="cannot save artifact /tmp/outputs/train_df_path/data" argo=true error="stat /tmp/outputs/train_df_path/data: no such file or directory"42Error: exit status 1
Tried running the sample tutorial kubeflow pipelines, they are running fine and does not throw any such file write error.
We have created a cluster and integrated Kubeflow pipelines from the GCP marketplace -> https://console.cloud.google.com/marketplace/details/google-cloud-ai-platform/kubeflow-pipelines
Need to know if there is a version issue or code is not proper or anything else. Have tried running a kfp v2 code too(from tutorials online) that worked well too.
The kubeflow pipeline should have worked and saved outputs to the bucket given.
Please help
LinkedIn
Twitter
