We have a notebook A.ipynb which imports a module B.py.
Running A.ipynb from the Vertex AI Workbench works fine - however when we submit A.ipynb to Executor, it only copies the A.ipynb file without the dependency.
We ended up copying and pasting the contents of the supporting file into the notebook, but it's not very good code reuse. It's probably possible to use gsutil to manually copy files up to a bucket and then download them when the script executes, however those copied files won't change when the original file changes in the git repository. So really it would be more ideal for the depenancy to be able to be specified when the file is submitted to the executor.
Is there some way to enable this? What do other people do who have files that are shared dependencies between notebooks?
LinkedIn
Twitter
