I am trying to load HF models however the workbench kernel crashes while verifying shards
The instance is using A100-40GB ( machine type: Accelerator Optimized: 1 NVIDIA Tesla A100 GPU, 12 vCPUs, 85GB RAM )
model_id = "mistralai/Mixtral-8x7B-v0.1"
model = AutoModelForCausalLM.from_pretrained(model_id)
The error message says -- Kernel died unexpectedly .
Every time it is crashing in "Loading checkpoint shards (42%) ... 8/19"
This happens with all models that I tried like Falcon-40b, mixtral-7b
Will appreciate some pointers.
LinkedIn
Twitter
