This website uses Cookies. Click Accept to agree to our website's cookie use as described in our Privacy Policy. Click Preferences to customize your cookie settings.
We are running deepseek:70b model with SSD disk in g2-standard-96 instance but we get approximately 25 TPS in benchmark tests, which is impossible.What is the reason ?
Model: L4x8 Image: Deep Learning VM for PyTorch 2.4 with CUDA 12.4 M127
LinkedIn
Twitter
