The fact
GPU capacity enables running 70-billion-parameter models locally without significant degradation.
Cost-benefit becomes favorable above 70 billion parameters for quantized models.
Click the link to read an article on the topic:
GPU capacity enables running 70-billion-parameter models locally without significant degradation.
Cost-benefit becomes favorable above 70 billion parameters for quantized models.