Mistral Small 3 24B GGUF quantization Evaluation results
Please note that the purpose of this test is to check if the model's intelligence will be significantly affected at low quantization levels, rather than evaluating which gguf is the best.
Regarding Q6_K-lmstudio: This model was downloaded from the lmstudio hf repo and uploaded by bartowski. However, this one is a static quantization model, while others are dynamic quantization models from bartowski's own repo.
gguf: https://huggingface.co/bartowski/Mistral-Small-24B-Instruct-2501-GGUF
Backend: https://www.ollama.com/
evaluation tool: https://github.com/chigkim/Ollama-MMLU-Pro
evaluation config: https://pastebin.com/mqWZzxaH