UnslothDynamic v3.0is the next iteration of our Dynamic quantization and a major improvement over Dynamic v2.0.
Also a huge thanks to all your support! We saw over 5.1 million Unsloth Qwen3.8 downloads in just 5 days!
Our new methodology composes of many new features and improvements. We now use a much higher-quality imatrix calibration dataset from diverse sources. The dataset is refined foragentic coding, chat, and multilingual performance. We also improvedlayer selectionand introduced many more quantization techniques to preserve as much model quality as possible.
Wedo not train on the imatrix calibration dataset, and we do NOT useQATorQAD. Everything is done throughpost-training quantization. Our imatrix file used is available for the community to test, evaluate, and use. We encourage researchers and developers to create variations and fine-tunes of Qwen3.8 using our Unsloth quants/imatrix. You can read ouroverfitting analysisas well.
We generally report top-1% accuracy like how for Kimi-K3 “Dynamic 1-bit reaches~78.9%top-1 accuracy while being62% smaller.” However top-1% is an argmax on 1 prediction, so it’s not really effective on gauging actual inference.
We created a dataset of 300 held out examples (NOT in calibration dataset) from Terminal-Bench 2.1 + DeepSWE + Harbor + MathArena 2025-26 + non-Latin/long-doc prompts and we did greedy argmax decoding for 32 tokens for BF16 vs all quants and providers. Seeoverfitting analysisfor more details on overfitting.
This allows us to gauge if there is overfitting and if quant outputs are similar to BF16’s trajectories over multiple tokens. This is a better metric than top-1% accuracy since we extend KLD top-1% to more like KLD top-1% at 32 tokens.
We ran KLD benchmarks for all providers as well and report Top-1% and KLD mean. At all levels especially on the smaller quant sizes, Unsloth UD-3 quants get up to +10% extra top-1% accuracy at the same disk space!
All plots remove the MTP head from the x axis when calculating disk space to provide a fair comparison to everyone.
When comparing to our older UD-2 on unseen Wikitext and Code, we show great improvement on KLD – the bigger ones not so much, so we still use our old UD-2 for the larger quants – we plan to experiment and improve them as well!
We also control for overfitting by using totally different datasets for calibration and remove all leakages as much as possible. We test KLD on these unseen datasets, and also we do NOT do QAD / QAT, just pure PTQ so overfitting is less of a concern vs other QAD / QAT approaches.
We’re introducingUnslothDynamic v2.0 quantization – a major upgrade to our previous quants. This new method outperforms leading quantization methods and sets new benchmarks forAider Polyglot, 5-shot MMLU and KL Divergence.