#quantization
11 articles
Latest
Five Papers Attack KV Cache Bottleneck With Quantization Methods
New approaches use statistical inference, rate-distortion theory, and learned eviction to reduce memory cost of long-context LLM inference.
Three Papers Advance Quantization Methods for Efficient LLM Deployment
Researchers tackle post-training quantization bottlenecks that distort model behavior under memory and latency constraints.
MiniMax M2.7 NVFP4 Hits New Speeds on Dual Blackwell GPUs
Quantized model achieves production-ready inference on professional-grade hardware.
MiniMax M2.7 Quantized Models Enable Local Deployment
Unsloth and community developers release multiple GGUF quantizations of MiniMax M2.7, making the model viable for consumer hardware.
LocalLLaMA Community Tackles Real-World AI on Tight Hardware
Reddit's open-source AI community solves practical problems with limited compute resources.
Users Ditch ChatGPT and Claude Over Rate Limits, Go Local
Workers hitting API caps mid-task are building free alternatives with open-source models.
APEX MoE and TurboQuant Deliver 33% Faster LLM Inference
New quantization techniques accelerate both inference and prompt processing for local model deployment.
Alibaba MNN Adds TurboQuant Support for Local LLM Inference
The framework now supports aggressive KV-cache compression, making on-device models faster to run.
TurboQuant Compresses LLM KV Cache to 3-4 Bits Without Accuracy Loss
New quantization algorithm enables longer context windows and 3.2× memory savings for local inference.
Google's TurboQuant Slashes LLM Memory Usage by 6x
New compression algorithm maintains output quality while dramatically reducing computational demands.
Qwen3.5 122B Outperforms Smaller Coder Next Model
A developer's switch to a larger model reveals counterintuitive gains in speed and output quality.