Tags · #quantization

WUSH-KV: Data-Adaptive Transforms for Low-Bit KV Cache Quantization

Read this because Optimizes long-context inference by reducing memory bandwidth bottlenecks through novel quantization techniques.

WUSH-KV reduces KV cache memory costs using data-adaptive transforms, achieving low perplexity at 2-bit quantization.

Read this because Builders should care because this method recovers capabilities usually lost during compression and quantization.

A new healing technique allows a compressed 4-bit model to outperform its full-precision original on seven benchmarks.