Read this because This method solves a critical memory bottleneck in looped architectures, allowing for longer contexts and higher throughput on constrained hardware by reusing key-value caches.
Researchers propose BeNice! to enable KV-cache sharing in looped models, reducing memory usage and improving inference throughput without quality loss.
Read this because Optimizes long-context inference by reducing memory bandwidth bottlenecks through novel quantization techniques.
WUSH-KV reduces KV cache memory costs using data-adaptive transforms, achieving low perplexity at 2-bit quantization.
Read this because Developers can improve model alignment and reward scores without expensive fine-tuning or parameter updates, preserving original model capabilities.
New MISVO method steers frozen language models via pre-logit vectors, balancing reward optimization with generation quality using local KL geometry.
Read this because Researchers and engineers working with diffusion-based language models can significantly reduce inference latency and memory overhead by adopting this training-free acceleration framework.
Flash-dLLM introduces an I/O-aware fused KV-cache kernel and parallel decoding strategy to accelerate diffusion LLMs without auxiliary models.
Read this because This compiler approach eliminates kernel launch overhead, making GPU CFR viable for large-scale games where CPUs previously dominated.
GPU-CFR compiles game trees to static dataflow, achieving up to 80x speedup over prior GPU implementations and beating CPU baselines.