All posts

Batching Decode Attention Across Requests Got Us 3.83x Real Throughput — Across Layers Got Us Nothing
Batching Decode Attention Across Layers Sounded Great — Until the Residual Stream Said No
Chasing the Mac-vs-CUDA Prefill Gap — and Finding a Wall Instead
Fusing Quantize and Pack Into One Metal Dispatch
The ROM Chip That Wasn't, and the 230× Speedup That Was
The Sign Was the Whole Paper: Debugging a KV Cache Compressor on Real Models
A 5.65× Metal Kernel
TensorOps Research: What We Learned Optimizing KV Caches
KIVI: The Most-Cited KV Cache Baseline, Implemented in MLX
Hands-On: Compressing Your First LLM with VeloxQuant-MLX
I Wrote a Metal Kernel to Stop My Mac From OOMing
TurboQuant + Metal Kernels: The Combined Writeup
Benchmark Results: 10 Models, 8 Compression Configs
I Ported a Google Research Paper to Apple Silicon
VeloxQuant-MLX: Fast KV Cache Quantization for Apple Silicon

Authors

Docs

Community

More

Rajveer Rathod