#inference
6 published articles
DeepSeek's 17 September V4.1-Flash report cuts global KV cache to 890 bytes per token and leaves the quality cost unmeasured
The 51-page arXiv paper pairs FP4 KV caching, cross-layer cache reuse and an encoder-decoder split. It reports no serving speed and no ablation for its two riskiest shortcuts.
MLPerf Inference v6.1 preview table for Vera Rubin NVL72 vs GB300 NVL72
This article covers NVIDIA's Vera Rubin NVL72 preview claims in the 16 September 2026 MLPerf Inference v6.1 round and the supporting MLCommons artifacts.
Jalapeño first silicon: OpenAI's inference ASIC claims 1.5-1.9x work/watt and a nine-month tapeout
OpenAI's first-results post (25 August 2026) and Broadcom unveil put measured first-party silicon on the record. This Stack piece inventories the architecture claims, maturity, and pick-vs against merchant GPUs.
Vera Rubin NVL72's first MLPerf Inference v6.1 numbers: up to 3.7x vs GB300 on Qwen3-VL
NVIDIA's 16 September 2026 preview submission puts Vera Rubin NVL72 on DeepSeek-R1 and Qwen3-VL. This article covers the benchmark results and software stack, separate from earlier Vera Rubin production news.
SGLang ships day-zero MoE kernels faster than most teams pick a runtime
Apache-2.0 serving framework. v0.5.14 landed 26 June 2026 with DeepSeek-V4 and Waterfill/LPLB load balancing.
DeepSeek-V4 paper reports 27% FLOPs and 10% KV cache at 1M tokens versus V3.2
arXiv:2606.19348 (April 26, 2026); MIT checkpoints on HuggingFace; CSA/HCA hybrid attention.