sgl-project/sglang gateway-v0.3.1: Release Gateway-v0.3.1
We're excited to announce SMG v0.3.1 – a game-changing release with 10-12x performance improvement and 99% memory reduction in cache-aware routing, plus enterprise-grade security!
Key points
- 🌲 Radix Tree / Cache-Aware Routing: 10-12x Faster + 99% Less Memory ⚡
- Our cache-aware routing can now handle over 216,000 cache insertions per second (up from 18,900), with latency dropping from 52.9 microseconds to just 4.6 microseconds per operation.
- Before: ~180 KB per node (DashMap default config on 170-core machines)
- Production-grade security for control plane APIs with native support for industry-standard OIDC providers: Google, Azure, Oracle, GitHub, and more.
Sources (1)
- [1]sgl-project/sglang gateway-v0.3.1: Release Gateway-v0.3.1GitHub: sgl-project/sglang · Jan 9, 06:18 AM
We're excited to announce SMG v0.3.1 – a game-changing release with 10-12x performance improvement and 99% memory reduction in cache-aware routing, plus enterprise-grade security!
## 🌲 Radix Tree / Cache-Aware Routing: 10-12x Faster + 99% Less Memory ⚡
Extractive summary: sentences quoted from the sources.
Before this
- Jan 1, 2026sgl-project/sglang v0.5.7
- Oct 15, 2025pytorch/pytorch v2.9.0: 2.9 Release Notes