SSR: Sparse Segment Reduction for Ternary GEMM Acceleration
In this paper, we introduce Sparse Segment Reduction (SSR), a ternary matrix multiplication method designed to accelerate the inference of ternary LLMs and general Ternary Weight Networks (TWNs).
ProofPaper ↗
Key points
- Large Language Models (LLMs) require substantial computational resources, limiting their deployment on resource-constrained hardware.
- Ternary LLMs mitigate these demands through weight quantization via ternary values, achieving significant compression often with 50-90% sparsity.
- However, existing approaches have limitations: methods optimized for ternary weights, such as BitNet, redundant segment reduction (RSR), and its improved version RSR++, do not exploit sparsity structures, while conventional sparse formats neglect ternary characteristics, foregoing dual optimization opportunities.
- Evaluation results show that SSR achieves 2.1-11.3x speedup over RSR++ on ternary GEMM with 45-95% sparsity.
Sources (1)
- [1]SSR: Sparse Segment Reduction for Ternary GEMM AccelerationarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 6, 02:15 PM
In this paper, we introduce Sparse Segment Reduction (SSR), a ternary matrix multiplication method designed to accelerate the inference of ternary LLMs and general Ternary Weight Networks (TWNs).
Large Language Models (LLMs) require substantial computational resources, limiting their deployment on resource-constrained hardware.
Extractive summary: sentences quoted from the sources.
Before this
- Sep 22, 2026vllm-project/vllm v0.30.0
- Aug 10, 2026vllm-project/vllm v0.27.0
- Jul 11, 2026vllm-project/vllm v0.25.0
- Jun 29, 2026vllm-project/vllm v0.24.0
- Jun 15, 2026vllm-project/vllm v0.23.0
- Jun 10, 2026DiffusionGemma: 4x faster text generation