Agents That Own Their Inference — Du'an Lightfoot & Khaja Omer, Akamai Technologies
Speculative decoding drops a demo model from roughly 58 tokens per second to 16 instead of speeding it up.
Proof1 independent outlet
Key points
- Du'an Lightfoot and Khaja Omer use that failed optimization to make a practical point: a faster inference configuration has to be measured against the actual model, hardware and workload.
- Qwen provides the example model for exploring how an agent's prompts, responses and repeated tool calls consume latency and memory.
- 1:51 - Agent loops and inference ownership
- 16:26 - Model precision and the first request
Sources (1)
- [1]Agents That Own Their Inference — Du'an Lightfoot & Khaja Omer, Akamai TechnologiesAI Engineer (YouTube) · Oct 11, 02:00 PM
Speculative decoding drops a demo model from roughly 58 tokens per second to 16 instead of speeding it up.
Du'an Lightfoot and Khaja Omer use that failed optimization to make a practical point: a faster inference configuration has to be measured against the actual model, hardware and workload.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 11, 2026UPDATE: Qwen 3.8 27B 140 tok/s on single RTX 3090 Megakernel: KL divergence 0.0009 vs llama.cpp
- Oct 10, 202648Gb VRAM speed AND quality ! (Qwen 3.8 27B Swift 1.5 W8A16)
- Oct 10, 2026Strata with Qwen3.8 Flash Next UD-Q4_K_XL
- Oct 9, 2026Microsoft's Decision-1 model enters the fast-growing AI decision model race
- Oct 9, 2026Qwen/Qwen-Image-2.1-Turbo
- Oct 8, 2026ConwayResearch/Underdog-Saluki-27B-1.0