Time Delay System for Sequence Modeling
Existing sequence models share a critical limitation: they lack an explicit, interpretable mechanism for content-aware temporal recall.
ProofPaper ↗
Key points
- LSTMs compress everything into a single hidden state, thereby losing information about specific past events.
- Transformers attend to all tokens indiscriminately at quadratic cost, while state-space models like Mamba compress history into a fixed-dimensional state without explicit content-addressable memory.
- To address this gap, we introduce a novel framework grounded in nonlinear time-delay systems, in which the effective delay is dynamically estimated based on the semantic similarity between current and historical inputs.
- Experimental results on text classification and generation outperform state-of-the-art methods.
Sources (1)
- [1]Time Delay System for Sequence ModelingarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 9, 01:43 AM
Existing sequence models share a critical limitation: they lack an explicit, interpretable mechanism for content-aware temporal recall.
LSTMs compress everything into a single hidden state, thereby losing information about specific past events.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 9, 2026Learning Representations to Guide Pixel Diffusion Transformers
- Oct 8, 2026Bridging Human and Model Attention: Explainability Analysis of CNN, Mamba, and ViT Architectures with Gaze-Based Validation
- Oct 8, 2026LEGO: A Lifting-Free Approach for Exocentric-to-Egocentric Video Generation
- Oct 8, 2026One Block, Multiple Depths: Recurrent Vision Transformers with Depth-Programmed Experts
- Oct 8, 2026Scaling to Tens of Thousands of Test-Time Iterations with Loop-Native Attention Residuals
- Oct 7, 2026HAN-Mamba: Hierarchical Selective State Space Networks for Multi-Scale Financial Volatility Forecasting