VEDJE: Video-Efficient Discriminative Joint Encoder for Scalable Video-Text Retrieval

Key points

• Finding the right video often requires distinguishing similar scenes in which different events occur.1
• VEDJE compresses features within sampled frames while keeping their representations separate in a reusable cache.1
• Feature-change prediction supplies an auxiliary training signal that improves retrieval from the compressed cache without adding work at query time.1
• On MSR-VTT, MSVD, DiDeMo, and ActivityNet, VEDJE improves R@1 over matched first-stage retrievers in both retrieval directions.1

Sources (1)

• [1]VEDJE: Video-Efficient Discriminative Joint Encoder for Scalable Video-Text Retrieval
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 8, 12:37 PM

  Joint matching improves retrieval, but processing rich video representations for each query is costly.

  Finding the right video often requires distinguishing similar scenes in which different events occur.

Extractive summary: sentences quoted from the sources.