ALIVE: Interaction-Aligned Object Insertion for First-Frame-Guided Video Editing

Key points

• Current video editors can insert objects but often struggle to make them participate in interactions such as being picked up or manipulated.1
• We curate 35,800 editing pairs combining 3D-rendered, model-generated, and real-world videos with general editing pairs from ROSE.1
• We further train a vision-language model (VLM) to predict interaction guidance from the same inputs.1
• We introduce the ALIVE-interaction benchmark to assess interaction fidelity, source preservation, and visual coherence using a unified VLM-based protocol, and evaluate on the general video object insertion benchmark.1

Sources (1)

• [1]ALIVE: Interaction-Aligned Object Insertion for First-Frame-Guided Video Editing
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 6, 05:58 PM

  We introduce ALIVE, a framework that makes inserted objects "alive" through coherent interactions with the source video's contents, using an edited first frame and an instruction naming only the added object.

  Current video editors can insert objects but often struggle to make them participate in interactions such as being picked up or manipulated.

Extractive summary: sentences quoted from the sources.