Beyond Perturbation Magnitude: Direction-Dependent Responses in Multimodal Geometric Representations

Key points

• This paper asks whether the response of a multimodal geometric score is determined primarily by the magnitude of the perturbation-induced displacement.1
• Using frozen cohorts from MSR-VTT (N=878) and DiDeMo (N=980), we apply controlled video blur and audio noise and analyze the response in the relational geometry on which the score is defined.1
• The closed-form first-order expansion of the Gramian volume yields the Directional Geometric Response (DGR): the projection of the displacement onto the local volume gradient, which jointly captures the clean operating point, displacement magnitude, and displacement direction.1
• DGR uses the observed degraded-state displacement and is therefore an explanatory quantity, not a deployment-time predictor: geometric response depends on where the representation operates, how far degradation moves the relational geometry, and in which direction it moves.1

Sources (1)

• [1]Beyond Perturbation Magnitude: Direction-Dependent Responses in Multimodal Geometric Representations
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 6, 03:25 PM

  Geometric alignment scores based on Gram determinants provide a compact way to model higher-order consistency among modalities, yet how such scores respond to modality degradation is poorly understood.

  This paper asks whether the response of a multimodal geometric score is determined primarily by the magnitude of the perturbation-induced displacement.

Extractive summary: sentences quoted from the sources.