Beyond Resolution: Object-to-Image Ratio Mismatch in Instance Retrieval

Key points

• Visual instance retrieval often fails when the same object appears at different apparent sizes in the query and gallery.1
• On a controlled benchmark of 3,021 Objaverse objects rendered at five camera distances, more than 80% of the cross-distance degradation is attributable to O2I mismatch rather than resolution for 9 of 12 pretrained backbones; multi-scale architectures cut the resolution-only effect to single digits yet remain equally susceptible.1
• The failure is also asymmetric: tight queries retrieve more reliably against wide gallery images than the reverse.1
• Guided by this analysis, query-side scale augmentation and an OWLv2 crop reranker reach state of the art on ILIAS 100M (29.2 mAP@1000 before reranking, 42.0 after) without training or modifying the precomputed gallery index, and a LoRA fine-tune matches the query-side gains at a single forward pass, showing that O2I robustness is learnable.1

Sources (1)

• [1]Beyond Resolution: Object-to-Image Ratio Mismatch in Instance Retrieval
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 8, 08:31 AM

  We show that the dominant cause is usually not resolution loss but object-to-image (O2I) ratio mismatch: the object occupies different fractions of the two images.

  Visual instance retrieval often fails when the same object appears at different apparent sizes in the query and gallery.

Extractive summary: sentences quoted from the sources.