We present AMES (Approximate Multimodal Enterprise Search), a unified multimodal late interaction retrieval architecture which is backend agnostic. AMES demonstrates that fine-grained multimodal late interaction retrieval can be deployed within a production grade enterprise search engine without architectural redesign. Text tokens, image patches, and video frames are embedded into a shared representation space using multi-vector encoders, enabling cross-modal retrieval without modality specific retrieval logic. AMES employs a two-stage pipeline: parallel token level ANN search with per document Top-M MaxSim approximation, followed by accelerator optimized Exact MaxSim re-ranking. Experiments on the ViDoRe V3 benchmark show that AMES achieves competitive ranking performance within a scalable, production ready Solr based system.
@article{arxiv.2603.13537,
title = {AMES: Approximate Multi-modal Enterprise Search via Late Interaction Retrieval},
author = {Tony Joseph and Carlos Pareja and David Lopes Pegna and Abhishek Singh},
journal= {arXiv preprint arXiv:2603.13537},
year = {2026}
}