Keywords
vision-in-the-loop
active visual acquisition
long-horizon search
simulation
Abstract
Three strands anchor this examination of vision-in-the-loop search: Multimodal clothing recognition for semantic search in unconstrained surveillance imagery; Efficient Search in a Panoramic Image Database for Long-term Visual Localization; and DeepVoyager-VL: Incentivizing vision-in-the-loop search for long-horizon multimodal agents. The synthesis centers on semantic image search, content-based retrieval, and evaluation protocols, separating recurring design principles from application-specific choices. It also considers what information is required for an independent reader to reproduce or audit the findings. Taken together, the references support a conditional design framework rather than a universal method ranking. The proposed agenda calls for explicit failure analysis, comparable baselines, and reporting that makes scope and uncertainty visible.
References
Zhang, H., Zhou, J., Zhao, R., Shan, Y., Chen, J., Zhou, B., Li, B., Wang, F., Wu, J., Tao, Z., Mei, L., Yu, X., Liu, L., Chen, C., & Zhang, W. (2026). DeepVoyager-VL: Incentivizing vision-in-the-loop search for long-horizon multimodal agents. arXiv. https://doi.org/10.48550/arXiv.2608.01827
Bibi, R., Mehmood, Z., Yousaf, R.-M., Saba, T., Sardaraz, M., & Rehman, A. (2020). Query-by-visual-search: multimodal framework for content-based image retrieval. Journal of Ambient Intelligence and Humanized Computing, 11(11), 5629-5648. https://doi.org/10.1007/s12652-020-01923-1
Tang, Z., Long, Z., & Fu, X. (2023). Universal Multimodal Neural Machine Translation Via Image Retrieval from Search Engines. . https://doi.org/10.2139/ssrn.4566495
Soleymani, M., Riegler, M., & Halvorsen, P. (2017). Multimodal Analysis of Image Search Intent. Proceedings of the 2017 ACM on International Conference on Multimedia Retrieval, 251-259. https://doi.org/10.1145/3078971.3078995
Halstead, M.-A., Denman, S., Sridharan, S., Tian, Y., & Fookes, C. (2019). Multimodal clothing recognition for semantic search in unconstrained surveillance imagery. Journal of Visual Communication and Image Representation, 58, 439-452. https://doi.org/10.1016/j.jvcir.2018.12.001
Strong, G., Hoeber, O., & Gong, M. (2010). Visual Image Browsing and Exploration (Vibe): User Evaluations of Image Search Tasks. Lecture Notes in Computer Science, 424-435. https://doi.org/10.1007/978-3-642-15470-6_44
He, R., Long, S., Sun, W., & Liu, H. (2024). A Multimodal Image Registration Method for UAV Visual Navigation Based on Feature Fusion and Transformers. Drones, 8(11), 651. https://doi.org/10.3390/drones8110651
Orhan, S., & Bastanlar, Y. (2021). Efficient Search in a Panoramic Image Database for Long-term Visual Localization. 2021 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW), 1727-1734. https://doi.org/10.1109/iccvw54120.2021.00198
Hamroun, M., Lajmi, S., Nicolas, H., & Amous, I. (2018). ISE: Interactive Image Search using Visual Content. Proceedings of the 20th International Conference on Enterprise Information Systems, 253-261. https://doi.org/10.5220/0006806702530261
Motter, B.-C., & Simoni, D.-A. (2007). The roles of cortical image separation and size in active visual search performance. Journal of Vision, 7(2), 6. https://doi.org/10.1167/7.2.6