Abstract
Research on vision-in-the-loop search is represented by distinct lines of work, including Query-by-visual-search: multimodal framework for content-based image retrieval, Multimodal clothing recognition for semantic search in unconstrained surveillance imagery, and Efficient Search in a Panoramic Image Database for Long-term Visual Localization. The references are mapped across visual acquisition, interactive exploration, and active visual search. Particular attention is paid to how evaluation protocols shape apparently conflicting conclusions and how those conclusions should be interpreted outside their original settings. The resulting evidence map identifies well-supported practices, unresolved tensions, and concrete priorities for comparative research. It emphasizes transparent assumptions, reproducible protocols, and evaluation measures that match the intended use.
References
Zhang, H., Zhou, J., Zhao, R., Shan, Y., Chen, J., Zhou, B., Li, B., Wang, F., Wu, J., Tao, Z., Mei, L., Yu, X., Liu, L., Chen, C., & Zhang, W. (2026). DeepVoyager-VL: Incentivizing vision-in-the-loop search for long-horizon multimodal agents. arXiv. https://doi.org/10.48550/arXiv.2608.01827
Bibi, R., Mehmood, Z., Yousaf, R.-M., Saba, T., Sardaraz, M., & Rehman, A. (2020). Query-by-visual-search: multimodal framework for content-based image retrieval. Journal of Ambient Intelligence and Humanized Computing, 11(11), 5629-5648. https://doi.org/10.1007/s12652-020-01923-1
Tang, Z., Long, Z., & Fu, X. (2023). Universal Multimodal Neural Machine Translation Via Image Retrieval from Search Engines. . https://doi.org/10.2139/ssrn.4566495
Soleymani, M., Riegler, M., & Halvorsen, P. (2017). Multimodal Analysis of Image Search Intent. Proceedings of the 2017 ACM on International Conference on Multimedia Retrieval, 251-259. https://doi.org/10.1145/3078971.3078995
Halstead, M.-A., Denman, S., Sridharan, S., Tian, Y., & Fookes, C. (2019). Multimodal clothing recognition for semantic search in unconstrained surveillance imagery. Journal of Visual Communication and Image Representation, 58, 439-452. https://doi.org/10.1016/j.jvcir.2018.12.001
Strong, G., Hoeber, O., & Gong, M. (2010). Visual Image Browsing and Exploration (Vibe): User Evaluations of Image Search Tasks. Lecture Notes in Computer Science, 424-435. https://doi.org/10.1007/978-3-642-15470-6_44
He, R., Long, S., Sun, W., & Liu, H. (2024). A Multimodal Image Registration Method for UAV Visual Navigation Based on Feature Fusion and Transformers. Drones, 8(11), 651. https://doi.org/10.3390/drones8110651
Orhan, S., & Bastanlar, Y. (2021). Efficient Search in a Panoramic Image Database for Long-term Visual Localization. 2021 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW), 1727-1734. https://doi.org/10.1109/iccvw54120.2021.00198
Hamroun, M., Lajmi, S., Nicolas, H., & Amous, I. (2018). ISE: Interactive Image Search using Visual Content. Proceedings of the 20th International Conference on Enterprise Information Systems, 253-261. https://doi.org/10.5220/0006806702530261
Motter, B.-C., & Simoni, D.-A. (2007). The roles of cortical image separation and size in active visual search performance. Journal of Vision, 7(2), 6. https://doi.org/10.1167/7.2.6
