کنفرانس مهندسی برق ایران

صفحه اصلی / سی و دومین کنفرانس بین المللی مهندسی برق

A Transformer-Based Model for Similar Fashion Image Retrieval with Image and Text Features

نویسندگان :

Zahra Sheykhvand¹ Milad Farzalizadeh² Majid Meghdadi³

1- دانشگاه زنجان 2- دانشگاه زنجان 3- دانشگاه زنجان

کلمات کلیدی :

Fashion Image Retrieval،Transformer،Multimodal Fusion،Image Similarity،Computer Vision

چکیده :

Similar fashion retrieval system has various applications in online shops and image-based recommender systems. Online shops employ textual metadata of products to search for products by customers. However, customer satisfaction is not fully guaranteed due to limitations caused by inaccuracies in input metadata and incorrect product categorization. These issues lead to confusion and hinder the attainment of desired products. This system significantly enhances user satisfaction by expediting image searches and finding desired products. This paper proposes a Transformer-based architecture specifically designed for searching and retrieving similar images in the fashion domain that uses both visual and descriptive product features for more efficient retrieval. In this architecture, the features extracting and image and text vector embeddings are crucial for establishing similarity. Therefore, DeiT, BLIP, and BERT transformer models have been employed. Since previous research focused solely on image features to determine similarity, this paper incorporates textual features as additional product descriptions to achieve the most accurate matches. The Evaluation of the proposed architecture on the DeepFashion data set demonstrates a remarkable 41.5% improvement in recall compared to the baseline paper and several previous research.