Fashion Retrieval via Graph Reasoning Networks on a Similarity Pyramid

Abstract

Matching clothing images from customers and online shopping stores has rich applications in E-commerce. Existing algorithms mostly encode an image as a global feature vector and perform retrieval via global representation matching. However, discriminative local information on clothes is submerged in this global representation, resulting in sub-optimal performance. To address this issue, we propose a novel Graph Reasoning Network (GRNet) on a Similarity Pyramid, which learns similarities between a query and a gallery cloth by using both initially pairwise multi-scale feature representations and matching propagation for unaligned ones. The query local representations at each scale are aligned with those of the gallery via a novel adaptive window pooling module. The similarity pyramid is represented by a Graph of similarity, where nodes represent similarities between clothing components at different scales, and the final matching score is obtained by message passing along edges. In GRNet, graph reasoning is solved by training a graph convolutional network, enabling to align salient clothing components to improve clothing retrieval. To facilitate future researches, we introduce a new benchmark FindFashion, containing rich annotations of bounding boxes, views, occlusions, and cropping. Extensive experiments show GRNet obtains new state-of-the-art results on three challenging benchmarks and all settings on FindFashion.