Pinview: Implicit Feedback in Content-Based Image Retrieval

Abstract

This paper describes Pinview, a contentbased image retrieval system that exploits implicit relevance feedback during a search session. The goal is to retrieve interesting images and the relevance feedback could be eye movements or clicks on the images. Pinview contains several novel methods that infer the intent of the user. From relevance feedback and visual features of images Pinview learns a similarity metric between images which depends on the current interests of the user. It then retrieves images with a specialized reinforcement learning algorithm that balances the tradeoff between exploring new images and exploiting the already inferred interests of the user. In practise, Pinview is integrated to the content-based image retrieval system PicSOM, so it is possible to apply it to realworld image databases. Preliminary experiments show that eye movements provide a rich input modality from which it is possible to learn the interests of the user.