Can Eye Movement Improve Prediction Performance on Human Emotions Toward Images Classification?

Abstract

Recently, image sentiment analysis has become more and more attractive to many researchers due to an increasing number of applications developed to understand images e.g. image retrieval systems and social networks. Many studies aim to improve the performance of the classifier by many approaches. This work aims to predict the emotional response of a person who is exposed to images. The prediction model makes use of eye movement data captured while users are looking at images to enhance the prediction performance. An image can stimulate different emotions in different users depending on where and how their eyes move on the image. Two image datasets were used, i.e. abstract images and images with context information, by using leave-one-user-out and leave-one-image-out cross-validation techniques. It was found that eye movement data is useful and able to improve the prediction performance only in leave-one-image-out cross-validation.