6533b850fe1ef96bd12a82fe

RESEARCH PRODUCT

FCA-based knowledge representation and local generalized linear models to address relevance and diversity in diverse social images

subject

description

Abstract In social image retrieval, the main goal is to offer a relevant but also diverse result set of images to the user. To address relevance and diversity at the same time, we propose a multi-modal procedure. This approach deals with the diversification problem using a two-step procedure based on the application of Formal Concept Analysis (FCA) to organize the text content of the images, followed by a Hierarchical Agglomerative Clustering (HAC) step to find the topics addressed by the images. FCA detects the latent concepts covered by the images in the result set, organizing them according to these concepts. In the second step, clustering is carried out to group together the ones with a similar concept. To assess the relevance, we use an adaptive multi-model relevance feedback algorithm which uses the low-level visual features to estimate a relevance measurement for all the images in the dataset. Several local logistic regression models are automatically adjusted to select the best performance for each topic. Finally, the images are ranked by selecting the highest probability image at each text cluster, generating a relevant but diverse ranked list. Diverse social images 2013, 2014, 2015, 2016 and 2017 datasets from the MediaEval benchmark are used to test the performance of our approach in a real scenario. Experimental results show that the proposed joint multimedia procedure works well on multi-concept and complex collections without adjusting parameters among collections, achieving the best results with a user-based relevance feedback algorithm. Our challenge is to achieve similar results with our automatic version.