A novel weighted ensemble approach for enhancing image retrieval effectiveness with deep learning models

Abstract

Content-based image retrieval (CBIR) is becoming increasingly important amid the rapid growth of image data. Traditional CBIR approaches, which rely on features such as color, shape, and texture, often face limitations in accuracy. Even when using features extracted from deep learning models, these approaches still fall short of fully meeting user expectations. To enhance retrieval effectiveness, this study introduces an ensemble approach that utilizes feature sets from multiple deep learning models. In our method, retrieved images are determined through an aggregation of recommendations from deep learning models, with each model’s vote assigned a specific weight. This weight is comprehensively evaluated based on the similarity between the recommended image and the query, the model’s reliability, and the distribution of images recommended by each model. To validate the effectiveness of this approach, we conducted experiments using VGG16, ResNet50, EfficientNetB0, DenseNet201, Swin, and Clip, pre-trained on ImageNet for feature extraction. Three model combinations, (1) VGG16, ResNet50, and EfficientNetB0, (2) ResNet50, EfficientNetB0, and DenseNet201, and (3) Swin and Clip, were explored within the proposed ensemble framework on the Oxford-17-Flowers, Caltech-101, CIFAR-10, and ISIC-2018 datasets. The suggested approach routinely outperforms individual models, according to experimental results, providing better retrieval accuracy on most datasets.