SIR-DL: AN ARCHITECTURE OF SEMANTIC-BASED IMAGE RETRIEVAL USING DEEP LEARNING TECHNIQUE AND RDF TRIPLE LANGUAGE
Author affiliations
DOI:
https://doi.org/10.15625/1813-9663/35/1/13097Keywords:
bag of visual word, deep learning, ontology, SBIR, similarity measure, similar imagesAbstract
The problem of finding and identifying semantics of images is applied in multimedia applications of many different fields such as Hospital Information System, Geographic Information System, Digital Library System, etc. In this paper, we propose the semantic-based image retrieval (SBIR) system based on the deep learning technique; this system is called as SIR-DL that generates visual semantics based on classifying image contents. At the same time we identify the semantics of similar images on Ontology, which describes semantics of visual features of images. Firstly, the color and spatial features of segmented images are we extracted and these visual feature vectors are trained on the deep neural network to obtain visual words vectors. The process of image retrieval is executed rely on semantic classification of SIR-DL according to the visual feature vector of the query image from which it produces a visual word vector. Then, we retrieve it on Ontology to provide the identities and the semantics of similar images corresponds to a similarity measure. In order to carry out SIR-DL, the algorithms and diagram of this image retrieval system are proposed after that we implement them on ImageCLEF@IAPR, which has 20,000 images. On the base of the experimental results, the effectiveness of our method is evaluated by the accuracy, precision, recall, and F-measure; these results are compared with some of works recently published on the same image dataset. It shows that SIR-DL effectively solves the problem of semantic-based image retrieval and can be used to build multimedia systems in many different fields.Metrics
References
N. R. A. Alzubi, A. Amira, "Semantic content-based image retrieval: A comprehensive study", Journal of Visual Communication and Image Representation, vol. 32, no. xx, pp. 20-54, 2017.
L. J. A.B. Spanier, D. Cohen, "A new method for the automatic retrieval of medical cases based on the radlex ontology", International Journal of Computer Assisted Radiology and Surgery, vol. 12, no. 3, pp. 471-484, 2017.
Y. Alqasrawi, "Bridging the gap between local semantic concepts and bag of visual words for natural scene image retrieval", International Journal of Sensors Wireless Communications and Control, vol. 6, no. 3, pp. 174-191, 2016.
BusinessInsider. (2018). [Online]. Available:www.businessinsider.com
L. S. C. Hernndez-Gracidas, "Markov random elds and spatial information to improve automatic image annotation", Advances in Image and Video Technology: Springer, Berlin, Heidelberg, 2007, pp. 879-892. [Online]. Available: https://link.springer.com/chapter/10.1007/
-3-540-77129-6 74
C. B. C. Kurtz, A. Depeursinge, "A semantic framework for the retrieval of similar radiological images based on medical annotations", IEEE International Conference on Image Processing, Paris, France: IEEE, 2004, pp. xx-xx. [Online]. Available: https://ieeexplore.ieee.org/document/7025454/
M. M.-y.-G. C.A. Hernndez-Gracidas, L.E. Sucar, "Improving image retrieval by using spatial relations", Multimedia Tools and Applications, vol. 62, no. 2, pp. 479-505, 2013.
P. Carbonetto. (2018). [Online]. Available: http://www.cs.ubc.ca/pcarbo/
Deloitte, "Photo sharing: trillions and rising", Deloitte Touche Tohmatsu Limited, Deloitte Global, Tech. Rep., 2016.
Deloitte(b). (2018). [Online]. Available: https://www2.deloitte.com
B. Z. Y. et al., "I2t: Image parsing to text description", Proceedings of the IEEE: IEEE, 2010, pp. 1485-1508. [Online]. Available: https://ieeexplore.ieee.org/document/5487377/
C. B. et al., "Emerging semantic-based applications", Semantic Web, Cham: Springer, 2016, pp. 39-83. [Online]. Available: https://link.springer.com/chapter/10.1007/978-3-319-16658-2 4
H. E. et al., "The segmented and annotated iapr tc-12 benchmark", Computer Vision and Image Understanding, vol. In press, no. xx, pp. xx-xx, 2009.
S. F. et al., "Image retrieval based on using hamming distance" Procedia Comp. Sci., vol. 73, no. xx, pp. 320-327, 2015.
Z. Z. V. S. G. Castanon, Y. Chen, "Efficient activity retrieval through semantic graph queries", International conference on Multimedia, Brisbane, Australia: ACM, 2015, pp. 391-400.[Online]. Available: https://dl.acm.org/citation.cfm?id=2806229
M. Grubinger, "Analysis and evaluation of visual information systems performance", School of Computer Science and Mathematics, Faculty of Health, Engineering and Science, Victoria University, Melbourne, Australia, Tech. Rep., 2007.
S. O. H. Cevikalp, M. Elmas, "Large-scale image retrieval using transductive support vector machines", Computer Vision and Image Understanding, vol. xx, no. xx, pp. 1-11, 2017.
D. R. J. Gantz, "The digital universe in 2020: Big data, bigger digital shadows, and biggest growth in the far east", IDC iView, EMC Corporation, Tech. Rep., 2012 and 2014.
S. H. P. W.-J. Z. Y. Z. J. L. J. Wan, D. Wang, "Deep learning for content-based image retrieval: A comprehensive study", International conference on Multimedia, Orlando, Florida, USA: ACM, 2014, pp. 157-166. [Online]. Available: https://dl.acm.org/citation.cfm?id=2654948
H. A. L. Deligiannidis, "Emerging Trends in Image Processing: Computer Vision", and Pattern Recognition, Graduate Texts in Mathematics. Elsevier, USA: Morgan Kaufmann, Waltham,
MA 02451, 2015.
Y. Li, "Semantic image similarity based on deep knowledge for eective image retrieval", Department of Computer Science., Hong Kong Baptist University, Tech. Rep., 2014.
E.-I. C. M.-H. Lee, S. Rho, "Ontology based user query interpretation for semantic multimedia contents retrieval", Multimedia Tools and Applications, vol. 73, no. 2, pp. 901-915, 2014.
H. S. M. Jiu, "Nonlinear deep kernel learning for image annotation", IEEE Trans. on Image Processing, vol. 26, no. 4, pp. 1820-1832, 2017.
A. T. M. Tzelepi, "Deep convolutional learning for content based image retrieval", Neurocomputing, vol. 275, no. xx, pp. 2467-2478, 2018.
L. G. P. Muneesawang, N. Zhang, "Multimedia Database Retrieval: Technology and Applications", Graduate Texts in Mathematics. Springer, New York Dordrecht London, 2014.
T. M.-T. S.-A. R. M. M. S. Jabeen, Z. Mehmood, "An effective content-based image retrieval technique for image visuals representation based on the bag-of-visual-wordsmodel", PLoS ONE,
vol. 13, no. 4, pp. 1-24, 2018.
H. S. Pandey, PriteeKhanna, "A semantics and image retrieval system for hierarchical image databases", Information Processing Management, vol. 52, no. 4, pp. 571-591, 2016.
P. T.-M. L. V. Vijayarajan, M. Dinakaran, "A generic framework for ontology based information retrieval and image retrieval in web data", Human-centric Computing and Information Sciences, vol. 6, no. 18, pp. 1-30, 2016.
J. Z.-N. C.-Y. W. X. Xie, X. Cai, "A semantic-based method for visualizing large image collections", IEEE Transactions on Visualization and Computer Graphics, IEEE Computer Society, vol. xx, no. xx, pp. 1-15, 2018.
T. X.-C. X.-K. Y. W.-Y. M. T. Z. Y. Bai, W. Yu, "Bag-of-words based deep neural network for image retrieval", International conference on Multimedia, Orlando, Florida, USA: ACM, 2014, pp. 229-232. [Online]. Available: https://dl.acm.org/citation.cfm?id=2656402
J. W.-Q. Y.-P. Y. Y. Cao, M. Long, "Deep visual-semantic hashing for cross-modal retrieval", Inter. Conf. on Knowl. Discovery and Data Mining, California, USA: ACM, 2016, pp. 1445-1454. [Online]. Available: https://dl.acm.org/citation.cfm?id=2939812
D. Z. Y. Chou, D.J. Lee, "Semantic-based brain mri image segmentation using convolutional neural network", Advances in Visual Computing, Cham: Springer, 2016, pp. 628-638. [Online].
Available: https://link.springer.com/chapter/10.1007/978-3-319-50835-1 56
W. H. Z. Zheng, H. Bu, "An approach to classify visual semantic based on visual encoding with the convolutional neural network", International Conference on Fuzzy Systems and Knowledge Discovery, Zhangjiajie, China: IEEE, 2015, pp. xx-xx. [Online]. Available:
Downloads
Published
How to Cite
Issue
Section
License
1. We hereby assign copyright of our article (the Work) in all forms of media, whether now known or hereafter developed, to the Journal of Computer Science and Cybernetics. We understand that the Journal of Computer Science and Cybernetics will act on my/our behalf to publish, reproduce, distribute and transmit the Work.2. This assignment of copyright to the Journal of Computer Science and Cybernetics is done so on the understanding that permission from the Journal of Computer Science and Cybernetics is not required for me/us to reproduce, republish or distribute copies of the Work in whole or in part. We will ensure that all such copies carry a notice of copyright ownership and reference to the original journal publication.
3. We warrant that the Work is our results and has not been published before in its current or a substantially similar form and is not under consideration for another publication, does not contain any unlawful statements and does not infringe any existing copyright.
4. We also warrant that We have obtained the necessary permission from the copyright holder/s to reproduce in the article any materials including tables, diagrams or photographs not owned by me/us.
