International Journal of Scientific & Technology Research

Home About Us Scope Editorial Board Blog/Latest News Contact Us
10th percentile
Powered by  Scopus
Scopus coverage:
Nov 2018 to May 2020


IJSTR >> Volume 9 - Issue 6, June 2020 Edition

International Journal of Scientific & Technology Research  
International Journal of Scientific & Technology Research

Website: http://www.ijstr.org

ISSN 2277-8616

Image Retrieval Using Features From Pre-Trained Deep CNN

[Full Text]



Vijayakumar Bhandi, Sumithra Devi K.A.



Deep convolution neural networks, Content based image retrieval, VGG16, Feature extraction, Multi-class image retrieval, Weather image processing,



Content based image retrieval (CBIR) systems use low level image representations to measure image similarity and fetch relevant images. Color, texture and shape properties are considered as important handcrafted features in traditional CBIR systems. The success of such a CBIR system depends on the choice of the handcrafted features being used. To use relevant handcrafted features, one needs to have a good knowledge of the domain where CBIR is being applied. Usage of inappropriate handcrafted features may widen the semantic gap and can to lead to poor retrieval results. Hence it is very important to extract features which are independent of prior domain understanding. In addition, it is beneficial if the features can be learnt automatically from an input image. Machine learning methods can be used for learning valuable representations from input image data. In machine learning area, Convolution neural networks (CNN) are able to create important expressive features from a given image data. Hence CNNs are well suited for image processing applications like classification, object recognition and clustering etc. Very large datasets, huge computing resources and processing time are required to train a deep CNN model effectively. There are many deep CNNs available which are pre-trained on massive datasets and distributed for public use. The knowledge learnt from these pre-trained deep CNN models can be applied to address image processing issues in new domains. VGG16 is a pre-trained 16 layer deep CNN model developed by Oxford Visual Geometry Group. In this paper, we have created a frame work to leverage VGG16 deep CNN model for extracting important features and use these features for image retrieval task. We apply this frame work for an interesting problem, to retrieve images from a weather images dataset. Results from our proposed CBIR frame work are compared with baseline CBIR which uses handcrafted features. Experimental results indicate that the features which are extracted from pre-trained deep CNN model perform better than handcrafted features when used for image retrieval application.



[1] M. R. Zare, R. N. Ainon and W. C. Seng, "Content-Based Image Retrieval for Blood Cells," 2009 Third Asia International Conference on Modelling & Simulation, Bali, 2009, pp. 332-335. doi: 10.1109/AMS.2009.103.
[2] S. Sreedevi and S. Sebastian, "Content based image retrieval based on Database revision," 2012 International Conference on Machine Vision and Image Processing (MVIP), Taipei, 2012, pp. 29-32.
[3] YuanYong Chen, "The image retrieval algorithm based on color feature," 2016 7th IEEE International Conference on Software Engineering and Service Science (ICSESS), Beijing, 2016, pp. 647-650.
[4] Z. Zhang, W. Li and B. Li, "An Improving Technique of Color Histogram in Segmentation-based Image Retrieval," 2009 Fifth International Conference on Information Assurance and Security, Xi'an, 2009, pp. 381-384. doi: 10.1109/IAS.2009.156.
[5] A. Nazir, R. Ashraf, T. Hamdani and N. Ali, "Content based image retrieval system by using HSV color histogram, discrete wavelet transforms and edge histogram descriptor," 2018 International Conference on Computing, Mathematics and Engineering Technologies (iCoMET), Sukkur, 2018, pp. 1-6.
[6] P. Chmelar and A. Benkrid, "Efficiency of HSV over RGB Gaussian Mixture Model for fire detection," 2014 24th International Conference Radioelektronika, Bratislava, 2014, pp. 1-4.
[7] M. A. Tahoun, K. A. Nagaty, T. I. El-Arief and M. A-Megeed, "A robust content-based image retrieval system using multiple features representations," Proceedings. 2005 IEEE Networking, Sensing and Control, 2005., Tucson, AZ, 2005, pp. 116-122.
[8] S. Orhan and Y. Bastanlar, "Training CNNs with image patches for object localisation," in Electronics Letters, vol. 54, no. 7, pp. 424-426, 5 4 2018.
[9] Y. Wei et al., "HCP: A Flexible CNN Framework for Multi-Label Image Classification," in IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 38, no. 9, pp. 1901-1907, 1 Sept. 2016.
[10] S. Xie and H. Hu, "Facial expression recognition with FRR-CNN," in Electronics Letters, vol. 53, no. 4, pp. 235-237, 16 2 2017. doi: 10.1049/el.2016.4328.
[11] C. Hsu and C. Lin, "CNN-Based Joint Clustering and Representation Learning with Feature Drift Compensation for Large-Scale Image Data," in IEEE Transactions on Multimedia, vol. 20, no. 2, pp. 421-429, Feb. 2018. doi: 10.1109/TMM.2017.2745702.
[12] S. Kido, Y. Hirano and N. Hashimoto, "Detection and classification of lung abnormalities by use of convolutional neural network (CNN) and regions with CNN features (R-CNN)," 2018 International Workshop on Advanced Image Technology (IWAIT), Chiang Mai, 2018, pp. 1-4.
[13] A. T. Vo, H. S. Tran and T. H. Le, "Advertisement image classification using convolutional neural network," 2017 9th International Conference on Knowledge and Systems Engineering (KSE), Hue, 2017, pp. 197-202.
[14] Vimina E.R., Poulose Jacob K, “Image Retrieval Using Local Color and Texture Features”, Zhang T. (eds) Mechanical Engineering and Technology. Advances in Intelligent and Soft Computing, vol 125. Springer, Berlin, Heidelberg, 2012
[15] D. Sherlin, D. Murugan, "A Case Study on Brain Tumor Segmentation Using Content based Imaging", International Journal of Scientific Research in Network Security and Communication, Vol.6, Issue.3, pp.1-5, 2018
[16] N. Rezazadeh, "Initialization of weights in deep belief neural network based on standard deviation of feature values in training data vectors", International Journal of Scientific Research in Computer Science and Engineering, Vol.5, Issue.4, pp.1-8, 2017
[17] Hongbo Mu, Dawei Qi, "Pattern Recognition of Wood Defects Types Based on Hu Invariant Moments", 2nd International Congress on Image and Signal Processing, 10.1109/CISP.2009.5303866, 17-19 Oct. 2009.
[18] Dongmei Han, Qigang Liu, Weiguo Fan, "A new image classification method using CNN transfer learning and web data augmentation", Expert Systems with Applications, Volume 95, Pages 43-56, April 2018.
[19] Y. Lou, G. Fu, Z. Jiang, A. Men and Y. Zhou, "PT-NET: Improve object and face detection via a pre-trained CNN model," 2017 IEEE Global Conference on Signal and Information Processing (GlobalSIP), Montreal, QC, 2017, pp. 1280-1284. doi: 10.1109/GlobalSIP.2017.8309167
[20] Dinesh Jackson Samuel R & Rajesh Kanna B, "Cybernetic microbial detection system using transfer learning", Multimedia Tools Appl, 2018.
[21] D. Marmanis, M. Datcu, T. Esch and U. Stilla, "Deep Learning Earth Observation Classification Using ImageNet Pretrained Networks," in IEEE Geoscience and Remote Sensing Letters, vol. 13, no. 1, pp. 105-109, Jan. 2016.
[22] Hussain M., Bird J.J., Faria D.R, "A Study on CNN Transfer Learning for Image Classification", UKCI 2018. Advances in Intelligent Systems and Computing, vol 840. Springer, 2018
[23] S. Akcay, M. E. Kundegorski, C. G. Willcocks and T. P. Breckon, "Using Deep Convolutional Neural Network Architectures for Object Classification and Detection Within X-Ray Baggage Security Imagery," in IEEE Transactions on Information Forensics and Security, vol. 13, no. 9, pp. 2203-2215, Sept. 2018.
[24] B. Yang, J. Cao, R. Ni and Y. Zhang, "Facial Expression Recognition Using Weighted Mixture Deep Neural Network Based on Double-Channel Facial Images," in IEEE Access, vol. 6, pp. 4630-4640, 2018.
[25] Manali Shaha, Meenakshi Pawar, "Transfer Learning for Image Classification", 2nd International Conference on Electronics, Communication and Aerospace Technology, ICECA, 2018.
[26] Alena Selimović, Blaž Meden, Peter Peer, Aleš Hladnik, "Analysis of Content-Aware Image Compression with VGG16", IEEE International Work Conference on Bioinspired Intelligence (IWOBI), 2018.
[27] J. Chang, J. Yu, T. Han, H. Chang and E. Park, "A method for classifying medical images using transfer learning: A pilot study on histopathology of breast cancer," 2017 IEEE 19th International Conference on e-Health Networking, Applications and Services (Healthcom), Dalian, 2017, pp. 1-4.
[28] L. Shao, F. Zhu and X. Li, "Transfer Learning for Visual Categorization: A Survey," in IEEE Transactions on Neural Networks and Learning Systems, vol. 26, no. 5, pp. 1019-1034, May 2015.
[29] Gopalakrishnan, Kasthurirangan & Khaitan, S.K. & Choudhary, Alok & Agrawal, Ankit. “Deep Convolutional Neural Networks with transfer learning for computer vision-based data-driven pavement distress detection”, in Construction and Building Materials. 157. 322-330, 2017, 10.1016/j.conbuildmat.2017.09.110.
[30] Ajayi, Gbeminiyi (2018), “Multi-class Weather Dataset for Image Classification”, Mendeley Data, v1
[31] Cewu Lu, Di Lin, Jiaya Jia, Chi-Keung Tang, "Two-class Weather Classification", IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2014
[32] K. F. Siddiqui, "Knowledge based weather image processing and classification," IGARSS'97. 1997 IEEE International Geoscience and Remote Sensing Symposium Proceedings. Remote Sensing - A Scientific Vision for Sustainable Development, Singapore, 1997, pp. 1944-1946 vol.4, doi: 10.1109/IGARSS.1997.609156.
[33] Lin, F., Wang, T. Metric learning for weather image classification. Multimed Tools Appl 77, 13309–13321 (2018). https://doi.org/10.1007/s11042-017-4948-7
[34] Zheng Zhang, Huadong Ma, Huiyuan Fu, Cheng Zhang, Scene-free multi-class weather classification on single images, Neurocomputing, Volume 207, 2016, Pages 365-373, ISSN 0925-2312, https://doi.org/10.1016/j.neucom.2016.05.015.
[35] Kazuhiro Otsuka, Tsutomu Horikoshi, Satoshi Suzuki, Image Sequence Retrieval for Forecasting Weather Radar Echo Pattern, MVA '98 IAPR Workshop on Machine Vision Applications, Nov. 17-19, 1998. Makuhari, Chiba, Japan [E]
[36] P. Reungjitranon and O. Chitsobhuk, "Weather Map Image Retrieval using Connected Color Region," 2008 International Symposium on Communications and Information Technologies, Lao, 2008, pp. 464-467, doi: 10.1109/ISCIT.2008.4700235 [F]
[37] S. Srinivasulu and P. Sakthivel, "Extracting spatial semantics in association rules for weather forecasting image," Trendz in Information Sciences & Computing(TISC2010), Chennai, 2010, pp. 54-57, doi: 10.1109/TISC.2010.5714608.