International Journal of Scientific & Technology Research

Home About Us Scope Editorial Board Blog/Latest News Contact Us
10th percentile
Powered by  Scopus
Scopus coverage:
Nov 2018 to May 2020


IJSTR >> Volume 9 - Issue 3, March 2020 Edition

International Journal of Scientific & Technology Research  
International Journal of Scientific & Technology Research

Website: http://www.ijstr.org

ISSN 2277-8616

Towards Building A Neural Conversation Chatbot Through Seq2Seq Model

[Full Text]



J.Prassanna, Khadar Nawas K, Christy Jackson J, Prabakaran R, Sakkaravarthi Ramanath



Chatbots, Seq2Seq, Word2vec.



Improvements in computation and processing power paved a way for Machine learning to be applied more efficiently in real-time and in a lot of applications. In which most prominent area is Natural Language Processing and Natural Language Understanding, which helps the computer to process and understands the natural language used by people. Thanks to deep learning models and architectures which made this process of making the system process and understand natural language, which makes the system more intelligent. Chatting agent’s AKA-Chatbot is one of the major use cases of Natural Language Processing and Natural Language Understanding, which can be used in different domains to engage customers and provide a response to customer’s queries. Though many chatbots use a retrieval-based model with the recent advancement of Deep Learning, we in this work use Neural Networks to train a chat model with a question and answer datasets that make models understand the patterns in it and behave intelligently. Here we build a domain-specific generative chatbot using Neural Networks to train a conversational Model which reads the pattern of data and reply answer when a new question is asked. Finally, we conclude by validating how relevant the response generated by the model to test data or test question and provide a further area of improvements to make the system more efficient and intelligent.



[1] Y. Bengio, R. Ducharme, P. Vincent, C. Jauvin, "A Neural Probabilistic Language Model", J. Machine Learning Research, vol. 3, pp. 137-1155, 2003.
[2] Y. Bengio, P. Simard and P. Frasconi, "Learning long-term dependencies with gradient descent is difficult", IEEE Transactions on Neural Networks, vol. 5, no. 2, pp. 157-166, March 1994.
[3] Sak H, Senior A W, Beaufays F, "Long short-term memory recurrent neural network architectures for large scale acoustic modeling", Proceedings of the 15th Annual Conference of the International Speech Communication Association. 2014, 338–342
[4] Ilya Sutskever , Oriol Vinyals , Quoc V. Le, "Sequence to sequence learning with neural networks", Proceedings of the 27th International Conference on Neural Information Processing Systems, p.3104-3112, December 08-13, 2014, Montreal, Canada
[5] Z. C. Lipton, "A Critical Review of Recurrent Neural Networks for Sequence Learning", The Computing Research Repository (CoRR), vol. abs/1506.00019, June 2015.
[6] S. Jean, K. Cho, R. Memisevic, and Y. Bengio, “On using very large target vocabulary for neural machine translation,” in Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, 2015, pp. 1–10.
[7] K. Iwata, "Extending the Peak Bandwidth of Parameters for Softmax Selection in Reinforcement Learning", IEEE Transactions on Neural Networks and Learning Systems, vol. 28, no. 8, pp. 1865-1877, Aug. 2017.
[8] Stroia, L. Itu, C. Niţă, L. Lazăr and C. Suciu, "GPU accelerated geometric multigrid method: Performance comparison on recent NVIDIA architectures", (2015) 19th International Conference on System Theory, Control and Computing (ICSTCC), Cheile Gradistei, 2015, pp. 175-179
[9] Chae-Gyun Lim, "A survey of temporal information extraction and language independent features," 2016 International Conference on Big Data and Smart Computing (BigComp), Hong Kong, 2016, pp. 447-449.
[10] Tomas Mikolov, Wen-tau Yih, and Geoffrey Zweig, "Linguistic regularities in continuous space word representations", HLT-NAACL (2013). Vol. 13, pages 746–751.
[11] Oriol Vinyals and Quoc V. Le. (2015), " A neural conversational model", CoRR, abs/1506.05869.
[12] Minghui Qiu, Feng-Lin Li, Siyu Wang, Xing Gao, Yan Chen, Weipeng Zhao, Haiqing Chen, Jun Huang, Wei Chu, "AliMe Chat: A Sequence to Sequence and Rerank based Chatbot Engine", Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics. (2017)