|
Framework For Proactive Visualization Of Text Based Narrative Using NLP
[Full Text]
AUTHOR(S)
Arnold Gabriel Benedict, Samiksha Shukla
KEYWORDS
Conditional Batch Normalization (CBN), Natural Language Processing (NLP), Probabilistic Modelling, Part-Of-Speech (POS), Tokenization.
ABSTRACT
Language is an essential mode, not only for human communication—but also for thinking. A story is conveyed or a report of an incident is being told, humans perceive the conveyed information in the form of visual insights. The increasing advancements in the field of artificial intelligence can help with the same in machines. This paper reflects on the internalization of stories from a cognitive perspective and outlines a scalable framework for supporting the visualization of narrative text data. This paper leverages natural language processing (NLP), probabilistic modelling of discourse knowledge, information extraction of narrative components (who, where, when, what) and the narrative visualization. The graphics knowledge base storage structure has been redesigned to obviate the necessity of having a larger database for all graphics entity. With the developed framework, any user can input unrestricted natural language for the dynamic generation of animated scenes. This provides users with direct visual output in response to their natural language input. This tool can potentially impact the way humans interact with computers and expand a completely new way of understanding conversations.
REFERENCES
[1] Nicholas L. Cassimatis: Chapter 2 - Artificial Intelligence and cognitive Modelling have the same problem, 2012, pp. 11-24.
[2] Takako Aikawa, Lee Schwartz, Michel Pahud: NLP Story Maker, Microsoft Research One Microsoft Way, Redmond, 2002, pp. 1-4.
[3] Ting-Hao (Kenneth) Huang , Francis Ferraro , Nasrin Mostafazadeh, Ishan Misra , Aishwarya Agrawal , Jacob Devlin , Ross Girshick, Xiaodong He , Pushmeet Kohli , Dhruv Batra , C. Lawrence Zitnick , Devi Parikh , Lucy Vanderwende , Michel Galley , Margaret Mitchell : Vi- sual Storytelling, Microsoft Research Carnegie Mellon University, Johns Hopkins University, University of Rochester, Virginia Tech, Facebook AI Research, 2016, pp. 1-7.
[4] Amit Dubey, Frank Keller, Patrick Sturt: Probabilistic Modeling of Discourseaware Sentence Processing, University of Edinburgh, 2013, pp. 425-451.
[5] David Bau, Hendrik Strobelt, IWilliam Peebles, Jonas Wulff, Bolei Zhou, JunYan Zhu, Antonio Torralba : Semantic Photo Manipulation with a Generative Image Prior, Special Interest Group on Computer Graphics and Interactive Techniques (SIGGRAPH 2019), pp. 59:1-59:11.
[6] Taesung Park, Ming-Yu Liu, Ting-Chun Wang, Jun-Yan Zhu: Semantic Image Synthesis with Spatially Adaptive Normalization, Computer Vision and Pattern Recognition, 2019, pp. 1-19.
[7] Han Zhang, Tao Xu, Hongsheng Li, Shaoting Zhang, Xiaogang Wang, Xiaolei Huang, Dim- itris Metaxas : StackGAN: Text to Photo-realistic Image Synthesis with Stacked Generative Adversarial Networks, IEEE International Conference on Computer Vision (ICCV), 2017, pp. 1-14.
[8] Aidan Clark, Jeff, Karen Simonyan : Efficient Video Generation on Complex Datasets, 2019, pp. 1-19.
[9] A. Jahanian, L. Chai, and P. Isola. : On the "steerability" of generative adversarial networks, 2019, pp. 1-25.
[10] Guojun Yin , Bin Liu, Lu Sheng, Nenghai Yu, Xiaogang Wang, Jing Shao : Semantics Dis- entangling for Text-to-Image Generation, Computer Vision and Pattern Recognition [cs.CV], 2019, pp. 2327-2336.
[11] Xihui Liu, Guojun Yin, Jing Shao, Xiaogang Wang, Hongsheng Li : Learning to Predict Layout-to-image Conditional Convolutions for Semantic Image Synthesis, Neural Information Processing Systems (NeurIPS 2019), pp. 1-14.
[12] Harm de Vries, Florian Strub, Jérémie Mary, Hugo Larochelle, Olivier Pietquin, Aaron Courville : Modulating early visual processing by language, Neural Information Processing Systems 30 (NIPS 2017), pp. 1-14
[13] Seunghoon Hong, Dingdong Yang, Jongwook Choi, Honglak Lee : Inferring Semantic Layout for Hierarchical Text-to-Image Synthesis, Computer Vision and Pattern Recognition (CVPR 2018), pp. 7986-7994.
[14] Xihui Liu, Guojun Yin, Jing Shao, Xiaogang Wang, Hongsheng Li : Learning to Predict Layout-to-image Conditional Convolutions for Semantic Image Synthesis, Neural Information Processing Systems (NeurIPS 2019), pp. 1-14
|
