基於注意力和想像力機制之符合語意描述動態場景生成系統(2/2)

Project Details

Description

With the development of artificial intelligence, excellent results have beenachieved in image recognition and semantic recognition. Using generativeadversarial networks for image generation is one of the technologies that havemade great progress in recent years. Usually humans would give a semanticdescription and generate an image through the description sentence. Theresearch in this field has attracted great attention of scholars recently. There aremany related studies using specific datasets for training and optimization, suchas bird datasets and flower datasets. The goal of this project is to use thedescriptive sentences to generate natural scene images and videos. Based onuser descriptions with time and place settings, the proposed system can generatedesired dynamic scenes.For scene generation, the data of the scene is collected to train the model. Inorder to make the generated images more diverse, this project plans to add animagination module to the model and use the hidden layer information to initializethe memory unit of the recurrent neural networks. Compared with the networkarchitectures proposed by previous researchers, adding this mechanism can helpincrease the diversity of generated images.After scene generation, motion is generated to make the scene move accordingto motion directions specified by the users. For example, the proposed systemcan generate waving trees, flowing river or cloud motions. The system proposedto use conditional code to generate motion of desired directions. Also, wepropose to use hierarchical architecture and enhance the loss function in ourframework to increase the length and enhance the quality of the generatedvideos. Because image generation technology has a high influence on theconstruction of training data sets, it can help the development of image processing and identification. At the same time, the entertainment industry has ahigh demand for the generation of images and videos. Therefore, theimplementation of this project can contribute to both academic development andeconomy.
StatusFinished
Effective start/end date1/08/2231/07/23

Keywords

  • Image Generation; Artificial Intelligence; Deep Learning; Imaginative Model; Attention Model; Generative Adversarial Networks

Fingerprint

Explore the research topics touched on by this project. These labels are generated based on the underlying awards/grants. Together they form a unique fingerprint.