Reconfigurable Deep Neural Network Techniques for Reinforcement Learning(1/3)

Recently because of the development of artificial intelligence techniques and prevailing mobile computing, incorporating a neural network computing acceleration unit becomes one of the features in the edge devices. This integrated project aims to realize the artificial intelligent computing accelerator, reconfigurable deep neural network engine, for edge devices so that the computing models can be adapted according to the different application scenarios. Both supervised learning and reinforcement learning processing elements are included. Thus, the learning capabilities can be changed and programmed to extend the possible applications. In addition, the combination of analog neuromorphic computing and digital neural processing elements offers the flexibility to alter the computation styles regarding the requirements of precision and network complexity to reduce the power consumption. This sub-project will design and develop the reconfigurable deep neural network techniques for reinforcement learning. Reinforcement learning plays an important role in recent AlphaGo games. It also offers the driving strength to enhance the self-learning capability of artificial intelligent devices. Many important techniques have been proposed and examined in Atari games. We aim to study the deep Q network algorithm and its related properties for reinforcement learning. Also, recent improvements, including double Q network, prioritized experience replay, and actor-critic techniques, will be evaluated to see the necessity and possibility to insert them into the hardware implementation. The read and write access to the experience replay memory will be studied so that the content in the replay memory of a constrained size can be utilized efficiently. A reconfigurable architecture will be designed to support applications with different models. Furthermore, the impact of finite precision effect of the deep Q network will be analyzed and discussed. The forward propagation and backward propagation are also considered in the accelerator to support the feasibility of training and inference in the edge devices. We target at designing a high-throughput reconfigurable deep neural network for reinforcement learning and integrate it in the system on chip so that a high-efficiency and low-power computing engine can be realized.