NNSplit-SØREN: Supporting the model implementation of large neural networks in a programmable data plane

David Chunhu Li, Muhamad Rizka Maulana, Li Der Chou

研究成果: 雜誌貢獻期刊論文同行評審

2 引文 斯高帕斯(Scopus)


Embedding neural network (NN) models in the data plane is one of the very promising and attractive ways to leverage the computational power of computer network switches. This method became possible with the advent of the P4 language controlling the programmable data plane. However, most data planes today have some constraints, such as a limited set of operations and limited memory size. The computational cost of training large-scale NNs is high. In addition, while complex large-scale NN architectures are often used to improve prediction accuracy, they affect the functional performance of the data plane because of factors such as numerous input parameters and complex model design. Therefore, determining how to reduce the performance cost incurred by implementing large NN models in the data plane is a critical issue that needs to be addressed. This research proposes a technique called Neural Network Split (NNSplit) to solve the performance problems of embedding a large NN in a data plane by splitting the NN layers across multiple data planes. To support layer splitting, a new protocol called SuppORting ComplEx Computation in the Network (SØREN) is also proposed. The SØREN protocol header carries the activation value and bridges the NN layers in all switches. A multi-class classification use case of network traffic is used as the context for the experimental analysis. Experimental results show that compared to non-splitting NN architectures, NNSplit can reduce memory usage by nearly 50% and increase network traffic throughput with the cost of a 14% increase in round-trip time. In addition, when the SØREN protocol is encapsulated into data packets, the average processing time of the switch is 773 µs, which has very little impact on the processing time of the packets. Experimental results also show that the proposed NNSplit–SØREN can support large NN models on the data plane with a small performance cost.

期刊Computer Networks
出版狀態已出版 - 2月 2023


深入研究「NNSplit-SØREN: Supporting the model implementation of large neural networks in a programmable data plane」主題。共同形成了獨特的指紋。