NNSplit-SØREN: Supporting the model implementation of large neural networks in a programmable data plane

Embedding neural network (NN) models in the data plane is one of the very promising and attractive ways to leverage the computational power of computer network switches. This method became possible with the advent of the P4 language controlling the programmable data plane. However, most data planes today have some constraints, such as a limited set of operations and limited memory size. The computational cost of training large-scale NNs is high. In addition, while complex large-scale NN architectures are often used to improve prediction accuracy, they affect the functional performance of the data plane because of factors such as numerous input parameters and complex model design. Therefore, determining how to reduce the performance cost incurred by implementing large NN models in the data plane is a critical issue that needs to be addressed. This research proposes a technique called Neural Network Split (NNSplit) to solve the performance problems of embedding a large NN in a data plane by splitting the NN layers across multiple data planes. To support layer splitting, a new protocol called SuppORting ComplEx Computation in the Network (SØREN) is also proposed. The SØREN protocol header carries the activation value and bridges the NN layers in all switches. A multi-class classification use case of network traffic is used as the context for the experimental analysis. Experimental results show that compared to non-splitting NN architectures, NNSplit can reduce memory usage by nearly 50% and increase network traffic throughput with the cost of a 14% increase in round-trip time. In addition, when the SØREN protocol is encapsulated into data packets, the average processing time of the switch is 773 µs, which has very little impact on the processing time of the packets. Experimental results also show that the proposed NNSplit–SØREN can support large NN models on the data plane with a small performance cost.