With the advantages of agility and mobility, unmanned aerial vehicles (UAVs) have been widely applied for various civil and military missions. To dynamically control and monitor UAV, it is necessary to broadcast their location information. However, flying in the aerial environment and the fixed operation location also make UAV communications more vulnerable to privacy attacks. In this paper, we present the machine learning (ML)-based attack of UAV-based wireless networks when an attacker can obtain both plaintext and ciphertext. The collected plaintext-ciphertext pairs can be used to train an ML classifier which can help decrypt the UAV messages. By simulations, we show that a simple neural network (NN) can decrypt UAV location data with high probability. Finally, we conclude the work and present a network coding based encryption scheme as our future research direction.