This paper presents a dynamic gesture recognition method based on the combination of the fuzzy features of the dynamic gesture track changes and the fuzzy neural network inference system. This method first classified the dynamic gestures roughly into circular gestures and linear gestures. Further, gestures were classified narrowly into up, down, left, right, clockwise, and counter-clockwise gestures. These six dynamic gestures, which are commonly used in IP-TV controlling, were introduced as the recognition goal in our dynamic gesture recognition system. The results show that this method has a good recognition performance and fault tolerance, and more applicable to real gesture-controlled human-computer interactive environment.