In this paper, we propose a tracking method via SIFT algorithm for recording the trajectory of human motion in image sequence. Instead of using a human model that present the human body to analyze motion. Only exact two feature points from the local region of a trunk, one for joints and one for limb. We calculate the similarity between two features of trajectories. The method of computing similarity is based on the "motion vector" and "angle". We can know the degree of the angle by the connect line from joint to limb in a plane which is using the core of object to be the center. The proposed method consists of two parts. The first is to track the feature points and output the file which record motion trajectory. The second part is to analyze features of trajectory and adopt DTW (Dynamic Time Warping) to calculate the score to show the similarity between two trajectories.