In this paper, a novel manifold learning algorithm termed orthogonal nearest neighbor feature space embedding (ONNFSE) is proposed to eliminate three drawbacks of the nearest feature space embedding (NFSE) approach. The first one is an extrapolation error, a feature line passes through two far neighbor points is selected for scatter matrix calculating when the distance of a specified point to this line is small. The calculated scatter matrix could not efficiently preserve the local topological structure among samples. The incorrect selection will reduce the recognition rates. The interpolation error is similar the extrapolation one. To remedy these two problems, the nearest neighbor feature space is built in the proposed ONNFSE. The last problem should be solved is the non-orthogonal eigenvectors found by the NFSE algorithm. The modified ONNFSE algorithm generates orthogonal bases which possess the more discriminating power. Experimental results are conducted to demonstrate the effectiveness of our proposed algorithm.