Anomaly detection for remote sensing has been intensely investigated in recent years. It is not an easy task since an anomaly has distinct unknown spectral features from its neighborhood, and it usually has small size with only a few pixels. Several methods are devoted to this problem, such as the well-known RX algorithm which takes advantage of the second-order statistics. The RX algorithm assumes Gaussian noise and uses sample covariance matrix for data whitening. However, when the anomalies pixel number exceeds certain percentage or the data is ill distributed, the sample covariance matrix can not represent the background distribution. In this case, the RX algorithm will not perform well. In this paper, we perform a computer simulation to analyze the performance of the RX algorithm under different circumstances, including the number of anomaly pixels, number of anomaly types, the distance of anomaly spectrum from background, the noise distribution, etc. Later we used AVIRIS data and utilized the characteristic of principle component analysis to estimate the covariance matrix and mean of the pixels of the background. We will analyze the performance of the RX algorithm by using the estimated covariance matrix with the original version.