Mixed images cannot be avoided in visual tracking since the transmitted scene may be captured with specular reflections. Since few previous methods tackle this important problem, this paper proposes a novel visual tracking method using Blind Source Separation (BSS) for mixed images. Based on the framework of particle filter with compensated motion model at the prediction stage for mobile cameras, this paper improves its correction stage by weighting particles using color histograms on the mixed image and intrinsic illumination image, based on the trichromatic and opponent-process theories, respectively. Moreover, the weighting of each particle is optimized using Maximum Likelihood (ML). Experimental results show that the proposed scheme effectively improves the tracking accuracy on mixed images.