In this paper, we propose a method to enhance performance of speaker verification by combining strengths of low-rank matrix recovery (LRR) based Fisher discriminant and weighed sparse representation (WSR) with auxiliary dictionary under total variability space. We use LRR based Fisher discriminant to make the training data more discriminate. Weighted sparse representation with auxiliary dictionary is applied to provide both sparsity and data locality structure, more robust and less sensitivity to outlier. Experiment results on utterances from Korean movie (“You Who Came From the Stars”) show that our proposed approach can significantly improve the performance of speaker verification and outperform the baseline sparse representation (SR)-ivector, LRR-SR-ivector and the other standard approaches in noisy environments.