We propose a new method to enhance performance of speaker verification by investigating a novel modification of adaptive Gaussian Mixture Model (GMM) training. This model is trained using a modified Expectation Maximization (EM) algorithm, combined with a modified Maximum A Posteriori (MAP) estimation based weight factor of observation probabilities, called the observation confidence. The observation confidence is calculated based on the SNR estimation. Based on this modified adaptive GMM training algorithm, we propose to construct GMM supervectors and i-vectors, which are considered as input feature vectors for SVM. Besides, the discriminant features for speaker verification are also exploited by using non-negative matrix factorization (NMF) in the GMM-supervector and i-vector space. Experiment results on utterances from Korean drama (“You came from the stars”) show that our proposed methods significantly outperform the baseline GMM-UBM, GMM-supervector and i-vector based SVM under various noisy conditions.