The major problems of recent object tracking methods are related to the inefficient detection of moving objects due to occlusions, noisy background and inconsistent body motion. This paper presents a robust method for the detection and tracking of a moving in infrared animal videos. The tracking system is based on adaptive optical flow generation, Gaussian mixture and Kalman filtering. The adaptive Gaussian model of optical flow (GMOF) is used to extract foreground and noises are removed based on the object motion. Kalman filter enables the prediction of the object position in the presence of partial occlusions, and changes the size of the animal detected automatically along the image sequence. The presented method is evaluated in various environments of unstable background because of winds, and illuminations changes. The results show that our approach is more robust to background noises and performs better than previous methods.