Recently, interest in deep learning is increasing around the world, and this technology can be applied in various fields including construction and civil engineering. In this study, a deep learning neural network model was applied to image processing required to develop automated intelligent excavators. In particular, this study conducted various experiments using video data with a lot of noise collected from real world construction sites. The major research methods and analysis include ROI (Region of Interest) extraction, input-output vector decision, prediction rule application, and experiments with a deep neural network. The results from the experiment suggest an optimal deep learning network configuration and a variety of methods to improve the recognition rate for automated intelligent excavator development. More importantly, the experimental results suggest efficient input vectors, such as flip down and flip left to right images, and a decision rule to improve recognition rates.