Uncertain motions of target caused by breath, heartbeat and body drift of patient can increase the target locating error during image guided interventions, and that may cause additional surgery trauma. The surgery navigation system with accurate motion tracking is important for improving the operation accuracy and reducing trauma. In this work, we propose an accurate and fast tracking algorithm in three-dimension (3D) ultrasound (US) sequences for US guided surgery to achieve moving object tracking. The idea of this algorithm is as follows. Firstly, feature pyramid architecture is introduced into Siamese network to extract multi-scale convolutional features. Secondly, to improve the network discriminative power and the robustness to ultrasonic noise and gain variation, we use the normalized cross correlation (NCC) to calculate the similarity between template block and search block. Thirdly, a fast NCC (FNCC) is proposed, which can perform the real-time tracking. Finally, a density peaks clustering approach is used to compensate the motion of target and further improve the tracking accuracy. The proposed algorithm is evaluated on CLUST dataset that includes 22 sets of 3D US sequences, and the mean error of 1.60 ± 0.97 mm compared with manual annotations is obtained. After comparing with other published works, the results show that our algorithm achieves the best performance. Ablation study proves that the leading results benefit from the feature pyramid architecture and FNCC. These findings show that our algorithm can improve the motion tracking accuracy in the image guided interventions. © 2021 Institute of Physics and Engineering in Medicine.