Identification of vascular structures from medical images is integral to many clinical procedures. Most vessel segmentation techniques ignore the characteristic pulsatile motion of vessels in their formulation. In a recent effort to automatically segment vessels that are hidden under fat, we motivated the use of the magnitude of local pulsatile motion extracted from surgical endoscopic video. In this article we propose a new approach that leverages the local orientation, in addition to magnitude of motion, and demonstrate that the extended computation and utilization of motion vectors can improve the segmentation of vascular structures. We implement our approach using four alternatives to magnitude-only motion estimation by using traditional optical flow and by exploiting the monogenic signal for fast flow estimation. Our evaluations are conducted on both synthetic phantoms as well as two real ultrasound datasets showing improved segmentation results with negligible change in computational performance compared to the previous magnitude only approach.