Ultrasonic technology is a promising tool for washing oily sludge to remove oil, however, little information has been provided on the pilot scale application of ultrasonic washing. In this study, the ultrasonic conditions were optimized based on the pilot trial of the effects of ultrasonic properties on oil removal from oily sludge. An ultrasonic power of 0.24 w/cm2 was necessary to overcome the energy threshold for oil washing at the frequency of 25 kHz. The removal rate was changed from 46.0% to 60.7% with oily sludge content from 25.0% to 42.0%, respectively. The addition of surfactants could improve oil recovery, and the optimizing content for sodium petroleum sulfonate and Span 80 was 0.3% and 0.03%, respectively. A series of pilot scale test indicated that the oil removal rate could be 82–90% by ultrasound assistant with surfactants (Span 80). Thin-layer chromatographic flame ionization detection showed that the removal rate by ultrasonic wash was dropped with the increase of composition polarity. However, under the assistance of surfactants (Span 80), all compositions could be effectively removed. This pilot study will greatly improve the application of ultrasonic technology in oily sludge treatment.