Ballistic motion of (quasi-)particles in helium systems with quantized and quasi-continuous spectrum is discussed under conditions when the mean free path is restricted by scattering on random surface inhomogeneities. The transport equation is derived for particles with arbitrary form of energy spectrum and without model assumptions on the structure of surface scattering operator. The results can be applied to quasiparticles in liquid helium systems with quadratic and linear spectra, spectrum with a gap, etc. The transport equation is relatively simple except for the case when the distance between quantized energy levels is comparable to the surface collision frequency. In three limiting cases the diffusion coefficient is calculated analytically for arbitrary correlations of surface inhomogeneities, and elsewhere – numerically for Gaussian correlations. The interwall correlation of surface inhomogeneities affects particle diffusion in a non-trivial way; sometimes, the effect of interwall correlations persists even in the quasiclassical limit.