We propose a robust curve and surface estimator based on M-type estimators and penalty-based smoothing. This approach also includes an application to wavelet regression. The concept of pseudo data, a transformation of the robust additive model to the one with bounded errors, is used to derive some theoretical properties and also motivate a computational algorithm. The resulting algorithm, termed the es-algorithm, is computationally fast and provides a simple way of choosing the amount of smoothing. Moreover, it is easily described, straightforwardly implemented and can be extended to other wavelet regression settings such as irregularly spaced data and image denoising. Results from a simulation study and real data examples demonstrate the promising empirical properties of the proposed approach.