Various methods have been proposed to construct Latin hypercube designs with small correlations. Orthogonal arrays have been used to construct Latin hypercube designs with multi-dimensional stratification. To integrate these two ideas, we propose a method to construct Latin hypercube designs with both controlled correlations and multi-dimensional stratification. For numerical integration, the constructed designs not only filter out lower-dimensional variance components as effectively as ordinary orthogonal array-based Latin hypercube designs, but also filter out bilinear terms more effectively. The proposed construction method entails no iterative searches. Sampling properties of the constructed designs are derived. Examples are given to illustrate the proposed construction method and the theoretical results.