Plastic gene expression underlies phenotypic plasticity and plastically expressed genes evolve under different selection regimes compared with ubiquitously expressed genes. Social insects are well-suited models to elucidate the evolutionary dynamics of plastic genes for their genetically and environmentally induced discrete polymorphisms. Here, we study the evolution of plastically expressed genes in the ant Cardiocondyla obscurior—a species that produces two discrete male morphs in addition to the typical female polymorphism of workers and queens. Based on individual-level gene expression data from 28 early third instar larvae, we test whether the same evolutionary dynamics that pertain to plastically expressed genes in adults also pertain to genes with plastic expression during development. In order to quantify plasticity of gene expression over multiple contrasts, we develop a novel geometric measure. For genes expressed during development, we show that plasticity of expression is positively correlated with evolutionary rates. We furthermore find a strong correlation between expression plasticity and expression variation within morphs, suggesting a close link between active and passive plasticity of gene expression. Our results support the notion of relaxed selection and neutral processes as important drivers in the evolution of adaptive plasticity.