Recent advances in synchrotron imaging allow us to study the three-dimensional (3D) histology of vertebrate fossils, including microfossils (e.g. teeth and scales) of early jawed vertebrates. These microfossils can often be scanned at submicron resolution (<1 μm) because of their small size. The resulting voxel (3D pixel) stacks can be processed into virtual thin sections revealing almost every internal detail of the samples, comparable to traditional thin sections. In addition, 3D models of the internal microanatomical structures, such as embedded odontodes and vasculature, can be assembled and examined in situ. Scales of two early osteichthyans, Psarolepis romeri from the Early Devonian of China and Andreolepis hedei from the Late Silurian of Sweden, were scanned using propagation phase-contrast synchrotron X-ray microtomography (PPC-SRμCT), and 3D models of internal canal systems and buried odontodes were created from the scans. Based on these new data, we review the evolutionary origin of cosmine and its associated pore-canal system, which has been long recognized as a synapomorphy of sarcopterygians. The first odontode that appeared during growth shows almost identical morphology in the two scales, but the second odontode of the Psarolepis scale shows a distinctive morphology with several pores on the surface. It is suggested that a shift from ridge-like odontode to pore-bearing odontode was the key step in the origin of cosmine, which was then elaborated further in more-derived sarcopterygians. We perform a detailed comparison between the two scales and propose a primary homology framework to generate microanatomical characters, which can be used in the phylogenetic analysis of early osteichthyans when more 3D data become available. Our results highlight the importance of 3D data for the study of histology and ontogeny of the dermal skeleton of early jawed vertebrates, especially scales of the polyodontode type. The traditional microvertebrate collection is not only useful for biostratigraphic studies, but also preserves invaluable biological information about the growth of vertebrate hard tissues. Today, we are only beginning to understand the biological meaning of the new 3D data. The increasing availability of such data will enable, and indeed require, a complete revision of traditional palaeohistological studies on early vertebrates.