The genomes of ray-finned fishes (Actinopterygii) are well known for their evolutionary dynamism as reflected by drastic alterations in DNA content often via regional and whole-genome duplications, differential patterns of gene silencing or loss, shifts in the insertion-to-deletion ratios of genomic segments, and major re-patternings of chromosomes via non-homologous recombination. In sharp contrast, chromosome numbers in somatic karyotypes have been highly conserved over vast evolutionary timescales – a histogram of available counts is strongly leptokurtic with more than 50% of surveyed species displaying either 48 or 50 chromosomes. Here we employ comparative phylogenetic analyses to examine the evolutionary history of alterations in fish chromosome numbers. The most parsimonious ancestral state for major actinopterygiian clades is 48 chromosomes. When interpreted in a phylogenetic context, chromosome numbers evidence many recent instances of polyploidization in various lineages but there is no clear indication of a singular polyploidization event that has been hypothesized to have immediately preceded the teleost radiation. After factoring out evident polyploidizations, a correlation between chromosome numbers and genome sizes across the Actinopterygii is marginally statistically significant (p = 0.012) but exceedingly weak (R2 = 0.0096). Overall, our phylogenetic analysis indicates a mosaic evolutionary pattern in which the forces that govern labile features of fish genomes must operate largely independently of those that operate to conserve chromosome numbers.