The structure of a genome can be described at its simplest by the number of chromosomes and the sex chromosome system it contains. Despite over a century of study, the evolution of genome structure on this scale remains recalcitrant to broad generalisations that can be applied across clades. To address this issue, we have assembled a dataset of 823 karyotypes from the insect group Polyneoptera. This group contains orders with a range of variations in chromosome number, and offer the opportunity to explore the possible causes of these differences. We have analysed this data using both phylogenetic and taxonomic approaches. Our analysis allows us to assess the importance of rates of evolution, phylogenetic history, sex chromosome systems, parthenogenesis, and genome size on variation in chromosome number within clades. We find that fusions play a key role in the origin of new sex chromosomes and that orders exhibit striking differences in rates of fusions, fissions, and polyploidy. Our results suggest that the difficulty in finding consistent rules that govern evolution at this scale may be due to the presence of many interacting forces that can lead to variation among groups.