Large-scale chromosome rearrangements, such as fissions and fusions, are a common feature of eukaryote evolution. They can have considerable influence on the evolution of populations, yet it remains unclear exactly how rearrangements become established and eventually fix. Rearrangements could fix by genetic drift if they are weakly deleterious or neutral, or they may instead be favoured by positive natural selection. Here we compare genome assemblies of three closely related Brenthis butterfly species and characterise a complex history of fission and fusion rearrangements. An inferred demographic history of these species suggests that rearrangements became fixed in populations with large long-term effective size (N_e). However, we also find large runs of homozygosity within individual genomes and show that a model of population structure with smaller local N_e can reconcile these observations. Using a recently developed analytic framework for characterising hard selective sweeps, we find that chromosome fusions are not enriched for evidence of past sweeps compared to other regions of the genome. Nonetheless, one chromosome fusion in the B. daphne genome is associated with a valley of diversity where genealogical branch lengths are distorted, consistent with a selective sweep. Our results suggest that drift is a stronger force in these populations than suggested by overall genetic diversity, but that the fixation of strongly underdominant rearrangements remains unlikely. Additionally, although chromosome fusions do not typically exhibit signatures of selective sweeps, a single example raises the possibility that natural selection may sometimes play a role in their fixation.