Bats are social animals and display a diverse variety of mating and social systems, with most species exhibiting some form of polygyny. Their social organization is fluid and individuals frequently switch partners and roosting sites. While harem-like social organization is observed in multiple tropical species, its importance is contested in many of them. In this study, we investigated the role of harems in the social organization of the old world fruit bat Cynopterus sphinx. Based on regular behavioural observations over a period of 20 months and genetic data from microsatellite markers, we observed that the social organization is flexible, individuals regularly shift between roosts and the social organization resembles a fission–fusion society. Behavioural and genetic analyses suggest that the harems are not strict units of social structure, and the colony does not show signatures of subdivision with harems as behavioural units. We also observed that there was no correlation between individuals with high association index and pairwise relatedness. Our findings indicate that similar to the mating system, the social organization of C. sphinx can also be categorized as a fission–fusion society, and hence the term ‘harem’ is a misnomer. We conclude that the social system of C. sphinx is flexible, with multi-male multi-female organization, and individuals tend to be loyal to a given area rather than a roost.