Disentangling the multiple factors controlling species diversity is a major challenge in ecology. Island biogeography and environmental filtering are two influential theories emphasizing respectively island size and isolation, and the abiotic environment, as key drivers of species richness. However, few attempts have been made to quantify their relative importance and investigate their mechanistic basis. Here, we applied structural equation modelling, a powerful method allowing test of complex hypotheses involving multiple and indirect effects, on an island-like system of 22 French Guianan neotropical inselbergs covered with rock-savanna. We separated the effects of size (rock-savanna area), isolation (density of surrounding inselbergs), environmental filtering (rainfall, altitude) and dispersal filtering (forest-matrix openness) on the species richness of all plants and of various ecological groups (terrestrial versus epiphytic, small-scale versus large-scale dispersal species). We showed that the species richness of all plants and terrestrial species was mainly explained by the size of rock-savanna vegetation patches, with increasing richness associated with higher rock-savanna area, while inselberg isolation and forest-matrix openness had no measurable effect. This size effect was mediated by an increase in terrestrial-habitat diversity, even after accounting for increased sampling effort. The richness of epiphytic species was mainly explained by environmental filtering, with a positive effect of rainfall and altitude, but also by a positive size effect mediated by enhanced woody-plant species richness. Inselberg size and environmental filtering both explained the richness of small-scale and large-scale dispersal species, but these ecological groups responded in opposite directions to altitude and rainfall, that is positively for large-scale and negatively for small-scale dispersal species. Our study revealed both habitat diversity associated with island size and environmental filtering as major drivers of neotropical inselberg plant diversity and showed the importance of plant species growth form and dispersal ability to explain the relative importance of each driver.