Biotic interactions have been rarely included in traditional species distribution models, wherein Joint Species Distribution Models (JSDMs) emerge as a feasible approach to incorporate environmental factors and interspecific interactions simultaneously, making it a powerful tool for analyzing the structure and assembly processes of biotic communities. However, the predictability and statistical robustness of JSDMs are largely unknown because of the lack of research efforts for those newly developed models. This study systematically evaluated the performances of five JSDMs in predicting the occurrence and biomass of multiple species, with a particular focus on diverse characteristics of sampling data, including type of response variables, number of sampling sites, and the number of species included in models. In general, most models yielded satisfactory performances on fitting to observed data and on the estimation of environmental effects; however, they showed less well performances in evaluating species associations, and their predictability had large variations. The JSDMs showed inconsistent performances between the goodness-of-fit and predictability in cross-validation, and the Boral model was relatively robust than others. The predictability of JSDMs was less influenced by sample sizes and substantially improved by incorporating rare species. This study contributes to an appropriate model selection and application of JSDMs.