Litter decomposition contributes largely to global carbon (C) and nitrogen (N) cycling, and it is strongly determined by litter quality and microbial community composition in ways that are poorly understood. Here, we conducted a 2-year field litter decomposition experiment by collecting leaf and root litter of crops (from cropland), shrubs (from shrubland), and wood (from woodland) and placing samples for decomposition in woodland soil in central China to investigate the effects of litter quality and microbial community composition on C and N loss of leaf and root litter of three species under different decomposition stages. Our results showed that the leaf litter C and N losses of shrubs were significantly higher than those of crops and wood, whereas the root litter C and N losses of crops were significantly higher than those of shrubs and wood. Generally, the leaf litter C and N losses of the three species were higher on average than those of fine root litter under the whole decomposition period. For the C loss of the three species, litter lignin and phosphorus as well as initial litter quality were predominant drivers of root litter decomposition, while litter lignin, cellulose, and hemicellulose concentrations were dominant for leaf litter decomposition. For N loss, litter stoichiometry and litter quality directly governed leaf and root litter N loss, and the initial litter quality largely regulated N loss at the late decomposition stage. Unexpectedly, the effect of microbial community composition on litter C and N loss was relatively weak and only exhibited an effect on litter C and N loss during the early stage of decomposition. Thus, our results revealed the huge disparity in C and N loss of plant species and litter types at different decomposition stages, which should be considered jointly when evaluating their roles in plant-soil feedbacks under global land use change.