Grain size effect on rock strength is a topic of great interest in solid earth science. From literature review, rock strength generally decreases with the increase of grain size for both silicate rock and carbonate rock in laboratory tests; however, some recent numerical simulation results conflict with those obtained in laboratory tests. To address this issue, the effect of grain size on rock strength is investigated numerically in this paper using the grain-based model by interpreting micro-cracking process in response to loading. The simulated rock strength is found to increase with increasing grain size for both homogeneous models and heterogeneous models, regardless the number of assembled disks in one mineral grain. The strength increase with increasing grain size is generally related to the number of assembled smooth-joint contacts along grain interfaces and the generation of grain boundary cracks in response to loading. The grain interfaces significantly weaken the integrity of the rock model similar to those inherent defects in real rock. As the grain size increases, fewer grain interfaces are built in the model and the strength becomes much higher. By solely changing the mineral grain size in a model, the mechanism of grain size effect as observed in laboratory tests cannot be replicated. To overcome this, a method of degradation of grain boundary strength parameters is proposed. The simulated strength using the proposed method becomes comparable with those obtained from laboratory tests. Degradation of grain boundary parameters with increasing grain size provides a plausible explanation for the grain size effect.