Crown and stem rust are major diseases of perennial ryegrass (Lolium perenne L.). Plant breeders and pathologists often rate rust severity in the field using the modified Cobb scale, but this method is subjective and labor intensive. A novel, open-source system using ImageJ and R was developed to quantify pustule number and area using digital images collected from spaced plants in the field. The computer-processing pipeline included development of training data for prediction of pixel identity using random forest and noise reduction spatial processing. Raters and the computer scored rust severity on plant images of varying complexity including whole-plant (WP), five-leaf (FL), and single-leaf (SL) image series. Computer accuracy was determined using the SL, while the FL series gave insight into the true value of WP severity. Rater ability was assessed using a panel of nine scientists with varying levels of disease rating experience. Results showed rater perceptions of crown rust severity were very consistent across images, but agreement on severity values for a given image were low. Rater consistency for stem rust severity was low and FL scores were not strongly correlated with WP scores (r = 0.36, P = 0.03), indicating low rater accuracy. The computer-processing pipeline was able to accurately discriminate, count, and quantify crown and stem rust pustules on leaf samples. Correlations between computer and the median rater score for crown rust were excellent (r > 0.90, P < 0.001) for all image series. Similar to raters, there was a lack of correlation between WP and FL series (r = 0.20, not significant) indicating that this technique is limited to leaf or stem samples for stem rust and not applicable to WP. However, the computer-processing pipeline shows promise in replacing visual rating of WP for crown rust.