Sperm swimming performance, including swimming speed and the proportion of motile cells, may strongly affect fertilization success. However, little is known about how methodological factors affect in vitro measurement of these parameters. We compare the swimming performance of sperm from House Sparrows (Passer domesticus) and Spanish Sparrows (Passer hispaniolensis) in two standard suspension media, at two different dilutions, and with different degrees of cell agitation. Further, we conduct a resampling analysis to investigate sample size effects. Sperm performance was generally reduced when sperm were diluted, or when suspended in phosphate-buffered saline (PBS) rather than a medium containing additional nutrients. Sperm performance was particularly low when they were diluted more and suspended in PBS, suggesting that seminal fluid may provide compounds that enhance performance but that these are less available following dilution. Mechanical agitation of the cells by vigorous pipetting increased the proportion of motile cells. Between-male repeatability, assessed on a single sample measured in multiple conditions, was moderate and significant, suggesting that similar results may be obtained regardless of the methodology used to assess sperm motion. We found no evidence of biased results when low numbers of cells per male were used in analysis, though precision increased substantially as sample size increased from five to 20 or more cells per male. We recommend using the same suspension media and similar sperm concentrations and levels of agitation, to the greatest degree possible, and including as many individuals as possible in analyses, even when some individuals are represented by few sperm cells.