1. Large body size is associated with many fitness advantages. Despite this, most species do not grow at their physiological maximum, suggesting costs to rapid growth. There are now many empirical examples of trade-offs with growth. 2. Despite the ubiquity of physiological delays, few studies have evaluated the duration over which growth costs occur. To address this question, we measured swimming ability in growth-manipulated Atlantic silversides (Menidia mendia). Fish were manipulated to grow at their maximum for two weeks and then were put on restricted rations so they grew slowly. We then compared swimming ability with fish that had always grown slowly. 3. Fast-grown fish had significantly poorer swimming ability, and continued to show a prolonged cost of this early period of rapid growth. We found that fish fully recovered normal swimming ability after ~1 month of growing slowly. Most surprisingly, the trajectory of recovery was not monotonic; performance actually decreased before it improved. 4. We conclude with a suggestion to develop a better understanding of the mechanisms linking growth to performance trade-offs. Our results suggest that reduced swimming performance following fast growth is unlikely to be completely explained by bioenergetic constraints. Additionally, there is need for more nuanced life history theory that incorporates prolonged growth costs to increase accuracy of growth rate prediction.