Coexistence of two or more populations by frequency-dependent selection is common in nature, and it often arises even in well-mixed experiments with microbes. If ecology is to be incorporated into models of population genetics, then it is important to represent accurately the functional form of frequency-dependent interactions. However, measuring this functional form is problematic for traditional fitness assays, which assume a constant fitness difference between competitors over the course of an assay. Here, we present a theoretical framework for measuring the functional form of frequency-dependent fitness by accounting for changes in abundance and relative fitness during a competition assay. Using two examples of ecological coexistence that arose in a long-term evolution experiment with Escherichia coli, we illustrate accurate quantification of the functional form of frequency-dependent relative fitness. Using a Monod-type model of growth dynamics, we show that two ecotypes in a typical cross-feeding interaction—such as when one bacterial population uses a byproduct generated by another—yields relative fitness that is linear with relative frequency.