The mitochondrial pyruvate carrier (MPC) is critical for cellular homeostasis, as it is required in central metabolism for transporting pyruvate from the cytosol into the mitochondrial matrix. MPC has been implicated in many diseases and is being investigated as a drug target. A few years ago, small membrane proteins, called MPC1 and MPC2 in mammals and Mpc1, Mpc2 and Mpc3 in yeast, were proposed to form large protein complexes responsible for this function. However, the MPC complexes have never been isolated and their composition, oligomeric state and functional properties have not been defined. Here, we identify the functional unit of MPC from Saccharomyces cerevisiae. In contrast to earlier hypotheses, we demonstrate that MPC is a hetero‐dimer, not a multimeric complex. When not engaged in hetero‐dimers, the yeast Mpc proteins can also form homo‐dimers that are, however, inactive. We show that the earlier described substrate transport properties and inhibitor profiles are embodied by the hetero‐dimer. This work provides a foundation for elucidating the structure of the functional complex and the mechanism of substrate transport and inhibition.