Ubiquitin-conjugating E2 enzymes are central to the ubiquitination cascade and have been implicated in cancer and other diseases. Despite strong interest in developing specific E2 inhibitors, the shallow and exposed active site has proven recalcitrant to targeting with reversible small-molecule inhibitors. Here, we used phage display to generate highly potent and selective ubiquitin variants (UbVs) that target the E2 backside, which is located opposite to the active site. A UbV targeting Ube2D1 did not affect charging but greatly attenuated chain elongation. Likewise, a UbV targeting the E2 variant Ube2V1 did not interfere with the charging of its partner E2 enzyme but inhibited formation of diubiquitin. In contrast, a UbV that bound to the backside of Ube2G1 impeded the generation of thioester-linked ubiquitin to the active site cysteine of Ube2G1 by the E1 enzyme. Crystal structures of UbVs in complex with three E2 proteins revealed distinctive molecular interactions in each case, but they also highlighted a common backside pocket that the UbVs used for enhanced affinity and specificity. These findings validate the E2 backside as a target for inhibition and provide structural insights to aid inhibitor design and screening efforts.