Fluorescent proteins are commonly used to label cells across organisms, but the unmodified forms cannot control biological activities. Using GFP-binding proteins derived from Camelid antibodies, we co-opted GFP as a scaffold for inducing formation of biologically active complexes, developing a library of hybrid transcription factors that control gene expression only in the presence of GFP or its derivatives. The modular design allows for variation in key properties such as DNA specificity, transcriptional potency, and drug dependency. Production of GFP controlled cell-specific gene expression and facilitated functional perturbations in the mouse retina and brain. Further, retrofitting existing transgenic GFP mouse and zebrafish lines for GFP-dependent transcription enabled applications such as optogenetic probing of neural circuits. This work establishes GFP as a multifunctional scaffold and opens the door to selective manipulation of diverse GFP-labeled cells across transgenic lines. This approach may also be extended to exploit other intracellular products as cell-specific scaffolds in multicellular organisms.