The rapid replenishment of release-ready synaptic vesicles (SVs) at a limiting number of presynaptic release sites is required to sustain high-frequency neurotransmission in CNS neurons. Failure to clear release sites from previously exocytosed material has been shown to impair vesicle replenishment and, therefore, fast neurotransmission. The identity of this material and the machinery that removes it from release sites have remained enigmatic. Here we show that the endocytic scaffold protein intersectin 1 clears release sites by direct SH3 domain-mediated association with a non-canonical proline-rich segment of synaptobrevin assembled into the SNARE complex for neuroexocytosis. Acute structure-based or sustained genetic interference with SNARE complex recognition by intersectin 1 causes a rapid stimulation frequency-dependent depression of neurotransmission due to impaired replenishment of release-ready SVs. These findings identify a key molecular mechanism that underlies exo-endocytic coupling during fast neurotransmitter release at central synapses.