Understanding how the various memory components are encoded and how they interact to guide behavior requires knowledge of the underlying neural circuits. Currently, aversive olfactory memory in Drosophila is behaviorally subdivided into four discrete phases. Among these, short- and long-term memories rely, respectively, on the γ and α/β Kenyon cells (KCs), two distinct subsets of the ∼2,000 neurons in the mushroom body (MB). Whereas V2 efferent neurons retrieve memory from α/β KCs, the neurons that retrieve short-term memory are unknown. We identified a specific pair of MB efferent neurons, named M6, that retrieve memory from γ KCs. Moreover, our network analysis revealed that six discrete memory phases actually exist, three of which have been conflated in the past. At each time point, two distinct memory components separately recruit either V2 or M6 output pathways. Memory retrieval thus features a dramatic convergence from KCs to MB efferent neurons.