Numerous transcriptional regulators of neurogenesis have been identified in the developing and adult brain, but how neurogenic fate is programmed at the epigenetic level remains poorly defined. Here, we report that the transcription factor Pax6 directly interacts with the Brg1-containing BAF complex in adult neural progenitors. Deletion of either Brg1 or Pax6 in the subependymal zone (SEZ) causes the progeny of adult neural stem cells to convert to the ependymal lineage within the SEZ while migrating neuroblasts convert to different glial lineages en route to or in the olfactory bulb (OB). Genome-wide analyses reveal that the majority of genes downregulated in the Brg1 null SEZ and OB contain Pax6 binding sites and are also downregulated in Pax6 null SEZ and OB. Downstream of the Pax6-BAF complex, we find that Sox11, Nfib, and Pou3f4 form a transcriptional cross-regulatory network that drives neurogenesis and can convert postnatal glia into neurons. Taken together, elements of our work identify a tripartite effector network activated by Pax6-BAF that programs neuronal fate.