Ethnopharmacological relevance: Xiaoyao San (XYS) is a classic Chinese herbal formula for treatment of depression. The present study aimed to investigate the antidepressant effects of XYS in a rat model of chronic unpredictable mild stress (CUMS) and the underlying mechanisms. Materials and methods: A CUMS rat model of depression was established via 4 weeks of unpredictable stimulation. Then the rats were orally administered paroxetine and XYS for 2 weeks with continued stress. Behavioral assessments, including an open field test (OFT), sucrose preference test (SPT) and forced swim test (FST), were conducted to evaluate the antidepressant effects of XYS. The concentrations in rat plasma of tryptophan (Trp) and its metabolic products, including kynurenine (Kyn) and quinolinic acid (QUIN), were determined using high performance liquid chromatography tandem mass spectrometry with electrochemical detection (HPLC-MS/MS). The mRNA and protein levels in rat hippocampus of depression-related brain derived neurotrophic factor (BDNF), cyclic AMP response element binding protein (CREB) and nerve cell adhesion molecule (NCAM) were determined by real-time qPCR and Western blot, respectively. Enzyme Linked Immunosorbent Assay (ELISA) was used to detect the activities of indoleamine 2,3-dioxygenase (IDO) and kynurenine-3-monooxygenase (KMO) in rat plasma. Results: The results showed that a successful CUMS rat model was established through 4 weeks of continuous unpredictable stimulation, as indicated by the significant decrease in locomotor activity and increase in immobility time in the OFT, reduction in body weight and food intake etc. Compared with the normal group, the concentrations of Kyn and QUIN had significantly (p < 0.05) decreased at day 28 in the control group, but then improved after drug treatment with paroxetine and XYS. There were no obvious changes in the activities of IDO and KMO. Compared with the normal group, the mRNA of NCAM, CREB and BDNF were significantly down regulated (p < 0.001) in the control group, BDNF gene was up-regulated by paroxetine or XYS treatment, NCAM and CREB gene did not change in XYS group, protein expressions of BDNF and CREB were significantly increased, and NCAM was significantly reduced (p < 0.05). Conclusions: XYS reversed the abnormalities of the tryptophan-kynurenine metabolic pathways in depressed rats and achieved an excellent antidepressant effect. Its direct impact may be observed as changes in biological indicators in rat hippocampus tissue.