Macrophages are the most abundant stromal cells associated with the host immune system in multiple malignancies including breast cancer. With proven clinical efficacy and no noticeable adverse effects, XIAOPI formula (XPS) has been approved for breast hyperplasia treatment by the State Food and Drug Administration of China (SFDA) in 2018. The existing knowledge about the anti-breast cancer activities and mechanisms of XPS has been very limited. The present study aimed to investigate whether XPS could exert an anti-breast cancer effect by regulating tumor-associated macrophages (TAMs) in tumor microenvironment. Herein, breast cancer cells and TAMs were co-cultured using the transwell co-culture system to simulate the coexistence of them. XPS could significantly inhibit the proliferation, colony formation, breast cancer stem cells (CSCs) subpopulation, mammosphere formation abilities as well as stemness-related genes expression in both human and mouse breast cancer cells in the co-culture system. Additionally, XPS could suppress M2 phenotype polarization as well as C-X-C motif chemokine ligand 1 (CXCL1) expression and secretion of TAMs. Notably, further mechanistic explorations verified TAMs/CXCL1 as the critical target of XPS in inhibiting breast CSCs self-renewal in the co-culture system as the exogenous CXCL1 administration could abrogate the inhibitory effect of XPS on breast CSCs self-renewal. More importantly, XPS significantly inhibited mammary tumor growth, breast CSCs subpopulation, and TAMs/CXCL1 activity in mouse 4T1-Luc xenografts in vivo without any detectable side effects. Taken together, this study not only uncovers the immunomodulatory mechanism of XPS in treating breast cancer but also sheds novel insights into TAMs/CXCL1 as a potential molecular target for breast CSCs elimination.