Millettiae speciosae Champ. Leguminosae (MSC), is a well-known Chinese herb traditionally used as food material and medicine for enhancing physical strength. Our preliminary study found that the aqueous extract of this herb (MSE) had an anti-fatigue effect. In this paper, we further separated MSE into total polysaccharides (MSP) and supernatant (MSS) by alcohol precipitation, and explored which fraction was active for its anti-fatigue effect. Mice were orally administered with MSP or MSS at the doses of 200, 400, and 800 mg/kg for 20 days and the anti-fatigue effect was assessed by exhaustive swimming exercise (ESE). The biochemical parameters related to fatigue after ESE and the in vitro antioxidant activity of active fraction were determined. Our results showed that MSP, instead of MSS, significantly extended the swimming time to exhaustion (p < 0.05), indicating that MSP is responsible for the anti-fatigue effect of MSE. In addition, MSP treatment increased the levels of glucose (Glu) and muscle glycogen, whereas it decreased the accumulations of blood urea nitrogen (BUN) and lactic acid (Lac). Moreover, ESE increased the levels of creatine phosphokinase (CK), lactic dehydrogenase (LDH), and malondialdehyde (MDA) but reduced superoxide dismutase (SOD) and glutathione (GSH) in plasma. In contrast, MSP inhibited all the above changes relating to fatigue. Furthermore, an in vitro antioxidant test revealed that MSP dose-dependently scavenged ·OH and DPPH free radicals. Taken together, these findings strongly suggested that MSP was able to alleviate physical fatigue by increasing energy resources and decreasing accumulation of detrimental metabolites. The antioxidant activity may crucially contribute to the observed anti-fatigue effect of MSP.