Chinese medicine has been used to treat diabetes symptoms for thousands of years. Dioscoreae rhizome or Shanyao is a Chinese medicinal herb that is routinely used in the treatment of diabetes mellitus (DM). The purpose of this study is to evaluate the evidence of the added benefits and safety of herbal formulae containing Shanyao in clinical studies and the possible mechanisms of Shanyao in the prevention and treatment of DM in experimental studies. We searched nine databases for randomized controlled trials (RCTs) that included Shanyao in the formulae in the treatment of type 2 DM. Furthermore, experimental studies on the prevention and treatment of DM by Shanyao in English- and Chinese-language databases were identified. Fifty-three moderate quality RCTs with herbal formulae containing Shanyao were identified. Results from meta-analysis indicated that Shanyao alone or formulae containing Shanyao in addition to conventional treatments could benefit people with type 2 DM in lowering blood glucose, blood lipids and reducing insulin resistance. Moreover, adverse events were significantly lower in the CHM plus conventional group than those in the conventional group. Shanyao may exert the benefit through various mechanisms including inhibition of α-glucosidase and DPP-IV activity, increase of endogenous GLP-1 and immune regulating activities. Evidence from this review suggested that there appeared to be added clinical benefits associated with the use of Shanyao for DM, whether as a food supplement or as a CHM combined with hypoglycemic agents with a good safety profile. Copyright © 2020 Sun, Di, Lu, Guo, Tang, Zhang, Xue and Fan.