To assess the evidence for the efficacy of acupuncture for non-specific low back pain (NSLBP), compared with sham or placebo therapies. We searched Cochrane CENTRAL to December 2016, and conducted searches from 1980 to December 2016 in PubMed, MEDLINE and Embase. There were no regional restrictions applied. We included only randomised controlled trials of adults with NSLBP. Placebo/sham procedures were required of the control interventions. The trials were combined using meta-analysis when the data reported allowed for statistical pooling. 14 trials (2110 participants) were included in the review, and 9 were included in the meta-analysis. Immediately after the acupuncture treatment we found statistically significant differences in pain reduction between acupuncture and sham or placebo therapy (standardised mean difference (SMD) -0.40, 95% CI -0.54 to -0.25; I2 7%; 753 participants; 9 studies), but there were no differences in function (weighted mean difference (WMD) -1.05, 95% CI -3.61 to 1.52; I2 79%; 462 participants; 4 studies). At follow-up, there were significant differences in pain reduction (SMD -0.46, 95% CI -0.82 to -0.09; I2 67%), but not in function (WMD -0.98, 95%CI -3.36 to 1.40; I2 87%). We conducted subgroup analyses both immediately after treatment and at follow-up. There is moderate evidence of efficacy for acupuncture in terms of pain reduction immediately after treatment for NSLBP ((sub)acute and chronic) when compared to sham or placebo acupuncture. PROSPERO registration no. CRD42017059438.