Acupuncture is a promising treatment for relieving pain and improving lower back function in clinical practice. However, evidence from randomized clinical trials (RCTs) remains controversial. Most RCTs conclude that acupuncture procedures for chronic low back pain (CLBP) had no significant difference in efficacy and belonged to placebo. We carefully reviewed and analyzed the methodology and implementation of sham acupuncture in RCTs. Controversial evidence of acupuncture for CLBP is only a microcosm of the evaluation methodological limitation of acupuncture. Inappropriate selection of sham acupuncture controls, rigorous RCT research models, and incorrect interpretation of results may contribute to negative evidence. Evaluating and disregarding the holistic efficacy of acupuncture with an explanatory RCT model based on evaluation drugs may be unwise. Moreover, sham acupuncture is often proven to be non-inert, unreasonable, and with low fidelity. Pitfalls of the explanatory RCT model and sham acupuncture design should be avoided. Establishing a new evaluation system that is in line with the clinical characteristics of acupuncture and obtaining high-quality evidence are difficult but promising tasks.