Hepatocellular carcinoma (HCC) is a lethal and aggressive human malignancy. The present study examins the anti-tumor effects of deubiquitylating enzymes (DUB) inhibitors in HCC. It is found that the inhibitor of ubiquitin specific peptidase 8 (USP8) and DUB-IN-3 shows the most effective anti-cancer responses. Targeting USP8 inhibits the proliferation of HCC and induces cell ferroptosis. In vivo xenograft and metastasis experiments indicate that inhibition of USP8 suppresses tumor growth and lung metastasis. DUB-IN-3 treatment or USP8 depletion decrease intracellular cystine levels and glutathione biosynthesis while increasing the accumulation of reactive oxygen species (ROS). Mechanistical studies reveal that USP8 stabilizes O-GlcNAc transferase (OGT) via inhibiting K48-specific poly-ubiquitination process on OGT protein at K117 site, and STE20-like kinase (SLK)-mediated S716 phosphorylation of USP8 is required for the interaction with OGT. Most importantly, OGT O-GlcNAcylates solute carrier family 7, member 11 (SLC7A11) at Ser26 in HCC cells, which is essential for SLC7A11 to import the cystine from the extracellular environment. Collectively, this study demonstrates that pharmacological inhibition or knockout of USP8 can inhibit the progression of HCC and induce ferroptosis via decreasing the stability of OGT, which imposes a great challenge that targeting of USP8 is a potential approach for HCC treatment.© 2023 The Authors. Advanced Science published by Wiley-VCH GmbH.