Brucea javanica oil (BJO), one of the main products of Brucea javanica, has been widely used in treating different kinds of malignant tumors. Quassinoids are the major category of anticancer phytochemicals of B. javanica. However, current researches on the anti-cancer effect of BJO mainly focused on oleic acid and linoleic acid, the common major components of dietary edible oils, essential and characteristic components of B. javanica like quassinoids potentially involved remained unexplored. In the current investigation, we developed an efficient HPLC method to detect brusatol, a characteristic quassinoid, and comparatively scrutinized the anti-hepatocellular carcinoma (anti-HCC) effect of BJO, brusatol-free BJO (BF-BJO), and brusatol-enriched BJO (BE-BJO) against hepatoma 22 (H22) in mice. High-performance liquid chromatography (HPLC) was utilized to identify the components in BJO. BE-BJO was extracted with 95 % ethanol. The anti-tumor effect of BJO, BF-BJO and BE-BJO was comparatively investigated, and the potential underlying mechanism was explored in H22 ascites tumor-bearing mice. The results indicated that BJO and BE-BJO significantly prolonged the survival time of H22 ascites tumor-bearing mice, while BF-BJO exhibited no obvious effect. BJO and BE-BJO exhibited pronounced anti-HCC activity by suppressing the growth of implanted hepatoma H22 in mice, including ascending weight, abdominal circumference, ascites volume and cancer cell viability, with a relatively wide margin of safety. BJO and BE-BJO significantly induced H22 cell apoptosis by upregulating the miRNA-29b gene level and p53 expression. Furthermore, BJO and BE-BJO treatment substantially downregulated Bcl-2 and mitochondrial Cytochrome C protein expression, and upregulated expression levels of Bax, Bad, cytosol Cytochrome C, caspase-3 (cleaved), caspase‑9 (cleaved), PARP and PARP (cleaved) to induce H22 cells apoptosis. Brusatol was detected in BJO and found to be one of its major active anti-HCC components, rather than fatty acids including oleic acid and linoleic acid. The anti-HCC effect of BJO and BE-BJO was intimately associated with the activation of miRNA-29b, p53-associated apoptosis and mitochondrial-related pathways. Our study gained novel insight into the material basis of BJO in the treatment of HCC, and laid a foundation for a novel specific standard for the quality evaluation of BJO and its commercial products in terms of its anti-cancer application. © 2020 The Authors