Precision medicine, applying knowledge of breast cancer's molecular subtypes, has improved the disease's prognosis. However, recurrence and chemoresistance are critical issues for breast cancer patients. PTPN4, a new potential therapeutic target, has not been studied sufficiently in breast cancer, and the potential role of PTPN4 in the prognosis of breast cancer patients is still unknown. In our study, data from 140 invasive breast cancer patients were retrospectively collected to identify the association between PTPN4 expression and clinical outcomes of these patients. The expressions of PTPN4 were detected by immunohistochemical analysis. Breast hyperplasia tissues showed higher expression of PTPN4. We found that PTPN4 expression was lower in breast cancer patients with relapse than in patients without relapse. Patients with an increased PTPN4 level had a significantly longer relapse-free survival and overall survival time. Decreased PTPN4 expression was an independent factor associated with relapse-free survival and overall survival, as shown by multivariate Cox regression analysis. The study found that PTPN4 is an attractive prognostic biomarker for predicting the clinical outcome and effective disease management of breast cancer patients.