Background Many clinical studies have shown that patients with non-small cell lung carcinoma (NSCLC) can benefit from immune checkpoint inhibitor (ICI) therapy; however, PD-L1 and tumor mutation burden (TMB), which are recommended by the NCCN guidelines, are still insufficient in predicting the response to and prognosis of immunotherapy. Given the widespread use of ICIs, it is important to find biomarkers that can predict immunotherapy outcomes in NSCLC patients, and the exploration of additional effective biomarkers for ICI therapy is urgently needed. Methods A total of 33 stage II-IV NSCLC patients were included in this study. We analyzed immune markers in biopsy and surgical tissue resected from these patients before treatment with ICIs. We examined the infiltration of immune cells and expression of PD-L1 in immune cells using fluorescent multiplex immunohistochemistry (mIHC) stained with CD8/CD68/CD163/PD-L1 antibodies. Results In this cohort, we observed that the levels of CD8+ T cells, CD8+PD-L1+ T cells, and CD68+CD163+ M2 macrophages in the total region were independent prognostic factors for progression-free survival (PFS) in NSCLC patients treated with ICIs (HR=0.04, P=0.013; HR=17.70, P=0.026; and HR=17.88, P=0.011, respectively). High infiltration of CD8+ T cells and low infiltration of CD8+PD-L1+ T cells throughout the region were correlated with prolonged PFS (P=0.016 and P=0.02, respectively). No statistically significant difference was observed for CD68+CD163+ M2 macrophages. The joint parameters CD8+ high/CD8+PD-L1+ low, CD8+ high/CD68+CD163+ low and CD8+PD-L1+ low/CD68+CD163+ low predicted better PFS than other joint parameters (P<0.01, P<0.01, and P<0.001, respectively), and they also demonstrated stronger stratification than single biomarkers. The response rate of patients with high infiltration of CD8+ T cells was significantly higher than that of those with low infiltration (P<0.01), and the joint parameters CD8+/CD8+PD-L1+ and CD8+/CD68+CD163+ also demonstrated stronger stratification than single biomarkers. Conclusions This retrospective study identified the predictive value of CD8+PD-L1+ T cells, CD8+ T cells, and CD68+CD163+ M2 macrophages in NSCLC patients who received ICIs. Interestingly, our results indicate that the evaluation of joint parameters has certain significance in guiding ICI treatment in NSCLC patients.