Small molecule therapeutics can be potent tools for cancer immunotherapy. They may be devised to target the tumor associated macrophages (TAMs) and regulatory T cells (Treg), which are major immunosuppressive cells in the tumor microenvironment. The infiltration and functionalization of these cells, which essentially promote tumor development, are mediated by the hyper-activation of the Jak-STAT3 signaling pathway. Here, we demonstrated that compound 9#, a novel inhibitor of Jak2, could suppress Jak2-STAT3 signaling in macrophages (peritoneal macrophages and THP-1 cells) and direct the macrophages toward the pro-inflammatory (M1-like) phenotype. When tested in ex vivo TAM culture and in vivo tumor models, compound 9# could reverse the phenotype of TAM from M2- to M1-type by promoting IL-12 expression. Further study suggested that compound 9# also inhibited the induction of Treg both in vitro and in vivo via blockage of Jak2 signaling. Finally, compound 9# potently increased the frequency and anti-tumor activity of CD4+ and CD8+ T lymphocytes, leading to effective suppression of tumor growth. Taken together, our findings indicated that compound 9# could be a potential candidate of small molecule therapeutics for cancer immunotherapy.