In tumors, cross talk between malignant and non-malignant cells (stroma) influences tumor growth, angiogenesis and metastasis. Stromal cells in tumors typically include vascular cells, fibroblasts and a heterogeneous population of inflammatory cells. Adipocytes may also be present. Adipose tissue is perhaps the least studied stromal cell “compartment” despite the fact that some tumors, particularly breast tumors, grow in close proximity to or physically interact with adipocytes. Apart from adipocytes and numerous blood vessels, adipose tissue harbors macrophages, which increase in proportion to adipose tissue mass. While circulating or bone marrow-derived macrophages play a well-defined role in tumor growth, it is less understood how resident adipose tissue-associated macrophages contribute to tumor progression. Here, we will review the role of adipose tissue in tumor growth and angiogenesis with emphasis on the specific functions of adipose tissue macrophages in these processes.