Tumors of various histological origins show abundant infiltration of myeloid cells from early stages of disease progression. These cells have a profound impact on antitumor immunity and influence fundamental processes that underlie malignancy, including neoangiogenesis, sustained cancer cell proliferation, metastasis and therapy resistance. For these reasons, development of therapeutic approaches to deplete or reprogram myeloid cells in cancer is an emerging field of interest. However, knowledge about the heterogeneity of myeloid cells in tumors and their variability between patients and disease stages is still limited. In this review, we summarize the most recent advances in our understanding about how the phenotype of tumor-associated macrophages, monocytes, neutrophils, myeloid-derived suppressor cells and dendritic cells is dictated by their ontogeny, activation status and localization. We also outline major open questions that will only be resolved by applying high-dimensional single-cell technologies and systems biology approaches in the analysis of the tumor microenvironment.