The development and progression of solid tumors such as colorectal cancer (CRC) are known to be affected by the immune system and cell types such as T cells, natural killer (NK) cells, and natural killer T (NKT) cells are emerging as interesting targets for immunotherapy and clinical biomarker research. In addition, CD3+ and CD8+ T cell distribution in tumors has shown positive prognostic value in stage I-III CRC. Recent developments in digital computational pathology support not only classical cell density based tumor characterization, but also a more comprehensive analysis of the spatial cell organization in the tumor immune microenvironment (TiME). Leveraging that methodology in the current study, we tried to address the question of how the distribution of myeloid derived suppressor cells in TiME of primary CRC affects the function and location of cytotoxic T cells. We applied multicolored immunohistochemistry to identify monocytic (CD11b+CD14+) and granulocytic (CD11b+CD15+) myeloid cell populations together with proliferating and non-proliferating cytotoxic T cells (CD8+Ki67+/-). Through automated object detection and image registration using HALO software (IndicaLabs), we applied dedicated spatial statistics to measure the extent of overlap between the areas occupied by myeloid and T cells. With this approach, we observed distinct spatial organizational patterns of immune cells in tumors obtained from 74 treatment-naive CRC patients. Detailed analysis of inter-cell distances and myeloid-T cell spatial overlap combined with integrated gene expression data allowed to stratify patients irrespective of their mismatch repair (MMR) status or consensus molecular subgroups (CMS) classification. In addition, generation of cell distance-derived gene signatures and their mapping to the TCGA data set revealed associations between spatial immune cell distribution in TiME and certain subsets of CD8+ and CD4+ T cells. The presented study sheds a new light on myeloid and T cell interactions in TiME in CRC patients. Our results show that CRC tumors present distinct distribution patterns of not only T effector cells but also tumor resident myeloid cells, thus stressing the necessity of more comprehensive characterization of TiME in order to better predict cancer prognosis. This research emphasizes the importance of a multimodal approach by combining computational pathology with its detailed spatial statistics and gene expression profiling. Finally, our study presents a novel approach to cancer patients' characterization that can potentially be used to develop new immunotherapy strategies, not based on classical biomarkers related to CRC biology.