A schematic of a biological system, i.e., a representation of its pieces, how they are combined, and what they do, would facilitate understanding its essential organization and alteration in pathogenesis or evolution. We present a computational approach for constructing tissue schematics (TSs) from high-parameter imaging data and a biological model for interpreting them. TSs map the spatial assembly of cellular neighborhoods into tissue motifs, whose modular composition, we propose, enables the generation of complex outputs. We developed our approach in human lymphoid tissue (HLT), identifying the follicular outer zone as a potential relay between neighboring zones and a core lymphoid assembly with modifications characteristic of each HLT type. Applying the TS approach to the tumor microenvironment in human colorectal cancer identified a higher-order motif, whose mutated assembly was negatively associated with patient survival. TSs may therefore elucidate how immune architectures can be specialized and become vulnerable to reprogramming by tumors.