Esophageal Adenocarcinoma (EAC) develops from Barrett’s Esophagus (BE), a chronic inflammatory state that can progress through a series of transformative dysplastic states before tumor development. While molecular and genetic changes of EAC tumors have been studied, immune microenvironment changes during Barrett’s progression to EAC remain poorly understood. In this study, we identify potential immunologic changes which can occur during BE to EAC progression. RNA Sequencing (RNA-Seq) analysis on tissue samples from EAC patients undergoing surgical resection demonstrated that a subset of chemokines and cytokines, most notably IL-6 and IL-8, increased during BE progression to EAC. xCell deconvolution analysis investigating immune cell population changes demonstrated that the largest changes in expression during BE progression occurred in M2 macrophages, pro B-cells, and eosinophils. Multiplex immunohistochemical staining of tissue microarrays showed increased immune cell populations during Barrett’s progression to high grade dysplasia. In contrast, EAC tumor sections were relatively immune poor, with a rise in PD-L1 expression and loss of CD8+ T-cells. These data demonstrate the EAC microenvironment is characterized by poor cytotoxic effector cell infiltration and increased immune inhibitory signaling. These findings suggest an immune suppressive microenvironment, highlighting the need for further studies to explore immune modulatory therapy in EAC.