Glycosaminoglycans (GAGs) are a family of complex carbohydrates vital to all mammalian organisms and involved in numerous biological processes. Chondroitin and dermatan sulfate, an important class of GAGs, are linear macromolecules consisting of disaccharide building blocks of N-acetylgalactosamine and two different uronic acids. The varying degree and the site of sulfation render their characterization challenging. Here, we combine mass spectrometry with cryogenic infrared spectroscopy in the wavenumber range from 1000 to 1800 cm–1. Fingerprint spectra were recorded for a comprehensive set of disaccharides bearing all known motifs of sulfation. In addition, state-of-the-art quantum chemical calculations were performed to aid the understanding of the differences in the experimental fingerprint spectra. The results show that the degree and position of charged sulfate groups define the size of the conformational landscape in the gas phase. The detailed understanding of cryogenic infrared spectroscopy for acidic and often highly sulfated glycans may pave the way to utilize the technique in fragment-based sequencing approaches.