The insulin-regulated aminopeptidase (IRAP) is a membrane-bound zinc metallopeptidase with many important regulatory functions. It has been demonstrated that inhibition of IRAP by angiotensin IV (Ang IV) and other peptides, as well as more druglike inhibitors, improves cognition in several rodent models. We recently reported a series of aryl sulfonamides as small-molecule IRAP inhibitors and a promising scaffold for pharmacological intervention. We have now expanded with a number of derivatives, report their stability in liver microsomes, and characterize the activity of the whole series in a new assay performed on recombinant human IRAP. Several compounds, such as the new fluorinated derivative 29, present submicromolar affinity and high metabolic stability. Starting from the two binding modes previously proposed for the sulfonamide scaffold, we systematically performed molecular dynamics simulations and binding affinity estimation with the linear interaction energy method for the full compound series. The significant agreement with experimental affinities suggests one of the binding modes, which was further confirmed by the excellent correlation for binding affinity differences between the selected pair of compounds obtained by rigorous free energy perturbation calculations. The new experimental data and the computationally derived structure-activity relationship of the sulfonamide series provide valuable information for further lead optimization of novel IRAP inhibitors.