The hydrogen bonding of water molecules in different solvents is studied systematically using X-ray absorption and Fourier transform infrared spectroscopy. We show that (i) at the lowest water concentration measured in benzene and acetonitrile the water molecules are mainly symmetrically bonded. The electronic structure of these water molecules differs from the one of gas or liquid water and is rather icelike. (ii) The clustering of water molecules upon increasing concentration is solvent-specific. Upon comparing the results for nonpolar benzene and polar chloroform, a preferential orientation of the water molecules around the benzene is observed. In polar acetonitrile, a shared solvation of water and acetonitrile molecules would lead to the formation of a water structure of rings and chains.