Aquaporins and aquaglyceroporins form a family of pore proteins that facilitate the efficient and selective flux of small solutes across biological membranes. We studied the selectivity of aquaporin-1 (AQP1) and the bacterial glycerol facilitator, GlpF, for O$_2$, CO$_2$, NH$_3$, glycerol, urea, and water. Using molecular dynamics simulations, we calculated potentials of mean force for solute permeation along the aquaporin channels and compared them with the alternative pathway across the lipid bilayer. For small solutes permeating through AQP1, a remarkable anticorrelation between permeability and solute hydrophobicity was observed, whereas the opposite trend was observed for permeation through the membrane. This finding renders AQP1 a selective filter for small polar solutes, whereas GlpF was found to be highly permeable for small solutes and permeable for larger solutes. Surprisingly, not solute-channel but water-channel interactions were found to be the key determinant underlying the selectivity mechanism of aquaporins. Hence, a hydrophobic effect, together with steric restraints, determines the selectivity of aquaporins.