The Src-homology-2 domain–containing phosphatase SHP2 is a critical regulator of signal transduction, being implicated in cell growth and differentiation. Activating mutations cause developmental disorders and act as oncogenic drivers in hematologic cancers. SHP2 is activated by phosphopeptide binding to the N-SH2 domain, triggering the release of N-SH2 from the catalytic PTP domain. Based on early crystallographic data, it has been widely accepted that opening of the binding cleft of N-SH2 serves as the key “allosteric switch” driving SHP2 activation. To test the putative coupling between binding cleft opening and SHP2 activation as assumed by the allosteric switch model, we critically reviewed structural data of SHP2, and we used extensive molecular dynamics (MD) simulation and free energy calculations of isolated N-SH2 in solution, SHP2 in solution, and SHP2 in a crystal environment. Our results demonstrate that the binding cleft in N-SH2 is constitutively flexible and open in solution and that a closed cleft found in certain structures is a consequence of crystal contacts. The degree of opening of the binding cleft has only a negligible effect on the free energy of SHP2 activation. Instead, SHP2 activation is greatly favored by the opening of the central β-sheet of N-SH2. We conclude that opening of the N-SH2 binding cleft is not the key allosteric switch triggering SHP2 activation.