In many situations, actual water markets will not allocate water resources optimally, largely because of the perceived social value of water. It is possible, however, to build optimizing models which, taking account of demand as well as supply considerations, can substitute for actual markets. Such models can assist the formation of water policies, taking into account user-supplied values and constraints. They provide powerful tools for the system-wide cost-benefit analysis of infrastructure; this is illustrated by an analysis of the need for desalination in Israel and the cost and benefits of adding a conveyance line. Further, the use of such models can facilitate cooperation in water, yielding gains that can be considerably greater than the value of the disputed water itself. This can turn what appear to be zero-sum games into win-win situations. The Middle East Water Project has built such a model for the Israeli-Jordanian-Palestinian region. We find that the value of the water in dispute in the region is very small and the possible gains from cooperation are relatively large. Analysis of the scarcity value of water is a crucial feature.