Irrigated agriculture is placing increasing pressure on finite freshwater resources, especially in developing countries, where water extraction is often unregulated, un-priced and even subsidized. To shift agriculture to a more sustainable use of water without harming the food security and livelihoods of hundreds of millions of smallholders, substantial improvements of water use efficiency will be required. Here, we use detailed hydroclimatic and agricultural data to estimate the potential for the widespread adoption of efficient irrigation technologies to halt the depletion of India's groundwater resources. Even though we find substantial technical potential for reversing water table declines, we show that the impacts are highly sensitive to assumptions about farmers' water use decisions. For example, we find that widespread adoption of proven technologies that include drip and sprinkler irrigation has the potential to reduce the amount of excessive extraction of groundwater by two thirds. However, under more realistic assumptions about farmers' irrigation choices, half of these reductions are lost due to the expansion of irrigated area. Our results suggest that without the introduction of incentives for conservation, much of the potential impact of technology adoption on aquifers may be lost. The analysis provides quantitative input to the debate of incentive versus technology based water policies.