Steering is a manifestation of quantum correlations that embodies the Einstein-Podolsky-Rosen (EPR) paradox. While there have been recent attempts to quantify steering, continuous variable systems remained elusive. We introduce a steering measure for two-mode continuous variable systems that is valid for arbitrary states. The measure is based on the violation of an optimized variance test for the EPR paradox, and admits a computable and experimentally friendly lower bound only depending on the second moments of the state, which reduces to a recently proposed quantifier of steerability by Gaussian measurements. We further show that Gaussian states are extremal with respect to our measure, minimizing it among all continuous variable states with fixed second moments. As a byproduct of our analysis, we generalize and relate well-known EPR-steering criteria. Finally an operational interpretation is provided, as the proposed measure is shown to quantify the optimal guaranteed key rate in semi-device independent quantum key distribution.