Rotation Symmetry Breaking Condensate in a Scalar Theory

Motivated by an analogy with the conformal factor problem in gravitational theories of the $R+R^2$-type we investigate a $d$-dimensional Euclidean field theory containing a complex scalar field with a quartic self interaction and with a nonstandard inverse propagator of the form $-p^2+p^4$. Nonconstant spin-wave configurations minimize the classical action and spontaneously break the rotation symmetry to a lower-dimensional one. In classical statistical physics this corresponds to a spontaneous formation of layers. Within the effective average action approach we determine the renormalization group flow of the dressed inverse propagator and of a family of generalized effective potentials for nonzero-momentum modes. Already in the leading order of the semiclassical expansion we find strong ``instability induced'' renormalization effects which are due to the fact that the naive vacuum (vanishing field) is unstable towards the condensation of modes with a nonzero momentum. We argue that the (quantum) ground state of our scalar model indeed leads to spontaneous breaking of rotation symmetry.