Radial fractional Laplace operators and Hessian inequalities

In this paper we deduce a formula for the fractional Laplace operator $(-\Delta)^{s}$ on radially symmetric functions useful for some applications. We give a criterion of subharmonicity associated with $(-\Delta)^{s}$, and apply it to a problem related to the Hessian inequality of Sobolev type: $$\int_{\mathbb{R}^n}|(-\Delta)^{\frac{k}{k+1}} u|^{k+1} dx \le C \int_{\mathbb{R}^n} - u \, F_k[u] \, dx,$$ where $F_k$ is the $k$-Hessian operator on $\mathbb{R}^n$, $1\le k < \frac{n}{2}$, under some restrictions on a $k$-convex function $u$. In particular, we show that the class of $u$ for which the above inequality was established in \cite{FFV} contains the extremal functions for the Hessian Sobolev inequality of X.-J. Wang \cite{W1}. This is proved using logarithmic convexity of the Gaussian ratio of hypergeometric functions which might be of independent interest.