Stable solution and extremal solution for fractional $p$-Laplacian

To our knowledge, this paper is the first attempt to consider the existence issue for fractional $p$-Laplacian equation: $(-\Delta)_p^s u= \lambda f(u),\; u> 0 ~\text{in}~\Omega;\; u=0\;\text{in}~ \mathbb{R}^N\setminus\Omega$, where $p>1$, $s\in (0,1)$, $\lambda>0$ and $\Omega$ is a bounded domain with $C^{1, 1}$ boundary. We first propose a notion of stable solution, then we prove that when $f$ is of class $C^1$, nondecreasing and satisfying $f(0)>0$ and $\underset{t\to \infty}{\lim}\frac{f(t)}{t^{p-1}}=\infty$, there exists an extremal parameter $\lambda^*\in (0, \infty)$ such that a bounded minimal solution $u_\lambda \in W_0^{s,p}(\Omega)$ exists if $\lambda\in (0, \lambda^*)$, and no bounded solution exists if $\lambda>\lambda^*$. Moreover, no $W_0^{s,p}(\Omega)$ solution exists for $\lambda > \lambda^*$ if in addition $f(t)^{\frac{1}{p-1}}$ is convex. To handle our problems, we show a Kato-type inequality for fractional $p$-Laplacian. We show also $L^r$ estimates for the equation $(-\Delta)_p^su=g$ with $g\in W_0^{s, p}(\Omega)^*\cap L^q(\Omega)$ for $q \geq 1$, especially for $q \le \frac{N}{sp}$. We believe that these general results have their own interests. Finally, using the stability of minimal solutions $u_\lambda$, under the polynomial growth or convexity assumption on $f$, we show that the extremal function $u_* =\lim_{\lambda\to\lambda^*}u_\lambda \in W_0^{s,p}(\Omega)$ in all dimensions, and $u^*\in L^{\infty}(\Omega)$ in some low dimensional cases.