Existence of Boundary Layers for the supercritical Lane-Emden Systems

We consider the following supercritical problem for the Lane-Emden system: \begin{equation}\label{eq00} \begin{cases} -\Delta u_1=|u_2|^{p-1}u_2\ &in\ D,\\ -\Delta u_2=|u_1|^{q-1}u_1 \ &in\ D,\\ u_1=u_2=0\ &on\ \partial D, \end{cases} \end{equation} where $D$ is a bounded smooth domain in $\mathbb{R}^N$, $N\geq4.$ What we mean by supercritical is that the exponent pair $(p,q)\in(1,\infty)\times(1,\infty)$ satisfies $\frac1{p+1}+\frac1{q+1}<\frac{N-2}N$. We prove that for some suitable domains $D\subset\mathbb{R}^N$, there exist positive solutions with layers concentrating along one or several $k$-dimensional sub-manifolds of $\partial D$ as $$\frac1{p+1}+\frac1{q+1} \rightarrow \frac{n-2}{n},\ \ \ \ \frac{n-2}{n}<\frac1{p+1}+\frac1{q+1}<\frac{N-2}N,$$ where $n:=N-k$ with $1\leq k\leq N-3$. By transforming the original problem \eqref{eq00} into a lower $n$-dimensional weighted system, we carry out the reduction framework and apply the blow-up analysis. The properties of the ground states related to the limit problem play a crucial role in this process. The corresponding exponent pair $(p_0,q_0)$, which represents the limit pair of $(p,q)$, lies on the critical hyperbola $\frac n{p_0+1}+\frac n{q_0+1}=n-2$. It is widely recognized that the range of the smaller exponent, say $p_0$, has a profound impact on the solutions, with $p_0=\frac n{n-2}$ being a threshold. It is worth emphasizing that this paper tackles the problem by considering two different ranges of $p_0$, which is contained in $p_0>\frac n{n-2}$ and $p_0<\frac n{n-2}$ respectively. The coupling mechanisms associated with these ranges are completely distinct, necessitating different treatment approaches. This represents the main challenge overcome and the novel element of this study..