On a generalization of R. Chapman's "evil determinant"

Let $p$ be an odd prime and $x$ be an indeterminate. Recently, Z.-W. Sun proposed the following conjecture: $$\det\left[x+\left(\frac{j-i}{p}\right)\right]_{0\le i,j\le \frac{p-1}{2}}=\begin{cases} (\frac{2}{p})pb_px-a_p & \mbox{if}\ p\equiv 1\pmod4, 1 & \mbox{if}\ p\equiv 3\pmod4, \end{cases}$$ where $a_p$ and $b_p$ are rational numbers related to the fundamental unit and class number of the real quadratic field $\mathbb{Q}(\sqrt{p})$. In this paper, we confirm the above conjecture of Sun based on Vsemirnov's decomposition of Chapman's "evil determinant".