Parametric binomial sums involving harmonic numbers

We present explicit formulas for the following family of parametric binomial sums involving harmonic numbers for $p=0,1,2$ and $|t|\leq1$. $$\sum_{k=1}^{\infty}\frac{H_{k-1}t^k}{k^p\binom{n+k}{k}}\quad \mbox{and}\quad \sum_{k=1}^{\infty}\frac{t^k}{k^p\binom{n+k}{k}}.$$ We also generalize the following relation between the Stirling numbers of the first kind and the Riemann zeta function to polygamma function and give some applications. $$\zeta(n+1)=\sum_{k=n}^{\infty}\frac{s(k,n)}{kk!}, \quad n=1,2,3,... .$$ As examples, \begin{equation*} \zeta(3)=\frac{1}{7}\sum_{k=1}^{\infty}\frac{H_{k-1}4^k}{k^2\binom{2k}{k}},\quad \mbox{and}\quad \zeta(3)=\frac{8}{7}+\frac{1}{7}\sum_{k=1}^{\infty}\frac{H_{k-1}4^k}{k^2(2k+1)\binom{2k}{k}}, \end{equation*} which are new series representations for the Ap\'{e}ry constant $\zeta(3)$.