One promising application of near-term quantum devices is to prepare trial wavefunctions using short circuits for solving different problems via variational algorithms. For this purpose, we introduce a new circuit design that combines graph-based diagonalization circuits with arbitrary single-qubit rotation gates to get Hamiltonian-based graph states ans\"atze (H-GSA). We test the accuracy of the proposed ansatz in estimating ground state energies of various molecules of size up to 12-qubits. Additionally, we compare the gate count and parameter number complexity of the proposed ansatz against previously proposed schemes and find an order magnitude reduction in gate count complexity with slight increase in the number of parameters. Our work represents a significant step towards constructing compact quantum circuits with good trainability and convergence properties and applications in solving chemistry and physics problems.