Matrix Realization of Gauge Theory on Discrete Group $Z_2$

We construct a $2\times 2$ matrix algebra as representation of functions on discrete group $Z_2$ and develop the gauge theory on discrete group proposed by Starz in the matrix algebra. Accordingly, we show that the non-commutative geometry model built by R.Conquereax, G.Esposito-Farese and G.Vaillant results from this approach directly. For the purpose of Physical model building, we introduce a free fermion Lagrangian on $M_4\times Z_2$ and study Yang-Mills like gauge theory.