Electrocardiogram (ECG) is one of the non-invasive and low-risk methods to monitor the condition of the human heart. Any abnormal pattern(s) in the ECG signal is an indicative measure of malfunctioning of the heart, termed as arrhythmia. Due to the lack of human expertise and high probability to misdiagnose, computer-aided diagnosis and analysis are preferred. In this work, we perform arrhythmia detection with an optimized neural network having piecewise linear approximation based activation function to alleviate the complex computations in the traditional activation functions. Further, we propose a self-learning method for arrhythmia detection by learning and analyzing the characteristics (period) of the ECG signal. Self-learning based approach achieves 97.28% of arrhythmia detection accuracy, and neural network with optimized activation functions achieve an arrhythmia detection accuracy of 99.56%.