This paper presents the maneuver of mouse pointer and performs various mouse operations such as left click, right click, double click, drag etc using gestures recognition technique. Recognizing gestures is a complex task which involves many aspects such as motion modeling, motion analysis, pattern recognition and machine learning. Keeping all the essential factors in mind a system has been created which recognizes the movement of fingers and various patterns formed by them. Color caps have been used for fingers to distinguish it from the background color such as skin color. Thus recognizing the gestures various mouse events have been performed. The application has been created on MATLAB environment with operating system as windows 7.