Abstract Mammography is a primary diagnostic measure for early detection of breast cancers that makes the patient realize the changes much earlier than they feel the changes in their breasts. The Computer Aided Diagnosis (CAD) system uses digitized mammography images and identifies the abnormalities present in breast. Deep learning methods learn the features of the image from the limited number of expert annotated data and predict the necessary objects. The performance of convolutional neural networks (CNN) in various image analysis tasks such as image detection, recognition and classification, have excelled in recent times. This paper proposes an automatic detection and classification of breast cancer lesions in mammograms by using highly accurate and advanced object detection deep learning method Faster R-CNN. The proposed CAD system uses 330 mammography images in which 121 annotated images are used for training the Faster R-CNN network. The proposed system generated a mAP (mean Average Precision) value of 0.857 for the testing set.