Nowadays, the occurrence of the recording capability of video has gained popularity in the mobile devices or surveillance systems. However, retrieval of video objects is still challenging. The main aim of this research is to design and develop a Deep Long Short-Term Memory (Deep-LSTM) systembased on the video object retrieval. Initially, the key frame gets extracted from the video and the objects are detected from the extracted key frames using the nearest neighborhood algorithm. Then, the trajectory of the objects detected is trackedbased on the Deep LSTM in such a way that the location of the detected objects will be tracked in the video.Once the video objects are tracked, the object retrieval mechanism is utilized by means of multiple query specific distance measures, such as Euclidean distance, Bhattacharyya distance,Canberra distance, Jaro–Winkler distance, tanimoto similarity and Hausdorff distance. However, the effectiveness of the proposed method is evaluated based on the performance metrics, such as precision, recall, Multiple Object Tracking Precision (MOTP) and F1-score. The results achieved will be compared with that of existing works for revealing the efficiency of the proposed method.