In this research, a framework incorporating hybrid features is proposed to improve the performance of vehicle type classification. The proposed model includes a camera response model to enhance the collected images and a Gaussian mixture model to localize the object of interest. The feature vectors are extracted from the pre-processed images using Gabor features, histogram of oriented gradients, and local optimal-oriented pattern. The hybrid set of features discriminate the classes better; further, an ant colony optimizer is used to reduce the dimension of the extracted feature vectors. Finally, deep neural network is used to classify the types of vehicles in the images. The proposed model was tested on the MIO vision traffic camera dataset and a real-world dataset consisting of videos of multiple lanes of a toll plaza. The proposed model showed an improvement in accuracy ranging from 0.28% to 8.68% in the MIO TCD dataset when compared to well-known neural network architectures like AlexNet, Inception V3, ResNet 50, VGG 19, Xception, and DenseNet.