Recently Kernel-Based Discriminative Dictionary (KDDL) for Video Semantic Content Analysis (VSCA) has become very popular research area, particularly in Human Computer Interactions and Computer Vision decades. Nonetheless, the existing KDDL approaches based on reconstruction error classification, coupled with sparse coefficients do not fully consider discrimination, which is essential for classification performance between video samples, despite their numerous successes. In addition, the size of video samples, an important parameter in kernel-based approaches is mostly ignored. To further improve the accuracy of video semantic classification, a VSC classification approach based on Sparse Coefficient Vector and a Virtual Kernel-based Weighted KNN is proposed in this paper. In the proposed approach, a loss function that integrates reconstruction error and discrimination is put forward. The experimental results show that this method effectively improves recognition and classification accuracy for VSCA compared with some state-of-the-art baseline approaches.