At present, network attack means emerge in endlessly. The detection technology of network attack must be constantly updated and developed. Based on this, the two stages of network attack detection (feature selection and traffic classification) are discussed. The improved bat algorithm (O-BA) and the improved random forest algorithm (O-RF) are proposed for optimization. Moreover, the NIS system is designed based on the Agent concept. Finally, the simulation experiment is carried out on the real data platform. The results showed that the detection precision, accuracy, recall, and F1 score of O-BA are significantly higher than those of references [17], [18], [19], and [20], while the false positive rate is the opposite (P < 0.05). The detection precision, accuracy, recall, and F1 score of O-RF algorithm are significantly higher than those of Apriori, ID3, SVM, NSA, and O-RF algorithm, while the false positive rate is significantly lower than that of Apriori, ID3, SVM, NSA, and O-RF algorithm (P < 0.05).