Detection of Different Windows PE Malware Using Machine Learning Methods

Abstract

The types and application areas of cyber attacks are increasing and diversifying. Accordingly, the effects of attacks are constantly increasing or changing every moment. Among the attacks, malware attacks also have diversified and gained a wide place in the cyber world. With the use of different techniques and methods, there are problems in detecting and preventing malware attacks. These problems cause the systems' cyber security not to be fully ensured. Due to these situations, different malware attacks are discussed in the study, and the effects of attacks on Windows security are examined. A test-bed called AyEs has been prepared. Different attacks have been carried out, such as screenshots, vnc, aimed at hijacking or corrupting the victim system. The AyEs dataset was created by listening to the system network packets obtained due to the attacks. The dataset was preprocessed and made suitable for analysis. Machine learning methods such as Naive Bayes, J48, BayesNet, IBk, AdaBoost and LogitBoost were used on the dataset to detect malware attacks. J48 and IBk methods, which were found to provide high performance as a result of the analyzes, were suggested in the study. In this way, detection systems suitable for possible attack situations against Windows systems will be implemented easily and effectively. In addition to attack detection, an active role will be assumed in determining the type of attack.