Using Deep Learning for Malware generation to bypass Malware detection mechanisms

Abstract

Machine learning (ML) has become increasingly accessible in recent years. Documentation about ML techniques and their implementation is widespread over the internet. As a result, they are increasingly used for malicious intent. The creation of malware variants that will not be detected by security measures is fastidious and difficult. The idea of this thesis is therefore to use deep learning techniques in order to automatically generate malware variants that can bypass classical detection techniques such as signature checking. The deep learning models used are encoder-decoder models that proved themselves as state-of-the art models for the task of text generation. Two main types of models have been tested in the scope of this thesis: variational auto-encoders (VAEs) and sequence-to-sequence models with attention. For each, different architectures have been implemented and tested. Convolutional and Recurrent Neural Network (RNN) VAEs are tested with different parameters and on different datasets. Different sequence-to-sequence architectures with a Bahdanau attention mechanism are also tested. The models are trained using a benign dataset since the goal is to add modifications to an initial code, it appears at this stage inappropriate to apply this specifically on malware files. This has the advantage to construct the dataset more easily since finding benign C files is a simpler task than finding enough malware files. A training dataset is also created, it is made up of variants of C code lines. Using modified lines instead of fully modified codes has the advantage that the produced database is composed of smaller sequences and should for this reason ease the learning of the deep learning models. To this end, an initial file is chosen and parsed into lines of code. A genetic algorithm is then used on each line to add random modifications to the initial line in order to create variants of this line. Those variants are then collected to form the dataset. Genetic algorithms (GAs) have already been used in other works and have shown good results in generating code variants. Hence, GAs were chosen to generate the variant datasets. Tests have been performed with the outputs of the GA. Files are reconstructed using the modified lines and tested in order to verify that the modifications added to the initial lines are effective also at the binary level of the file. To this end, the reconstructed files are compiled and the output of the compiler is used to measure the similarity between the initial code and the reconstructed code. The goal is to have a low similarity rate so that the different codes are as different as possible from their initial code. This thesis is a first step in the area of malware variant code generation using deep learning methods. The tested deep learning models seem to have a great difficulty in learning to generate such variant sequences and therefore do not lead to revolutionary predictions. However, the results allow to draw important conclusions to direct future research in the field.