역공학 방해 기술 자동 탐지 및 무력화 방안 연구

Abstract

DBI(dynamic binary Instrumentation)는 바이너리 파일이 실행되고 있는 중간에 분석 코드를 삽입하는 도구로, 현재 명령어, API(Application Program Interface), 메모리, 레지스트 등 다양한 정보를 추출하여 바이너리 파일을 분석할 수 있도록 한다. 분석가들은 DBI를 악성코드 분석에 이용하려 하지만, 악성코드에 적용된 역공학 방해 기술들이 DBI를 탐지하고 분석을 하지 못하도록 만든다. 결국, 악성코드를 분석하기 위해서는 역공학 방해 기술을 무력화하는 것이 필요하다.

본 논문에서는 이러한 역공학 방해 기술을 무력화시키기 위한 방안을 설명한다. 기존에 알려진 29개의 역공학 방해 기술들을 대상으로 DBI의 분석을 방해하는 기술들을 조사한 결과, 8개의 역공학 방해 기술들을 찾았다. 이 역공학 방해 기술들 각각의 무력화 방안을 찾았고, 이 기술들의 무력화 방안들을 하나의 알고리즘으로 종합하였다. 이 알고리즘이 정상적으로 동작하는지 확인하기 위해, 역공학 방해 기술들을 적용하는 7개의 상용 프로텍터로 패킹된 파일들을 대상으로 테스트를 진행하였다. 그 결과, 4개의 상용 프로텍터가 DBI의 분석을 방해하는 기술을 적용하는 것을 발견하였고, 이 상용 프로텍터로 패킹된 파일들을 대상으로 알고리즘을 적용한 DBI의 경우는 역공학 방해 기술들을 무력화시키면서 분석이 진행되는 것을 볼 수 있었다.

이와 같이, 본 논문에서 제시한 알고리즘을 적용하면 악성코드의 역공학 방해 기술들을 무력화시키면서 분석이 진행될 수 있도록 한다. 또한, 추후에 발견되는 역공학 방해 기술에 대한 무력화 방법의 참고 자료가 될 것이고, 새롭게 발견되는 역공학 방해 기술들의 무력화 방안을 알고리즘에 추가 적용이 가능할 것으로 예상된다.| DBI(Dynamic Binary Instrumentation) is a tool for inserting instrumentation code during the execution of a binary file. It extracts various information such as current instruction, API(application program interface), memory, register. Analysts are trying to use DBI for reversing malware, but the anti-reverse engineering techniques applied to malware make it impossible to analyze the malware. After all, in order to analyze the malware, it is necessary to bypass the anti-reverse engineering techniques.

In this paper, we describe how to bypass the anti-reverse engineering techniques. As a result of investigating the techniques that interfere with the DBI analysis of 29 known anti-reverse engineering techniques, we found eight anti-reverse engineering techniques. We have found a way to bypass each of these anti-reverse engineering techniques and have combined them into a single algorithm. In order to verify that the algorithm works properly, we tested the files packed with seven commercial protectors applying anti-reverse engineering techniques. As a result, we found that four commercial protectors apply the techniques that interfere with the analysis of DBI. In case of the DBI applying the bypassing algorithm to the files packed with these commercial protectors, analysis is performed while bypassing the anti-reverse engineering techniques.

In this way, applying the bypassing algorithm suggested in this paper enables the analysis to proceed while bypassing the anti-reverse engineering techniques of the malware. In addition, it will be a reference for bypassing the anti-reverse engineering techniques found for future, and it is expected that it will be possible to apply the new method of bypassing anti-reverse engineering techniques to the bypassing algorithm.; DBI(Dynamic Binary Instrumentation) is a tool for inserting instrumentation code during the execution of a binary file. It extracts various information such as current instruction, API(application program interface), memory, register. Analysts are trying to use DBI for reversing malware, but the anti-reverse engineering techniques applied to malware make it impossible to analyze the malware. After all, in order to analyze the malware, it is necessary to bypass the anti-reverse engineering techniques.

In this paper, we describe how to bypass the anti-reverse engineering techniques. As a result of investigating the techniques that interfere with the DBI analysis of 29 known anti-reverse engineering techniques, we found eight anti-reverse engineering techniques. We have found a way to bypass each of these anti-reverse engineering techniques and have combined them into a single algorithm. In order to verify that the algorithm works properly, we tested the files packed with seven commercial protectors applying anti-reverse engineering techniques. As a result, we found that four commercial protectors apply the techniques that interfere with the analysis of DBI. In case of the DBI applying the bypassing algorithm to the files packed with these commercial protectors, analysis is performed while bypassing the anti-reverse engineering techniques.

In this way, applying the bypassing algorithm suggested in this paper enables the analysis to proceed while bypassing the anti-reverse engineering techniques of the malware. In addition, it will be a reference for bypassing the anti-reverse engineering techniques found for future, and it is expected that it will be possible to apply the new method of bypassing anti-reverse engineering techniques to the bypassing algorithm.